Sie sind auf Seite 1von 36

Oracle Database Acceleration Discovery Day

Oracle on Solaris:

Tips, Tricks & Best Practices

Andreas Neuhold

Commercial Products Sales Lead Sun Microsystems

> Oracle & Solaris : #1

> Oracle

DB on

Solaris

> Oracle

DB in

Containers

> Oracle

in Container

Clusters

> Resources

Agenda

1 > Oracle DB on Solaris > Oracle DB in Containers > Oracle in Container Clusters
1 > Oracle DB on Solaris > Oracle DB in Containers > Oracle in Container Clusters

2

Source: Oracle + Sun Strategy Webcast 3
Source: Oracle + Sun Strategy Webcast 3

Source: Oracle + Sun Strategy Webcast

Source: Oracle + Sun Strategy Webcast 3

3

Number of Unique Customers Entering Tars for Mainstream Unix Platforms – 2H FY2007, 2H FY2008,

Number of Unique Customers Entering Tars for Mainstream Unix Platforms – 2H FY2007, 2H FY2008, 2H FY2009

8000 7000 6000 5000 4000 2H2007 3000 2H2008 2H2009 2000 1000 0
8000
7000
6000
5000
4000
2H2007
3000
2H2008
2H2009
2000
1000
0

Solaris SPARC

64

Solaris x64

HP-UX ItaniumHP-UX PA-RISC

5000 4000 2H2007 3000 2H2008 2H2009 2000 1000 0 Solaris SPARC 64 Solaris x64 HP-UX ItaniumHP-UX

AIX 64

4

Percentage of UNIX Customers for mainstream UNIX platforms per Service Request – 2HFY2009 27% 36%

Percentage of UNIX Customers for mainstream UNIX platforms per Service Request – 2HFY2009

27% 36% 14% 10% 13%
27%
36%
14%
10%
13%

Solaris SPARCper Service Request – 2HFY2009 27% 36% 14% 10% 13% Solaris x64 HP-UX Itanium HP-UX PA-

Solaris x64Request – 2HFY2009 27% 36% 14% 10% 13% Solaris SPARC HP-UX Itanium HP-UX PA- RISC AIX

HP-UX Itanium

HP-UX PA- RISC36% 14% 10% 13% Solaris SPARC Solaris x64 HP-UX Itanium AIX 5L 64-bit • Total Percentage

AIX 5L 64-bit13% Solaris SPARC Solaris x64 HP-UX Itanium HP-UX PA- RISC • Total Percentage of Customers per

Total Percentage of Customers per UNIX O/S

46% Solaris

27% HP-UX

27% AIX

PA- RISC AIX 5L 64-bit • Total Percentage of Customers per UNIX O/S • 46% Solaris

5

> Oracle & Solaris : #1

> Oracle

DB on

Solaris

> Oracle

DB in

Containers

> Oracle

in Container

Clusters

> Resources

Agenda

1 > Oracle DB on Solaris > Oracle DB in Containers > Oracle in Container Clusters
1 > Oracle DB on Solaris > Oracle DB in Containers > Oracle in Container Clusters

6

• Oracle DB on Solaris – based on > “Best Practices for Optimal Configuration of

Oracle DB on Solaris – based on

> “Best Practices for Optimal Configuration of Oracle Databases on Sun Hardware”

by Allan Packer and Bob Sneed

> Oracle Openworld 2009, San Francisco, CA

http://wikis.sun.com/download/attachments/128484865/OOW09 Optimal Oracle DB Perf 1_1.pdf

2009, San Francisco, CA http://wikis.sun.com/download/attachments/128484865/OOW09 Optimal Oracle DB Perf 1_1.pdf 7

7

Oracle on Solaris

Oracle on Solaris • Solaris 10 is the #1 UNIX platform for Oracle deployments • Solaris

Solaris 10 is the #1 UNIX platform for Oracle deployments

Solaris benefits from many years of performance enhancements designed to benefit Oracle

Solaris offers outstanding -

> Scalability, to hundreds of CPUs

> Observability (e.g. Dtrace)

> Virtualization (container / zones, @ no additional cost)

> File System technology (ZFS – no limits)

> Robustness, Security, Application support,

no additional cost) > File System technology (ZFS – no limits) > Robustness, Security, Application support,

8

More than 15 years of joint Solaris/Oracle DBMS Optimizations • Intimate Shared Memory (1993) /

More than 15 years of joint Solaris/Oracle DBMS Optimizations

Intimate Shared Memory (1993) / Dynamic Intimate Shared Memory (2002)

Scheduler control optimizations (1997)

64-bit memory space (1998)

/dev/poll (1998)

Reconfiguration Coordination Manager (2002)

Locality Groups (2002)

Large Page Support (2002, 2005)

FX – Fixed Priority Class (2002)

Direct IO with concurrent writes (2002)

Sun Cluster optimizations for Oracle RAC (multiple)

http://wikis.sun.com/display/Performance/Key+Sun+technologies+for+running+Oracle

optimizations for Oracle RAC (multiple) http://wikis.sun.com/display/Performance/Key+Sun+technologies+for+running+Oracle 9

9

The Dark Underbelly of DB Workloads: Data Stalls • Data stalls dominate execution time >

The Dark Underbelly of DB Workloads:

Data Stalls

Data stalls dominate execution time

> 46%-64% of execution time wasted on data stalls

> True for both OLTP and DSS database workloads

> Even more execution time wasted on instruction stalls > Especially for OLTP, where you waste a further 10%-15%

This problem applies to typical multi-core chips

> Designed with sophisticated out-of-order cores and aggressive speculation

Source: “Database Servers on Chip Multiprocessors: Limitations and Opportunities” September 2006,Carnegie Mellon University

“Database Servers on Chip Multiprocessors: Limitations and Opportunities” September 2006,Carnegie Mellon University 10

10

Everything Changes with Sun's CoolThreads CMT Systems • Data stalls drop to 13% > 76%-80%

Everything Changes with Sun's CoolThreads CMT Systems

Data stalls drop to 13%

> 76%-80% of execution time spent on usefull computation

> True for both OLTP and DSS database workloads

> Instruction stalls are negligible

> END RESULT: 70% higher throughput than traditional systems

BUT: not good for unsaturated workloads

> Single-threaded workloads perform 41% worse

> So CMT is a win, provided you keep the CPUs busy

Source: “Database Servers on Chip Multiprocessors: Limitations and Opportunities” September 2006,Carnegie Mellon University

“Database Servers on Chip Multiprocessors: Limitations and Opportunities” September 2006,Carnegie Mellon University 11

11

A Simple Analogy: Cars are Good! 12

A Simple Analogy: Cars are Good!

A Simple Analogy: Cars are Good! 12

12

Buses are good, too

Buses are good, too 13
Buses are good, too 13
Buses are good, too 13

13

Oracle Performance Expectations • Most OLTP environments run very well with CMT > Understand response

Oracle Performance Expectations

Most OLTP environments run very well with CMT

> Understand response time components

> DB CPU usually around 20% of response time

> IO, Network, and application tier makes up the rest

Bad code is a bottleneck on any server

> CPU-bound bad code bites you sooner on CMT servers

Serial Batch jobs run slower with CMT

> Check the usual suspects

> Execution Plans, Indexes, Business Logic

> Divide and Conquer

> Increase Concurrency and Parallelism

> Execution Plans, Indexes, Business Logic > Divide and Conquer > Increase Concurrency and Parallelism 14

14

Oracle on SPARC Servers – Summary • Sun's CMT Server Advantages > Low power consumption

Oracle on SPARC Servers – Summary

Sun's CMT Server Advantages

> Low power consumption

> Maximum rack computational density

> Logical Domains (LDoms) powerful and bundled (by use of Solaris 10)

Sun's M-Series Server Advantages

> Outstanding scalability

> Enterprise-class RAS features

> Flexible domaining

Solaris Containers run on both CMT and M-Series

> Secure virtual OS instances, negligible perf overhead

> Capped Containers recognized for licensing purposes

> Secure virtual OS instances, negligible perf overhead > Capped Containers recognized for licensing purposes 15

15

Best Practices Categories • Avoiding known performance bugs > Some are surprisingly high-impact >

Best Practices Categories

Avoiding known performance bugs

> Some are surprisingly high-impact

> Architecturally-dependent factors are not widely-known

Copying proven patterns, avoiding known antipatterns

> Filesystem and network choices

> “Big knob” tunables

> Platform- or application-specific settings (some not widely-known)

In general, being disciplined about performance

> Holistically, including the application code, solution architectures, operational plans, backup and recovery, business continuance

> Methods, tools, interdisciplinary and interdepartmental alignment, frameworks, metrics, processes

Oracle Performance Bugs & Patches • 10.2.0.4 is a Good Place to be, with these

Oracle Performance Bugs & Patches

10.2.0.4 is a Good Place to be, with these fixes (among others)

> Bug #6814520 - Compute efficiency of block checksum code on SPARC was very poor

> Bug #5752399 - For enabling direct I/O on certain filesytems, filesystemio_options=setall was broken

10.2.0.4 has point patches available for these

> Excessive client attach latency & high cost of session initiation

> Bug #8737425 - Excessive close() calls on new session creation

> Scalability/consolidation issue: kstat overhead excessive with many instances running

Patch #8531434 – "KSTAT CALLS BY MMNL/CJQ0 INCUR HIGH ”

> RE: Oracle NUMA features, see Metalink #759565.1

>

SYS

> Patch #8199533 - Disables NUMA features as new default

Oracle NUMA features, see Metalink #759565.1 > SYS > Patch #8199533 - Disables NUMA features as

17

Filesystem/Feature Selection • Performance Best Practice: use an unbuffered, concurrent I/O stack and consistent

Filesystem/Feature Selection

Performance Best Practice: use an unbuffered, concurrent I/O stack and consistent contiguous ondisk data placement

> Solutions with these characteristics perform similarly

> RAW

> UFS direct I/O

> QFS with samaio or qwrites

> ASM

> VxFS with ODM, QIO, or direct with 'cio' (VxFS has new 'cio' option; ODM no longer an extra fee)

> NFS with direct I/O (Of course, performance depends greatly with NFS server factors and network factors)

Each solution has its own footnotes

> See: http://blogs.sun.com/bobs

NFS server factors and network factors) • Each solution has its own footnotes > See: http://blogs.sun.com/bobs

18

Control LGWR Priority on Purpose • Run LGWR at Fixed Priority (FX) 60, even when

Control LGWR Priority on Purpose

Run LGWR at Fixed Priority (FX) 60, even when FSS is in use

> Otherwise, TS scheduling will reduce its priority when it gets busy - exactly opposite of what one would want!

Other background processes may also benefit from special scheduling, but LGWR is uniquely critical path to all transactions

This is especially important on systems with high aggregate CPU utilization or systems with slower single-thread speeds

This is especially important on systems with high aggregate CPU utilization or systems with slower single-thread

19

RAC: Use Jumbo Frames! • Jumbo frames delivers multiple benefits > Reduces GC latency >

RAC: Use Jumbo Frames!

Jumbo frames delivers multiple benefits

> Reduces GC latency

> Reduces CPU utilization, %sys, and hardware interrupts

> Avoids scaling challenges of IP packet reassembly

> Radically improves scaling of RAC on CMT CPUs

Implementation notes

> Oracle specifies dedicated interconnects for RAC; jumbo frames should not be an 'infrastructure project'

See also

> http://blogs.sun.com/glennf/entry/jumbo_frames_with_oracle_rac

an 'infrastructure project' • See also > http://blogs.sun.com/glennf/entry/jumbo_frames_with_oracle_rac 20

20

For more information

For more information Check out the Sun Performance Wiki! > For Sun Performance information generally >

Check out the Sun Performance Wiki!

> For Sun Performance information generally

> Servers, Storage, and Software

> For Oracle on Sun Performance information in particular

http://wikis.sun.com/display/performance

and Software > For Oracle on Sun Performance information in particular http://wikis.sun.com/display/performance 21

21

> Oracle & Solaris : #1

> Oracle

DB on

Solaris

> Oracle

DB in

Containers

> Oracle

in Container

Clusters

> Resources

Agenda

> Oracle DB on Solaris > Oracle DB in Containers > Oracle in Container Clusters >
> Oracle DB on Solaris > Oracle DB in Containers > Oracle in Container Clusters >

22

Whitepaper 23
Whitepaper 23

Whitepaper

Whitepaper 23

23

Solaris Containers

Solaris Containers • Provide a virtual OS environment for applications; ideal for consolidation, e.g. Oracle Dbs

Provide a virtual OS environment for applications; ideal for consolidation, e.g. Oracle Dbs

Secure execution environment with priviliges granted from the global zone

> No way of reaching out of the zone except via the network

Many controls for resource management

> CPU sets, dedicated CPUs, max CPUs

Most resources must be mapped into a container explicitely

> e.g. filesystems, devices, etc.

CPUs, max CPUs • Most resources must be mapped into a container explicitely > e.g. filesystems,

24

Solaris 10 Container • OS Virtualization > One single OS installation App Database System Appl.Srv.
Solaris 10 Container • OS Virtualization > One single OS installation App Database System Appl.Srv.
Solaris 10 Container
• OS Virtualization
> One single OS installation
App
Database
System
Appl.Srv.
> Multiple runtime environments, appear
as “regular” Solaris OS to applications
and users
Container Container
Container
• Flexible, software-defined boundaries
OS
> “Can't see outside zone”
> “Can't affect activity outside zone”
• Simple isolation
Server
> Network, disks, memory, CPU,
processes, environment
• Each Container has it's own identity
(/etc/ )
What is a Solaris 10 Container ??
It is NOT an additional OS installation. But it is a separated runtime environment, within an
existing Solaris 10 OS installation.
Analogy with Databases: One single RDBMS installation, providing multiple separated
database instances.

25

Solaris Container – Technical view (Whole-root Zone) Global Zone cpu-shares = 50 cpu-shares = 30

Solaris Container – Technical view

(Whole-root Zone)

Global Zone

Container – Technical view (Whole-root Zone) Global Zone cpu-shares = 50 cpu-shares = 30 memory_target =
Container – Technical view (Whole-root Zone) Global Zone cpu-shares = 50 cpu-shares = 30 memory_target =

cpu-shares = 50

cpu-shares = 30 memory_target = 3G

cpu-shares = 20 memory_target = 2G

= 30 memory_target = 3G cpu-shares = 20 memory_target = 2G em/initial_size_MB = 4096 abap/heap_area_total =
= 30 memory_target = 3G cpu-shares = 20 memory_target = 2G em/initial_size_MB = 4096 abap/heap_area_total =

em/initial_size_MB = 4096 abap/heap_area_total = 2048

Local Zone

Local Zone

Local Zone

 
  ERP System
  ERP System

ERP System

  ERP System

Database

Database

/

/

/

 

/

 

/etc

/etc

/etc

/etc

/lib

/lib

/lib

/lib

/opt

/opt

/opt

/opt

/platform

/platform

/platform

/platform

/usr

/usr

/usr

/usr

/var

/var

/var

/var

/u01/ora/product/

/u01/ora/product/

/usr/sap/P01

/u01/ora/fra/CRM1

/u01/ora/fra/CRM2

/sapmnt/P01

/u02/ora/data

/u02/ora/data

/oracle/P01

/dev

→ lofs mount

Solaris 10 Kernel

/sapmnt/P01 /u02/ora/data /u02/ora/data /oracle/P01 /dev → lofs mount Solaris 10 Kernel 26

26

Solaris Containers

Solaris Containers A convenient virtual platform for Oracle Solaris 10 Containers have unique features, valuable

A convenient virtual platform for Oracle

Solaris 10 Containers have unique features, valuable especially in Oracle environments !!

Flexibility

> Fast creation (takes only couple of minutes, no new OS Installation !), Reboot, Migrate, Clone.

Consolidation

> Isolate applications software stacks and services.

> "Security island" (EAL4+ certification).

> Resource Management. Limit or assure resources (CPU, Memory, IO bandwidth) for Containers. Online and dynamically.

Standardization & Administration

> Reuse one quality-assured Solaris OS implementation for the system landscape.

> Only one OS installation for multiple separated runtime environments. Thus providing not only a server consolidation but also an OS consolidation.

Built-in

> Contained in Solaris 10. No additional (3 rd party) software installation. No additional license fees or support costs.

in Solaris 10. No additional (3 r d party) software installation. No additional license fees or

27

Solaris Containers

Solaris Containers A convenient virtual platform for Oracle Part2 Solaris 10 Containers have unique features, valuable

A convenient virtual platform for Oracle Part2 Solaris 10 Containers have unique features, valuable especially in Oracle environments !!

Scalable

> Up to 8192 Container. Nearly no performance degradation. (SAP Benchmark:

1,25% for 4 Containers).

Platform Support

> Available for all Solaris 10 platforms: Intel, AMD, UltraSPARC on Sun, Dell, HP, IBM, Fujitsu/Siemens and more

Oracle Support

> Oracle does support “single instance” in Solaris 10 Containers.

> NewNew: Oracle will certify and support RAC in Solaris 10 Containers. (Status:

QA).

> Container (CPU pool or CPU cap) is accepted as Oracle license limitation.

Proven

> Many customer references

Container (CPU pool or CPU cap) is accepted as Oracle license limitation. • Proven > Many

28

Highly Available Solaris Containers • Containers can be moved between Servers • Containers can be

Highly Available Solaris Containers

Containers can be moved between Servers

Containers can be made HA using Oracle Solaris Cluster

> HA Container agent

> Blackbox approach

> Zone nodes

> Switch Applications between “static” containers

> Container Clusters

> Full virtual cluster

nodes > Switch Applications between “static” containers > Container Clusters > Full virtual cluster 29

29

> Oracle & Solaris : #1

> Oracle

DB on

Solaris

> Oracle

DB in

Containers

> Oracle

in Container

Clusters

> Resources

Agenda

> Oracle DB on Solaris > Oracle DB in Containers > Oracle in Container Clusters >
> Oracle DB on Solaris > Oracle DB in Containers > Oracle in Container Clusters >

30

Blueprint

Blueprint 31
Blueprint 31
Blueprint 31

31

Container Clusters - why • A logical step forward – a virtual cluster for a

Container Clusters - why

A logical step forward – a virtual cluster for a virtual OS

Delegation of cluster administration

Encapsulated cluster

> Global components only manageable from the global zone

Primary target application:

> Oracle Real Application Cluster

components only manageable from the global zone • Primary target application: > Oracle Real Application Cluster

32

Oracle Solaris Container Cluster Best Environment For Multi-tier Deployments Node 1 Node 2 Node 3

Oracle Solaris Container Cluster

Best Environment For Multi-tier Deployments

Node 1

Node 2

Node 3

Node 4

Finance Container Finance Finance Cluster with RAC 10g Sales Container Sales Sales Cluster with RAC
Finance
Container
Finance
Finance
Cluster with
RAC 10g
Sales
Container
Sales
Sales
Cluster with
RAC 10g
Engineering
Engineering
Engineering
Container
Engineering
Engineering
Cluster with
RAC 11g

Finance Database

Engineering Database Sales Database

Application fault isolation Security isolation Consolidation use cases

Multiple Oracle RAC databases

Test & Development databases

Multiple data center tier / multiple application

Oracle RAC databases • Test & Development databases • Multiple data center tier / multiple application

33

> Oracle & Solaris : #1

> Oracle

DB on

Solaris

> Oracle

DB in

Containers

> Oracle

in Container

Clusters

> Resources

Agenda

> Oracle DB on Solaris > Oracle DB in Containers > Oracle in Container Clusters >
> Oracle DB on Solaris > Oracle DB in Containers > Oracle in Container Clusters >

34

Resources

Cool Tools

> http://cooltools.sunsource.net/

Blueprints

Tools > http://cooltools.sunsource.net/ • Blueprints > http://wikis.sun.com/display/BluePrints/Main •

> http://wikis.sun.com/display/BluePrints/Main

CoolThreads Experts Q&A Portal

> http://www.sun.com/secure/servers/coolthreads/tnb/contact.jsp

Performance Wiki

> http://wikis.sun.com/display/performance

Blogroll

> Glenn Fawcett - http://blogs.sun.com/glennf

> Denis Sheahan - http://blogs.sun.com/deniss

> Tanel Poder - http://blog.tanelpoder.com

> Greg Rahn - http://structureddata.org

> Bob Sneed - http://blogs.sun.com/bobs (paused)

> Allan Packer - http://blogs.sun.com/allanp

> Bob Sneed - http://blogs.sun.com/bobs (paused) > Allan Packer - http://blogs.sun.com/allanp 35

35

Oracle Database Acceleration Discovery Day

Oracle on Solaris:

Tips, Tricks & Best Practices

Andreas Neuhold

andreas.neuhold@sun.com