GUOB-Revealing Explaining SPARC Supercluster

Revealing and Explaining
The Oracle SPARC SuperCluster
Eduardo Guedes
Systems Sales Consultant
Oracle Systems Brazil
Copyright 2014 Oracle and/or its affiliates. All rights reserved. | Oracle Confidential Internal/Restricted/Highly Restricted
Safe Harbor Statement

The following is intended to outline our general product direction. It is intended for
information purposes only, and may not be incorporated into any contract. It is not a
commitment to deliver any material, code, or functionality, and should not be relied upon
in making purchasing decisions. The development, release, and timing of any features or
functionality described for Oracles products remains at the sole discretion of Oracle.
Agenda
1
Introduction
The Engineered Systems Evolution
SuperCluster Overview
Virtualization on Oracle SuperCluster
Oracle T5-8 SuperCluster Configuration Options
A SuperCluster Configuration Example
About me:
Over 16 years of IT experience including 13 + years as a Certified Oracle
DBA, Administrating Oracle Database on several platforms like Linux,
Solaris, AIX, Exadata and SuperCluster.
Experienced on Oracle Engineered Systems focusing on Exadata and SPARC
SuperCluster
Oracle employee since 2010 and worked under Global Solution Center, ACS
and Systems Team as Database Specialist on Oracle HW
Sun Certified System Administrator for Solaris 8 and 9, OCA/OCP Solaris 11,
OCP Oracle Database versions 8, 8i, 9i, 10g and 11g, OCM 11g.
Detailed profile on
http://education.oracle.com/education/otn/EGuedes.htm
Engineered Systems Evolution

Oracle Engineered Systems Timeline
2008
- Exadata
V1
2009
2010
- Exadata V2 - Exadata
X2
- Exalogic
X2
2011
2012
- ODA V1
- Exadata
- T4-4
X3
SuperCluster - Exalogic
X3
- Exalytics
- Big Data
Appliance
2013
Exadata X4 Exalogic X4
ODA X3-2
ODA X4-2
(End of 2013)
Big Data
Appliance X4-2
Exalytics T5-8
T5-8
SuperCluster
M6
SuperCluster
2014
Virtual Compute
Appliance X4-2
Oracle Database
Backup Logging
Recovery
Appliance
Oracle Engineered Systems
Big Data Exalogic Virtual

Exalytics
Appliance Elastic Compute
Cloud Appliance
Database
Appliance
SuperCluster
Copyright 2014 Oracle and/or its affiliates. All rights reserved. |
Exadata
Backup,
Logging
and
Recovery
Appliance
SuperCluster Overview
SPARC Processors (T5 or M6) for compute of application domains
Solaris Operating System (Solaris 10 or Solaris 11)
Infiniband Network, 10Gb Network and FC (Available for configuration)
Multi-Purpose Storage for application, stage, filesystems and databases

Exadata Storage for high performance database
Compatible with Exalogic technology
Virtualization technologies (PDom, LDom and Zones)
Redundant and prepared for high-availability requirements.
Oracle T5-8 SuperCluster Components

Full Rack
- 2 x SPARC T5-8 compute nodes
Exadata Storage Servers

InfiniBand Switches
Oracle ZFS Storage ZS3-ES array
8 x T5 Processors @ 3.6 GHz
2 TB of memory (128 x 16GB)
8 x 1.2 TB internal drive (SAS)
8 x Sun Dual 10 GbE SFP+ PCIe 2.0 Low

Profile Adapter (2 port)
8 x Sun Quad Data Rate (QDR) InfiniBand PCIe

Low Profile HCA (2 port)
- 8 x Exadata Storage Servers X4-2

- Oracle ZFS Storage ZS3-ES array
SPARC T5-8 Compute Nodes
- 3 x Sun Data Center InfiniBand Switch 36

- 1 x GbE Management Switch
SuperCluster M6-32 Components

Solution Components:
SPARC M6-32
From 16 to 32 M6 processors and from 8 to 32TB of Memory
Storage Rack
9 * X4-2 Exadata Storage Servers
Additional capacity can be added with Exadata Storage
Expansion racks
ZS3-ES Storage Appliance
80TB raw capacity
3 x 36-port Infiniband Switches
GbE Management Switch

Virtualization is supported without costs
SPARC M6-32 -> PDom, LDom and Zones
SPARC T5-8 -> LDom and Zones
Servers can be separated in Domains (Physical or

Logical Domains) for application and database
Database domains must run on Solaris 11 and
access the Exadata Storage
Application Domains access other Storages and
can be used to applications, non-Oracle
databases and database versions < 11.2

For application domains, in order to use Exalogic, the
oparating Systems must be Solaris 11
Application Domains can run on Solaris 10 or Solaris 11
In order to isolate servers, both database and application
domains can run over Zones
If necessary, other Solaris versions can be used throught
Solaris Branded Zones
The optimized configuration in order to avoid
performance issues must be choosen (One option of
deployment configuration must be used)

Zones also have rules, so that, the optimized
configuration protects from any kind of performance
issues related to misuse/misconfiguration.
When using Zones, some cores are reserved on global

Zones and at least one core must be given to the nonglogal Zones
At Least 4 cores must be reserved when using Giant, Large or
Medium Domains (Domains configuration will be explained later)
At Least 2 cores must be reserved when using Small Domains
(Domains configuration will be explained later)
Zones and LDOMs are considered hard partitioning for

lecensing

What should you think when using Zones on SSC:
Plan and create ISCSI LUNs for non-Glogal Zones
Using anets simplifies the management of the network interfaces
Plan to use processor sets and resource pools in order to control the
available core/threads
Capped-memory is not supported on SuperCluster Zones
If using Zones on Database Domains with Zones and there are Zones
created for Databases, do not use the Global Zone for Databases
When using RAC on Zones, plan to create Zones with a similar quantity of
Cores/Threads in order to have a good balance of the workload

What should you think when using Zones on SSC:
If using RAC databases on Zones, create zones on Database Domains from different
T5-8 Servers.
Calculate the right size for the Exadata Grid Disks in order to not waste Exadata
Storage Space
When configuring the network interfaces for Zones, think that you should configure
the Infiniband Partition ID on those in order to communicate with Exadata Storage
or ZFS Storage

Layer 1 Virtualization
Oracle T5-8 SuperCluster
Domains with zero performance

overhead
Any mix of Database Domains and
Application Domains
Oracle Solaris 11 or Oracle Solaris
10 for Application Domains
Layer 2 Virtualization
SPARC T5-8 Compute Node

DB Domain 1
DB
Zone
DB
Zone
DB
Domain
2
App
Domain
1
SPARC T5-8 Compute Node

DB
Domain
1
DB
Domain
2
App
Domain
1
Zone
Zone
DB
Zone
Zone
Clusterware
Both Database Domains and

Application Domains support Oracle
Solaris Zones
App
Domain
2
DB
Zone
DB
Zone
Zone
Solaris Cluster
Zone

1 to 4 LDoms per compute node (half rack)
1 to 8 LDoms per compute node (full rack)

Almost any combination of Database Domains and Application
Domains
Application Domains can boot either Oracle Solaris 10 or Oracle
Solaris 11
SPARC T5-8 Nodes configured independently

Domain Type
Domain Footprint on
Oracle SuperCluster
Access to
Exadata
Storage
Servers
Database Domain
Up to four (half rack)

or eight (full rack)
domains per SPARC
T5-8 node
Yes
Can be custom configured

post install
Supported
No
Yes
No
Configured for optional

Exalogic Elastic Cloud
Software, general
application binaries, and
logfiles
Supported
Yes
Yes
No
Configured for general

application binaries and
logfiles
Supported
No
Yes
Application Domain
booting Oracle
Solaris 11
Up to four (half rack) or

eight (full rack) domains
per SPARC T5-8 node
Application Domain
booting Oracle
Solaris 10
Up to three (half rack)

or seven (full rack)
domains per SPARC
T5-8 node
Access to Shared Storage

on Included ZFSSA
SAN access via

Optional Fiber
Channel HBAs
Suitable to
host
Exalogic
Software
Zones
Support

Multiple domains of any type supported in a single node
Flexible Node Layouts

Nodes can be configured independently
Note that it makes sense to configure as identical twins in most cases
CPU/Memory can be resized for each domain after initial install

Granularity of CPU is 4 cores
Granularity of Memory is 64GB (half rack) or 128GB (full rack)
CPU and Memory can be resized independently
Domains are named as Giant (Only on full RACK), Large, Medium

or Small
Half-Rack Single-Domain Configurations

Possible Domain Types
SPARC T5-8 Node
Database Domain
Domain
net0
Application Domain
(Oracle Solaris 11 only)
CPU/Memory Install
Options
Default
Domain 1
net3
12
13
14
15
S0
S0
S0
S0
S0
S0
S0
S0
S1
S1
S1
S1
S1
S1
S1
S1
HDD0
HDD1
HDD4
HDD5
HDD6
HDD7
HDD2 HDD3
4 sockets / 1TB
Boot and /u01 Disk
Mgmt port
(Active)
Boot and /u01 Mirror
Mgmt port
(Standby)
Half-Rack Two-Domain Configurations

Domain 1
Domain 2
Database Domain
Database Domain
SPARC T5-8 Node

Domain 1
Database Domain
Application Domain
(Oracle Solaris 11 or Oracle Solaris 10)
Application Domain
Application Domain
CPU/Memory
Install Options
Domain 1
Default
2 sockets / 512GB
Domain 2
Domain 2
net1
net0
net3
net2
12
13
14
15
S0
S0
S0
S0
S0
S0
S0
S0
S1
S1
S1
S1
S1
S1
S1
S1
HDD0
HDD1
HDD2
HDD3
HDD4
HDD5
HDD6
HDD7
2 sockets / 512GB
Option 2
1 socket / 256GB
3 sockets / 768GB
Option 3
3 sockets / 768GB
1 socket / 256GB
Option 4 (Special
case#)
4 Cores / 32GB
60 Cores / 1TB less 32GB
Boot and /u01 Disk

Mgmt port (Active)
Mgmt port (Standby)
Half-Rack Three-Domain Configurations

Domain 1
Domain 2
Domain 3
Database Domain
Database Domain
Database Domain
Database Domain
Application Domain
(Oracle Solaris 11 or
Oracle Solaris 10)
Database Domain
Database Domain
Application Domain
Oracle Solaris 10)
Application Domain
Oracle Solaris 10)
Application Domain
(Oracle Solaris 11
only)
Application Domain
Oracle Solaris 10)
Application Domain
Oracle Solaris 10)
SPARC T5-8 Node

Domain 1
S0
S1
S1
HDD0
CPU/Memor
y Install
Options
Domain 1
Domain 2
net1
net0
S0
HDD1
S0
Dom 2
vnet1
S0
S1
S1
HDD2 HDD3
Domain 3
Default
2 sockets /
512GB
1 socket /
256GB
1 socket /
256GB
Option 2
1 socket / 256GB
2 sockets /
512GB
1 socket /
256GB
Option 3
1 socket / 256GB
1 socket /
256GB
2 sockets /
512GB
12
Dom 3
vnet3
13
net2
14
net3
15
S0
S0
S1
S1
HDD4
Mgmt port
(Active)
Mgmt port
(Standby)
S0
S1
S1
HDD5 HDD6
Boot and /u01 Disk
S0
HDD7
Half-Rack Four-Domain Configurations

Domain 1
Domain 2
Domain 3
Domain 4
Database Domain
Database Domain
Database Domain
Database Domain
Database Domain
Application Domain
Oracle Solaris 10)
Database Domain
Database Domain
Database Domain
Application Domain
Oracle Solaris 10)
Application Domain
Oracle Solaris 10)
Database Domain
Database Domain
Application Domain
Oracle Solaris 10)
Application Domain
Oracle Solaris 10)
Application Domain
Oracle Solaris 10)
Application Domain
Oracle Solaris 10)
Application Domain
Oracle Solaris 10)
Application Domain
Oracle Solaris 10)
Application Domain
(Oracle Solaris 11
only)
SPARC T5-8 Node

Dom 1
net0
CPU/Memory
Install Options
Domain
1
Domain
2
Domain
3
Domain
4
Default
1 socket / 1TB
1 socket / 1TB
1 socket / 1TB
1 socket / 1TB
S0
S0
S1
S1
HDD0
HDD1
Dom 2
net1
vnet1
S0
S0
S1
S1
HDD2
vnet3
HDD3
Dom 3
vnet0
12
Dom 4
vnet2
net2
13
14
15
S0
S0
S1
S1
HDD4 HDD5
Boot and /u01 Disk
Mgmt port
(Active)
Mgmt port
(Standby)
net3
S0
S1
S0
S1
HDD6 HDD7
Full-Rack Two-Domain Configurations

Domain 1
Domain 2
Database Domain
Database Domain
Database Domain
Application Domain
Application Domain
SPARC T5-8 Node

Domain 1
Application Domain
net0
CPU/Memory
Install Options
Domain 1
Domain 2
Default
4 sockets / 1TB
4 sockets / 1TB
Option 2
2 sockets / 512GB
6 sockets / 1536GB
Option 3
6 sockets / 1536GB
2 sockets / 512GB
Option 4
1 socket / 256GB
7 sockets / 1792GB
Option 5
7 sockets / 1792GB
1 sockets / 256GB
Domain 2
net2
net1
10
11
12
13
net3
14
15
S0
S0
S0
S0
S0
S0
S0
S0
S1
S1
S1
S1
S1
S1
S1
S1
HDD0
HDD1
HDD4
HDD5
HDD6
HDD7
HDD2 HDD3
Boot and /u01 Disk
Mgmt port
(Active)
Mgmt port
(Standby)
Full-Rack Single-Domain Configurations

Possible Domain Types
SPARC T5-8 Node
Database Domain
Domain
Application Domain
net0
CPU/Memory Install
Options
Domain 1
Default
8 sockets / 2TB
net3
10
11
12
13
14
15
S0
S0
S0
S0
S0
S0
S0
S0
S1
S1
S1
S1
S1
S1
S1
S1
HDD0
HDD1
HDD4
HDD5
HDD6
HDD7
HDD2 HDD3
Boot and /u01 Disk
Mgmt port
(Active)
Mgmt port
(Standby)
Full-Rack Four-Domain Configurations

Domain 1
Domain 2
Domain 3
Domain 4
Database Domain
Database Domain
Database Domain
Database Domain
Database Domain
Application Domain
Oracle Solaris 10)
Database Domain
Database Domain
Application Domain
Oracle Solaris 10)
Application Domain
Oracle Solaris 10)
Database Domain
Database Domain
Application Domain
Oracle Solaris 10)
Application Domain
Oracle Solaris 10)
Application Domain
Oracle Solaris 10)
SPARC T5-8 Node
Application
Domain
(Oracle Solaris 11
only)
Application Domain
Oracle Solaris 10)
Domain 1
Domain 4
Database Domain
net0
Application Domain
Oracle Solaris 10)
Domain 3
Domain 2
Application Domain
Oracle Solaris 10)
vnet1
net1
S0
S0
S1
S1
S0
S0
S1
S1
vnet0
vnet3
Domain
1
Domain
2
Domain
3
Domain
4
Default
2 sockets / 512GB
2 sockets / 512GB
2 sockets / 512GB
2 sockets / 512GB
Option 2
1 socket / 256GB
2 sockets / 512GB
2 sockets / 512GB
3 sockets / 768GB
Option 3
3 sockets / 768GB
1 socket / 256GB
2 sockets / 512GB
2 sockets / 512GB
Option 4
3 sockets / 768GB
2 sockets / 512GB
1 socket / 256GB
2 sockets / 512GB
Option 5
3 sockets / 768GB
2 sockets / 512GB
2 sockets / 512GB
1 socket / 256GB
10
11
12
13
S0
S0
S1
S1
HDD4 HDD5
HDD0 HDD1 HDD2 HDD3
CPU/Memory Install
Options
net2
vnet2
Boot and /u01 Disk
Mgmt port
(Active)
Mgmt port
(Standby)
net3
14
15
S0
S0
S1
S1
HDD6 HDD7
Full-Rack Eight-Domain Configurations

SPARC T5-8 Node
Domains can be any mix of Database

Domains and Application Domains.
Application Domains can boot either
Oracle Solaris 11 or Oracle Solaris 10.
Note that the first (Control) domain must

boot Solaris 11. This domain is typically a
Database Domain.
CPU/Memory
Install Options
Domans 1
through 8
Default
1 socket / 256GB
Dom 1
net0
vnet1
net1
S0
S0
HDD0
HDD1
S1
S1
Dom 2
S0
vnet3
Dom 3
vnet0
vnet2
Dom 5
Dom 4
vnet1
vnet3
vnet0
vnet2
Dom 6
vnet1
10
vnet3
11
S0
vnet0
12
Dom 8
net2
vnet2
13
S0
HDD2 HDD3
S1
Dom 7
14
S0
HDD4 HDD5
S1
S1
Boot and /u01 Disk
Mgmt port
(Active)
Mgmt port
(Standby)
S1
net3
15
S0
S0
HDD6 HDD7
S1
S1
A SuperCluster Configuration Examples

Requirements:
Customer needs Isolation from two different clusters of Oracle Databases

One application is HA and should be protected by a Cluster in cases of failure
One application must use Solaris 9 and no HA
Customer acquires a T5-8 Half SuperCluster
Solution:
Deployment of the SuperCluster using four Small Domains, 2 database domains and 2
application domains
One application domain will run Solaris 10 and will have a Solaris 9 branded Zone
Oracle Cluster will be used to protect the HA application with a Cluster of 2 nodes each
node on a different T5-8 server.
Two RAC databases using 2 cluster of 2 nodes, Grid disks for Data, Reco and DBFS_DG will
be created on Exadata for each RAC Database.
A SuperCluster Configuration Examples

Requirements:
Customer wants to consolidate 8 Databases in a RAC configuration using different Clusters
Customer wants to use the 4 different environments that must on different machines,
isolated from each other, no HA for those environments.
Customer acquires a T5-8 Half SuperCluster
Solution:
Deployment of the SuperCluster using one medium Database Domain, Small 2 application
domains
8 RAC databases using 8 clusters of 2 nodes, based on Zones, Grid disks for Data, Reco and
DBFS_DG will be created on Exadata for each RAC Database.
SuperCluster Reviewing
Engineered Systems for Oracle Databases
Engineered Systems for Applications
Supports Non-Oracle Databases as well
Supports multiple Oracle database versions not only >=11.2 versions
Build on the best Oracle Technologies, like: SPARC processors, Exadata
Storage, Exalogic SW, Infiniband, etc.
Supports Virtualization (Pdom, Ldom and Zones)
Integrates with legacy storage if needed and configured.
Uses Infiniband Network.
30
31

GUOB-Revealing Explaining SPARC Supercluster

Hochgeladen von

Dokumentinformationen

Copyright

Verfügbare Formate

Dieses Dokument teilen

Dokument teilen oder einbetten

Freigabeoptionen

Stufen Sie dieses Dokument als nützlich ein?

Sind diese Inhalte unangemessen?

Copyright:

Verfügbare Formate

GUOB-Revealing Explaining SPARC Supercluster

Hochgeladen von

Copyright:

Verfügbare Formate

Revealing and Explaining

The Oracle SPARC SuperCluster

Safe Harbor Statement

The Engineered Systems Evolution

Virtualization on Oracle SuperCluster

Oracle T5-8 SuperCluster Configuration Options

A SuperCluster Configuration Example

Engineered Systems Evolution

Oracle Engineered Systems

Big Data Exalogic Virtual

Copyright 2014 Oracle and/or its affiliates. All rights reserved. |

Multi-Purpose Storage for application, stage, filesystems and databases

Oracle T5-8 SuperCluster Components

- 2 x SPARC T5-8 compute nodes

Exadata Storage Servers

8 x T5 Processors @ 3.6 GHz

2 TB of memory (128 x 16GB)

8 x 1.2 TB internal drive (SAS)

8 x Sun Dual 10 GbE SFP+ PCIe 2.0 Low

8 x Sun Quad Data Rate (QDR) InfiniBand PCIe

- 8 x Exadata Storage Servers X4-2

SPARC T5-8 Compute Nodes

- 3 x Sun Data Center InfiniBand Switch 36

Copyright 2014 Oracle and/or its affiliates. All rights reserved. |

SuperCluster M6-32 Components

Copyright 2014 Oracle and/or its affiliates. All rights reserved. |

Virtualization on Oracle SuperCluster

Servers can be separated in Domains (Physical or

Virtualization on Oracle SuperCluster

Copyright 2014 Oracle and/or its affiliates. All rights reserved. |

Virtualization on Oracle SuperCluster

When using Zones, some cores are reserved on global

Zones and LDOMs are considered hard partitioning for

Copyright 2014 Oracle and/or its affiliates. All rights reserved. |

Virtualization on Oracle SuperCluster

Copyright 2014 Oracle and/or its affiliates. All rights reserved. |

Virtualization on Oracle SuperCluster

Copyright 2014 Oracle and/or its affiliates. All rights reserved. |

Oracle T5-8 SuperCluster Configuration Options

Domains with zero performance

SPARC T5-8 Compute Node

SPARC T5-8 Compute Node

Both Database Domains and

Copyright 2014 Oracle and/or its affiliates. All rights reserved. |

Oracle T5-8 SuperCluster Configuration Options

1 to 8 LDoms per compute node (full rack)

Copyright 2014 Oracle and/or its affiliates. All rights reserved. |

Oracle T5-8 SuperCluster Configuration Options

Up to four (half rack)

Can be custom configured

Configured for optional

Configured for general

Up to four (half rack) or

Up to three (half rack)

Access to Shared Storage

SAN access via

Copyright 2014 Oracle and/or its affiliates. All rights reserved. |

Oracle T5-8 SuperCluster Configuration Options

Flexible Node Layouts

CPU/Memory can be resized for each domain after initial install

Domains are named as Giant (Only on full RACK), Large, Medium

Half-Rack Single-Domain Configurations

SPARC T5-8 Node

Boot and /u01 Mirror