IBM Client Center

IBM Flash Systems And Oracle

Improving your IO throughput and response time with IBM FlashSystems

Ronan Bourlier

ronan.bourlier@fr.ibm.com

2013 IBM Corporation

IBM Client Center

Agenda
IBM Oracle Center (IOC)
IBM Flash Systems Technology
Why Flash ? Why now ?
IBM Flash Systems Portfolio

IBM Flash Systems with Oracle Database

Some Benchmarks at IOC
Oracle configuration exemple

2013 IBM Corporation

The IBM Oracle Center (aka IOC)

OUR MISSION
Help IBM customers to deliver integrated solutions with Oracle Software
Products on IBM Infrastructures
OUR STRENGH
Cross platform team with strong knowledge on Oracle products and a wide
network within IBM and Oracle ecosystem
OUR ACTIVITIES
Convince : Briefings & Conferences
Build : Architecture, Design, Sizing
Demonstrate : Proof-of-Concept, Benchmarks
Deliver : Publications & Workshops
COVERED PRODUCTS
IBM Platforms (System z,Power, System x, Total Storage)
Oracle Technologies (Oracle DB, RAC, ASM, Dataguard)
Oracle Applications (EBS, Siebel & OBI & OWI)
Entry point to other on Industry Solutions (BRM, iFlex, RETEK, Weblogic)

Mixed IBM / Oracle Architectures

3

18-Nov-13

2013 IBM Corporation

Agenda
IBM Oracle Center (IOC)
IBM Flash Systems Technology
Why Flash ? Why now ?
IBM Flash Systems Portfolio

IBM Flash Systems with Oracle Database

Some Benchmarks at IOC
Oracle configuration exemple

18-Nov-13

2013 IBM Corporation

Evolution of Education in Storage Performance

1970s

1980s

1990s

2000s

2010s

In the
beginning,
there were iron
rings and
copper coils

Data growth
occurs and
storage is
measured by
what it can hold

Storage
measured by
the quantity of
data transferred
in a given
interval

I/O throughput
now a function
of requests
and less focus
on bytes

You
Are
Here

Persistence

Capacity

Bandwidth

IOPS

2013 IBM Corporation

Applications see time waiting, not IOPS

All client requests take time

I/O
I/O Time
Time

Network
Network
Time
Time

CPU
CPU Time
Time

Newer CPUs help

I/O
I/O Time
Time

Network
Network
Time
Time

CPU
CPU
Time
Time

Newer CPUs + FlashSystem magnify the gains

I/O
I/O
Time
Time

Network
Network
Time
Time

CPU
CPU
Time
Time

Time Recovered
Query Time

2013 IBM Corporation

Why Flash..Why Now?

In the last 10 years

CPU Speed: Performance increase roughly 8-10x

DRAM Speed: Performance increase roughly 7-9x

Network Speed: Performance increase of 100x

Bus Speed: Performance increased roughly 20x

Disk speed: Performance increased 2X

2013 IBM Corporation

Why Flash..Why Now?

In the last 10 years with Flash

CPU Speed: Performance increase roughly 8-10x

DRAM Speed: Performance increase roughly 7-9x

Network Speed: Performance increase of 100x

Bus Speed: Performance increased roughly 20x

Disk speed: Performance increased

30x

2013 IBM Corporation

The world of storage

s Lowest Latency

Very Expensive

us

Consistent Performance

Server
Server Flash
Flash

Low Latency
SAN
SAN Shared
Shared Flash
Flash Only
Only Array
Array
Server
Server SSD
SSD
Hybrid
Hybrid Array
Array or
or SAN
SAN SSD
SSD
ms

Fair Latency
Expensive HDD

Tier
Tier or
or Cache
Cache

Array
Array HDD
HDD

Unbalanced Performance

Lowest Performance
sec

Lower Cost
2013 IBM Corporation

Agenda
IBM Oracle Center (IOC)
IBM Flash Systems Technology
Why Flash ? Why now ?
IBM Flash Systems Portfolio

IBM Flash Systems with Oracle Database

Some Benchmarks at IOC
Oracle configuration exemple

10

18-Nov-13

2013 IBM Corporation

IBM Flash Systems

Flash Choice

SLC

eMLC

FlashSystem-710

FlashSystem-720

FlashSystem-810

FlashSystem-820

1 -5 TB

5 or 10 TB

2 - 10 TB

10 or 20 TB

Latency R/W
(microseconds)

90/25

90/25

110/25

110/25

IOPS

450K

500K

400K

450K

Bandwidth

4GB/s

5GB/s

4GB/s

4GB/s

Models
Capacity

Interface 4-Ports
Data Protection

8Gb FC or 40Gb IB
VSR

VSR+2D

8Gb FC or 40Gb IB
VSR

VSR+2D

All units 1U form factor, less than 500 Watts

2013 IBM Corporation

IBM FlashSystem 820/720: Logical Components

Hardware-only data path with extreme LOW latency
Custom FPGA-based data movement decreases latency vs. software
Distributed out-of-data-path CPU processing
High Performance (IOPS), High Bandwidth

Interface Controller

Management Module
CPUs (16)

RAID Controller

Flash Modules (12)

2013 IBM Corporation

Variable Stripe RAID (VSR)

Patented VSR allows RAID stripe sizes to vary.

If one die fails in a ten-chip stripe, only the failed die is bypassed, and then data is
restriped across the remaining nine chips Reusing the remaining capacity in both
the chip & the stripe

Higher availability; Fewer maintenance events due to flash failures
10 Chips

16
Planes

FAI
L

2013 IBM Corporation

Variable Stripe RAID (VSR)

Form Factor SSD

Flash failure = Disk failure
Requires top-level RAID
Relatively frequent hot-swaps

No Parity
Enterprise Flash Drive or Memory Module
Flash failure = Degraded state
within module
Performance impact on RAID set
Hot-swap to resolve

Parity

FlashSystem with Variable Stripe RAID

Preserves Flash life
Preserves performance
Re-parity data in microseconds

Less maintenance touches

while still preserving the
life, protection, and
performance of the Day-1
experience

Parity
2013 IBM Corporation

2D Flash RAID (FlashSystem 720/820)

External
Interfaces
(FC, IB)
RAID
Controllers

Variable Stripe
RAID within
Flash Modules

TMS
2D Flash RAID

(9 data + 1 parity)
Self-healing Flash Mgmt:
Preserves life
Preserves performance

RAID 5 across Flash Modules

(10 data + 1 parity + 1 hot spare)

Flash Ctrlr Protection

Brings hot-swap to
Flash cards
2013 IBM Corporation

How Latency translates to Application Efficiency

I/O Serviced by Disk
Issue/Process I/O request
Wait for I/O to be serviced

CPU Utilization = ~4%

~ 100+100 s
~ 5,000 s

Less Work,
More Wait

time
Waiting for I/O

CPU Utilization
Work

Waiting for I/O

Work

Waiting for I/O

Work

I/O Serviced by FlashSystem

Issue/Process I/O request

~ 100+100 s

Wait for I/O to be serviced

~ 200 s

time
CPU Utilization

Work
Work

Work
Work

Wait

Work
Work

Wait

Work
Work

Wait

Work
Work

Work
Work

Wait

Wait

Wait

2013 IBM Corporation

Agenda
IBM Oracle Center (IOC)
IBM Flash Systems Technology
Why Flash ? Why now ?
IBM Flash Systems Portfolio

IBM Flash Systems with Oracle Database

Some Benchmarks at IOC
Oracle configuration exemple

17

18-Nov-13

2013 IBM Corporation

Benchmark Setup Stressing Test Reference on traditional hard disk

based storage

LPAR1 : Running Oracle Database Enterprise Engine

LPAR0 : Running SwingBench

SwingBench

Workload

Oracle 11gR2
VIOS1

VIOS2

SwingBench Partition :
Power7+
12 Go RAM
AIX 7.1 TLXXX
ora_data
2 To

Rootvg
20 Go

Oracle Database Partition : 11.2.0.3 Single Instance

ora_bin
20 Go

Power7+
64 Go RAM
AIX 7.1 TLXXX

Hard disk based storage

3 Volume Groups on storage (rootvg,oracle binaries,oracle

datafiles)
Filesystem : JFS2
2013 IBM Corporation

Benchmark Setup Stressing Test Mix IBM Flash & Traditional hard disk
based storage

LPAR1 : Running Oracle Database Enterprise Engine

LPAR0 : Running SwingBench

SwingBench

Workload

Oracle 11gR2
VIOS1

VIOS2

SwingBench Partition :
Power7+
12 Go RAM
AIX 7.1 TLXXX

rootvg
20 Go
ora_data
2 To

Oracle Database Partition : 11.2.0.3 Single Instance

Power7+

ora_bin
20 Go

64 Go RAM
AIX 7.1 TLXXX

FlashSystems

Hard disk based Storage

2 Volume Groups on storage (rootvg,oracle binaries)

1 Volume Group on IBM Flash System (oracle datafiles)
Filesystem : JFS2
2013 IBM Corporation

IO rate seen from OS

Around 5000 IOPS on the standard storage system disk/LUN

Around 250 000 IOPS on the FlashSystems disk/LUN

Factor 50 in performance increase

How will this improve Oracle performances ?

2013 IBM Corporation

Orion
Orion Normal Test
350000

8000

300000

7000
6000
5000

200000

4000
150000

IO (#)

IO (#)

250000

FlashSystems
Legacy

3000

100000

2000

50000

1000

0
6
57
48
10 8
8
42
52 44
21
26 2
7
10
13 6
53
65 8
76
32 4
38
16
92
81
96
40
48
20
24
10
2
51
6
25
8
12

64

32

16

Latency (us)

Orion simulates Oracle queries to storage systems

FlashSystems improves by a factor 20 the average latency seen by Oracle Database

2013 IBM Corporation

AWR Comparaisons with OLTP

Hard disk based storage

the db file sequential

read with is 15 times
better.
IBM FlashSystem

Give more time for

CPU.

2013 IBM Corporation

Give it a try

Estimation of performance improvement

Send us a AWR report on one hour of your most busy workload
We will send back an estimation of performance improvement thanks
to FlashSystem

Try and Buy
Want to try it in your infrastructure ?
Contact your local IBM commercial for a try
Keep it if you like it !

2013 IBM Corporation

Agenda
IBM Oracle Center (IOC)
IBM Flash Systems Technology
Why Flash ? Why now ?
IBM Flash Systems Portfolio

IBM Flash Systems with Oracle Database

Some Benchmarks at IOC
Oracle configuration exemple

24

18-Nov-13

2013 IBM Corporation

Solutions in Order of Ease

2013 IBM Corporation

Oracle Architecture

DB Servers

ASM
LVM

Boost Performance
Boost Redundancy
- Without Disruption
- Without Risk
- Without Feature Loss

READS

SAN
SAN

WRITES

SAN
SAN

ASM FG2
IBM Flash System

ASM FG1
ACTIVE DATA

ACTIVE DATA

20 TB

Mirror

100 TB
20 TB
ARCHIVE DATA

TRANSITIONAL DATA
5 TB

26

TRANSITIONAL DATA
5 TB

2013 IBM Corporation

Preferred Read Mirror

Reads are serviced from the preferred mirror, in this case the Flash Storage

Writes are synchronous to both mirrors

Eliminates read poisoning of the disk array
Read poisoning: Writes are delayed by multiple head repositioning due to
reads

Can be used with any existing SAN as long as it has preferred read capability.

In initial testing Oracle ASM was used, but SVC will work as well

2013 IBM Corporation

Preferred Read - Example

Storage performance @10,000 IOPS for a given app
Read/Write Ratio @ 80% Reads / 20% Writes
Reads: 8,000 / Sec
Writes: 2,000 / Sec

Introduce IBM FlashStorage as

Primary Copy of new mirror

8,000 Reads / Sec now at extremely low latency

Storage was 10,000 IOPS

 Now 10,000+ Writes / Sec

R/W ratio does not change; No

change in the app

Storage does 10,000 Writes & IBM FlashStorage does 10,000 Writes & 40,000 Reads

= 5x Previous storage
performance
Texas Memory Systems, Inc., an IBM Company - The World's Fastest Storage

2013 IBM Corporation

A way to look at TCO and Storage Economics

Normalized $ / IOPS
11x Less

1.3 Million IOPS

43K+ Transactions p/sec

13K Updates per second
IBM
2,500Spindles
FlashSystem + 128 SSDs

Energy

5,000
Spindles

Space
26x Less

IBM
2,500Spindles
FlashSystem + 128 SSDs
ibm.com\storage\flash

80x Less

5,000
Spindles

IBM
2,500Spindles
FlashSystem + 128 SSDs

5,000
Spindles
2013 IBM Corporation

Beyond Latency: Data Economics

SPC-1 Comparison FlashSystem vs. SSDs

IBM FlashSystem 820

Midrange SSD-based Array

Price per GB

Latency
(minimum)

Power

Space

Capacity
density (GB/U)

IOPS density
(IOPS/U)

75%

45%

30%

75%

97%

67%

better

better

better

better

better

better

Source: SPC-1/E Result #AE00006 (IBM FlashSystem 820), 8/16/2013, and SPC-1 Result #A00134 (HP 3PAR StoreServ 7400 Storage System (with SSDs)), 5/23/2013.
Data indicated by * is derived from SPC data, not reported directly. More footnotes coming

Under Embargo Until Nov. 19, 2013

2013 IBM Corporation

IBM Flash Storage Sweet Spots

Do More, Do it Faster
Virtual
Infrastructures

OLTP
Databases

VDI, Consolidated virtual

infrastructures, user profiles

Financial, gaming, real-time billing,

trading, real-time monitoring, query
acceleration (DB2/Oracle)

Analytical
Applications (OLAP)
HPC,
Computational
Applications
Simulation, modeling, rendering, FS
metadata, scratch space, video on
demand, thread efficiency

Business intelligence, batch processing,

ERP systems, reporting, massive data
feeds

Cloud-scale
Infrastructures
On-demand computing, content distribution,
web, caching, metadata, GPFS, active file
management

2013 IBM Corporation

IBM Flash Impact

Boost performance without re-architecting applications!

85% Reduction
In batch processing
times

90% Reduction
In OLTP times

75% Footprint
Reduction
Store one petabyte in
a single floor tile. Add
compression and add
up to 100% more

150-200 s
Latency

80% Reduction
Energy Usage

Enterprise
Reliability
High Availability,
2D Flash RAIDTM
and Variable
Stripe RAIDTM

2013 IBM Corporation

How to contact us

ioc@fr.ibm.com

33

18-Nov-13

2013 IBM Corporation

DISCLAIMER
No part of this document may be reproduced or transmitted in any form without written permission from IBM Corporation.
Product data has been reviewed for accuracy as of the date of initial publication. Product data is subject to change without
notice.
This information could include technical inaccuracies or typographical errors. IBM may make improvements and/or changes in
the product(s) and/or program(s) at any time without notice. Any statements regarding IBM's future direction and intent are
subject to change or withdrawal without notice, and represent goals and objectives only.
The performance data contained herein was obtained in a controlled, isolated environment. Actual results that may be obtained
in other operating environments may vary significantly. While IBM has reviewed each item for accuracy in a specific situation,
there is no guarantee that the same or similar results will be obtained elsewhere. Customer experiences described herein are
based upon information and opinions provided by the customer. The same results may not be obtained by every user.
Reference in this document to IBM products, programs, or services does not imply that IBM intends to make such products,
programs or services available in all countries in which IBM operates or does business. Any reference to an IBM Program
Product in this document is not intended to state or imply that only that program product may be used. Any functionally
equivalent program, that does not infringe IBM's intellectual property rights, may be used instead. It is the user's responsibility
to evaluate and verify the operation on any non-IBM product, program or service.
THE INFORMATION PROVIDED IN THIS DOCUMENT IS DISTRIBUTED "AS IS" WITHOUT ANY WARRANTY, EITHER
EXPRESS OR IMPLIED. IBM EXPRESSLY DISCLAIMS ANY WARRANTIES OF MERCHANTABILITY, FITNESS FOR A
PARTICULAR PURPOSE OR INFRINGEMENT. IBM shall have no responsibility to update this information. IBM products are
warranted according to the terms and conditions of the agreements (e.g. IBM Customer Agreement, Statement of Limited
Warranty, International Program License Agreement, etc.) under which they are provided. IBM is not responsible for the
performance or interoperability of any non-IBM products discussed herein.
Information concerning non-IBM products was obtained from the suppliers of those products, their published announcements
or other publicly available sources. IBM has not tested those products in connection with this publication and cannot confirm
the accuracy of performance, compatibility or any other claims related to non-IBM products. Questions on the capabilities of
non-IBM products should be addressed to the suppliers of those products.
IBM Corporation 2013

2013 IBM Corporation