Specifying Optical Fiber for HighHighSpeed Data Centers

John Kamino OFS

jkamino@ofsoptics.com

Agenda g
Multimode Fiber Market Drivers Multimode Standards and Applications Value Proposition p Future of Multimode Fiber Conclusions

IP Traffic Growth
Global IP traffic will quadruple from 2010 to 2015 Annual global IP traffic will reach nearly 1 zettabyte (1x1021 bytes) by the end of 2015 15% of traffic will be from non-PC devices by 2015 In 2015 wired devices will account for 46% of IP traffic, ff while h l Wi-Fi and d wireless l d devices will ll account for 54%.
Cisco Visual Networking Index: F Forecast t and d Methodology, M th d l 2010 2010-2105 2105 June 2, 2011

IP Traffic Growth
PB/Month
" Cisco Visual Networking Index (VNI): Forecast and Methodology, 2010-2015" June 1, 2011

90000 80000 70000 60000 50000 40000 30000 20000 10000 0 2010 2011 2012 2013 Mobile data Managed IP Fixed Internet

32% CAGR!

2014

2015

Mobile: Includes mobile data and Internet traffic generated by handsets, notebook cards, and mobile broadband gateways Internet: Denotes all IP traffic that crosses an Internet backbone Managed IP: Includes corporate IP WAN traffic, IP transport of TV/VoD

Worldwide Data Center Server Growth

Data rates continue to increase Server port growth resumed in 2010 Fiber volumes will continue to increase Significant 10GigE growth beginning in 2010

Ethernet Fiber Transceivers

LightCounting "Worldwide Sales of Optical Transceivers (Historical Data and Forecast)" March 28, 2011

16,000,000
1 00 GigE

14,000,000 12,000,000 10,000,000 8,000,000 6 000 000 6,000,000 4,000,000 2,000,000 -

40 GigE 1 0 GigE g LRM total 1 0 GigE LX4 1 0 GigE SR Total GigE Fiber Total Fast Ethernet

2008

2009

2010

2011

2012

2013

2014

2015

What is happening pp g today? y

PERCS Supercomputer 2011

Roadrunner Supercomputer 2008

Need for Higher Density and Lower Power Interconnects for Future HPC and Servers, January, 2011 Petar Pepeljugoski, Marc Taubenblatt, and Jeff Kash IBM

New York Times, February 16, 2011

Next Generation Speeds p Coming! g

http://connectedplanetonline.com/business_services/news/ieee-eyes-next-ethernet-speed-standard/

North America Multimode Fiber Demand

Multimode Fiber Demand - North America CRU February 2011 1,200 1 000 1,000 800 600 400 200 0 2008 2009 2010 2011 2012 2013 2014 2015 2016
OM4 OM3 OM2 OM1

IP traffic and server growth drive fiber demand Virtualization increasing server usage and d bandwidth b d idth demands d d Servers requiring multiple Ethernet connections Bandwidth requirements Redundancy 10Gbps server links drive 40Gbps uplinks

Agenda g
Multimode Fiber Market Drivers Multimode Standards and Applications Value Proposition p Future of Multimode Fiber Conclusions

Multimode Fiber Types yp

Multimode
(described in the industry using primarily the ISO/IEC 11801 designations)

Industry Standards
Fiber Type 62.5/125 50/125 50/125 50/125
(1) ( ) (2)

ISO/IEC 11801 (1) OM1 (2) OM2 OM3 OM4

IEC 60793-2-10 A1b A1a.1 A1a.2 A1a.3

TIA/EIA 492AAAA 492AAAB 492AAAC 492AAAD

ITU-T --G.651.1 -----

OM1 is typically a 62 62.5um 5um fiber fiber, but can also be a 50um fiber fiber. OM2 is typically a 50um fiber, but can also be a 62.5um fiber. "Generic Cabling for Customer Premises" "Product Specifications - Sectional Specification for Category A1 Multimode Fibres" "Detail Specification for Class 1a Graded-Index Multimode Optical Fibers" "Characteristics of a 50/125 um Multimode Graded Index Optical Fibre Cable for the Optical Access Network"

ISO/IEC 11801 IEC 60793-2-10 TIA/EIA-492AAAx ITU-T G.651.1

Multimode Fiber Types, Performance Grades

Min Bandwidth Wavelength
(nm)

Fiber Type OM1

62.5 m

Max Loss
(dB/km)

(MHzkm)

OFL BW 200 500 500 500 1500 500 3500 500

EMB n.s. n s n.s. n.s. n.s. 2000 n.s. 4700 n.s.

OFL BW =
Overfilled Launch Bandwidth

850 1300 850 1300 850 1300 850 1300

3.5 3 5 1.5 3.5 1.5 3.5 1.5 3.5 1.5

OM2
50 m

EMB =
Effective Modal Bandwidth (also known as Laser BW)

OM3
50 m

OM4
50 m

OMx designations are from ISO/IEC 11801 International Cabling Standard

OM4 Multimode Fiber Standardization

Complete in both TIA and IEC
TIA-492AAAD TIA 492AAAD IEC 60793-2-10, Fiber Type A1a.3

Specifications:
Effective Modal Bandwidth (EMB) >/= 4700 MHz-km Allows 2 methods for verification DMD Masks or EMBc
DMD Mask method shown to be more stringent

OFL Bandwidth @850nm >/= 3500 MHz-km

Ensures performance with sources that launch more power in outer modes.

OFL Bandwidth @1300nm >/= 500 MHz-km

Ensures backward compatibility for FDDI, 100BASE-FX, 1000BASE-LX, etc.

Why is OM4 important?

Fiber dominates in Access to Distribution and Distribution to Core links.

Alan Flatman Principal Consultant, LAN Technologies, UK Long Data Center Links vs. Length IEEE802.3ba, Jan. 2008, Flatman_01_0108

Evolution Of Short Reach Applications

Data Rate (Mbps) 40 000 40,000 10,000

Trends: T d LEDs Lasers (faster) OM1 OM4 ( (farther) )

40/100 GbE 10 GbE 1 GbE 850 or 1300 Laser 50 OM2 50 OM3 50 OM4 62.5 OM1 SM 850 or 1300 Laser 50 OM3 50 OM4 SM 850 or 1300 Laser 50 OM3 50 OM4 SM

1000

100

FDDI Ethernet 1300 LED 62.5m

Fast Ethernet 1300 LED 62.5 m

10

850 LED 62.5 m

1

1985

1990

1995

2000

2010

Applications pp Mapping pp g
Application Link Speed
Data Center Building Backbone Lg. Data Center Building Backbone Very Lg. Data Center Building Backbone Building Backbone Campus Backbone Campus Campus Backbone Backbone

1 Gb/ Gb/s

OM3/OM4 Multimode Fiber LaserWave 300 or LaserWave 550 Fiber OM4 Multimode LaserWave 550 Fiber

10 Gb/s

40 Gb/s OM4 Multimode LaserWave 550 Fiber 100 Gb/s OS1/OS2 Single-mode Fiber AllWave Fiber

Link Distance

100m

150m

300m

550m

1000m

>1000m

40G & 100G Ethernet (IEEE 802.3ba)

Reach & Media:
40 Gb/s for servers, HPC, SAN, NAS
10km on SMF (1310nm) 100m on OM3 MMF (850nm) 150m 150 on OM4 MMF (850 nm) ) 7m over copper 1m over backplane
40GBASE-LR4 40GBASE-SR4 40GBASE SR4 40GBASE-SR4 40GBASE-CR4 40GBASE-KR4

100 Gb/s for switching, routing, aggregation

40km on SMF (1310nm) 10km on SMF (1310nm) 100m on OM3 MMF (850nm) 150m on OM4 MMF (850nm) 7m over copper
100GBASE-ER4 100GBASE-LR4 100GBASE-SR10 100GBASE-SR10 100GBASE-CR10

High Speed Short Reach Technologies: Multiple Fiber Parallel Systems

for 40G:
One 12-fiber cable
duplex link 8 active fibers

12 Fiber MPO connector One wavelength per fiber 4 x 10 Gb/s

High Speed Short Reach Technologies: Multiple Fiber Parallel Systems

for 100G:
Two 12 Fiber Cables, or 24 fiber Cable
20 Active Duplex link

MPO connector
2 x 12 fiber 1 x 24 fiber

One wavelength per fiber 10 x 10 Gb/s

40G & 100G Ethernet MDI Recommendations

References f MPO interface f reqs/specs q /p of f IEC 61754-7.

40GBASE SR4 40GBASE-SR4

Left 4 fibers are Tx Right 4 fibers are Rx (inner 4 fibers unused) ( )

100GBASE-SR10

Inner 10 fib I fibers, T Top R Row are R Rx Inner 10 fibers, Bot Row are Tx
(outermost fibers both rows unused)

Inner 10 fib I fibers, L Left ft Sid Side are T Tx Inner 10 fibers, Right Side are Rx
(outermost fibers each side unused)

Inner 10 fibers, Top are Rx Inner 10 fibers, Bot are Tx

(outermost fiber Top & Bot unused)

Option A
(recommended)

Option p B

Option p C

40G & 100G Ethernet (continued)

Reduced reach of 100m on OM3, 150m on OM4 compared d to 10G (300m (300 on OM3, OM3 550m 550 on OM4) is i due to relaxation of transmitter spectral width: from 0.45 to 0.65 nm

There have been no changes to the fiber itself!

150 150m on OM4 expected t d to t support t 95%+ 95% of f Data D t Center links.

Data Center Environment

Migration to 10Gb/s
Cat6/6a copper Active Cable

Migration to 40 & 100Gb/s

OM3 or OM4 fiber fib (OM4 fiber recommended in latest draft of TIA942A)

OM3 or OM4 fiber (OM4 fiber recommended in latest draft of TIA942A)

Data from Alan Flatmann presented to IEE 802.3 High Speed Study Group January 2008

Applications pp
Data Center Todays Focus
OM4 multimode fiber is the recommended media in the latest draft of TIA-942A, Telecommunications Infrastructure Standard for Data Centers Only O l OM3 and d OM4 fib fibers are recognized i d in i the th latest l t t draft of TIA-942A Moving rapidly to optical based networking
Driven by bandwidth requirements
Servers are capable of utilizing 10G speeds Availability of 40/100G uplinks will drive adoption of 10G at the server

Driver for multimode fiber growth p Local Area Network Enterprise

Agenda g
Multimode Fiber Market Drivers Multimode Standards and Applications Value Proposition p Future of Multimode Fiber Conclusions

Cost implications p (100 ( G) )

Copper Twinax Cable Distance Transceiver Price Cable price Power use per port 7m OM3 Multimode Fiber 100m OM4 Multimode Fiber 150m OS1 Single-mode Single mode Fiber 10 km

CAPEX CAPEX

50 w

5w

5w

20+ w

OPEX

Example p Pricing g
100GBase-LR4 single-mode

http://www.pcmicrostore.com/SearchResult.aspx?q=kw:Juniper+100Gbase-LR4

100GBase-SR10 multimode (est)

$15,000 Scott Kipp, Brocade OFCNFOEC 2011 March 2, 2011

Power Consumption p
Lower power consumption critical as link density and speed increase
100G CFP single-mode transceivers consume 20+ watts 100G CXP multimode transceivers consume ~5 watts

Savings ~ 15 watts/transceiver Cooling C li another th 15 watts/transceiver tt /t i 4 transceivers/server i / ( (2 on each h end), d) multiplied l i li d b by hundreds or thousands of servers per data center can consume significant amounts of power!!

Single-mode vs. Multimode Module Size

Significantly larger f t i t for footprint f single-mode i l d CFP module Much lower faceplate density 4 single-mode modules in 1U footprint vs. 16-32 multimode modules!

Comparison between Singlemode and Multimode Fiber Systems Traditionally, optoelectronics have driven the cost difference between
single mode and multimode single-mode Single-mode CWDM system
Pro: Con: Con: Con: Lower cabling cost Significantly higher transceiver cost Higher power consumption Larger size

OM3 and OM4 multimode parallel systems

Pro: Much lower transceiver cost using existing 10Gb/s VCSELs Pro: Lower power consumption Pro: Smaller footprint Con: Higher cabling cost

What does this tell us?

Data centers will move towards more optical interconnects as bandwidth requirements increase Multimode solutions have key advantages over both single-mode i l d and d copper solutions l i Fiber bandwidth will be a key parameter Lower power consumption will be a key driver So what about new bend insensitive multimode fibers?

What is bend insensitive multimode fiber? (The simple description)

Bend loss 100 turns 37.5 mm radius Standard 850 nm 50/125 fiber 0.5 dB 1300 nm 0.5 dB Bend insensitive multimode fiber 850 nm 0.5 dB 1300 nm 0 5 dB 0.5 Bend loss 2 turns 15 mm radius 850 nm 1 dB 1300 nm 1 dB 850 nm 0.1 dB 1300 nm 0 3 dB 0.3 Bend loss 2 turns 7.5 mm radius 850 nm ? dB 1300 nm ? dB 850 nm 0.2 dB 1300 nm 0 5 dB 0.5
7.5 7 5 mm attenuation performance not specified for standard 50/125 fiber

What parameters impact system performance?

Interoperability Interoperabilit / Connection properties Bandwidth Reliability

It is desirable for optical fibers to be optimized for system performance

Comparison of wave guides

Standard Multimode Fiber Bend Insensitive Multimode Fiber

Trench is added to the BI-MMF to improve macrobend performance

Comparison p of wave guides g

Standard Multimode Fiber Bend Insensitive Multimode Fiber
Strongly guided modes

Weakly yg guided modes Leaky modes

In both STD and BI BI-MMF MMF mode groups 1 17 are strongly guided In STD MMF mode groups 18 19 are loosely guided In BI-MMF mode groups 18 and 19 are strongly guided
leaky y mode groups g p > 19 can also propagate p p g

Mode groups 18-19 are ignored in bandwidth calculations and IEEE 10Gb/s link model!!

Impact of BI-MMF beam expansion Reduced Performance

Higher loss when BI-MMF is connected to Std-MMF Similar trends likely when different BI-MMFs are connected Interoperability may be an issue with some BI BI-MMFs MMFs
Median Std.Dev. (dB) (dB) 0 22 0.22 0.29 0 06 0.06 0.09 + 3 (dB) 0 40 0.40 0.56

Fiber A Stand. OM3 BI-MMF Design Y BI-MMF Design X

Fiber B Stand. OM3 Stand. OM3 Stand. OM3

0.31

0.11

0.64

What about other parameters that impact system performance?

Interoperability Interoperabilit / Connection properties Bandwidth Reliability

It is desirable for optical fibers to be optimized for system performance

DMD Measurements of Comparable EMBc Fibers

BI-MMF 1

MW18 0.094 EMBc 5555

MW23 0.094 OFL850 6876

SLW 0.081

BI-MMF 2

MW18 0.088 EMBc 5512

MW23 0.223 OFL850 3522

SLW 0.094

Light traveling in high order modes was ignored when EMBc weight functions were developed, is this assumption still valid for BI-MMF?

Systems Performance of Comparable EMBc fibers

1.E-04 1.E-05 1 E 06 1.E-06 1.E-07 1.E-08
BE ER
Back to Back BI-MMF 1 BI-MMF 2

1.E-09 1.E-10 1.E-11 1.E-12 1 E 13 1.E-13 1.E-14 1.E-15

-16 -15 -14 -13 -12 -11 -10 -9 -8 received average power (dBm) ( )

1.2dB 3dB

Note: EMBc measurements are similar, but DMD Outer Mask Width shows significant higher order mode problems

Some BI-MMF fibers have large system penalties and others do not!

Current OM3 and OM4 bandwidth measurement criteria developed for standard MMF may need to be modified

BI-MMF 1 BI-MMF 2

MW18 0.094 0.088

MW23 0.094 0.223

SLW 0.081 0.094

EMBc 5555 5512

OFL850 6876 3522

Standard Multimode Fibers

Higher order modes quickly attenuated Maximum transmission distance! 1010101010101010101010101010101010101
10 Gbps
850 nm Laser

Detector

Core Cladding

Bend Insensitive Multimode Fiber

Higher Order Modes Propagate in BIMMF!!

Lower Bandwidth and Transmission Distance

10101010101??? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?
10 Gbps
850 nm Laser

Detector

Core Cladding

Higher order modes not accounted for with current bandwidth measurements and standards!

What about other parameters that impact system performance?

Interoperability Interoperabilit / Connection properties Bandwidth Reliability

It is desirable for optical fibers to be optimized for system performance

Fiber Reliability
Fiber reliability is a function of:
Optical O ti l R Reliability li bilit
Maintaining optical signal through the link Sufficient bandwidth to support future system upgrades

Mechanical Reliability
Inherent glass quality intrinsic strength Proof test level extrinsic strength Packaging cable design Deployment

Reliability y
Bend loss at 850-nm (2.5 Turns) for OM3 fiber types 10

Ben ndloss (dB) )

Mechanical reliability concerns

9 8 7 6 5 4 3 2 1 0 2 3 4 5

Stand. BOMMF BIMMF 11 BIMMF 22 BOMMF BIMMF 33 BOMMF

Optical reliability concerns

Risky Bend Warning!

6 7 8 9 10 11 12 13 14 15 Mandrel radius (mm)

Bend insensitive fiber is not a substitute for poor installation practices!

Relative diameter

r=?? r ?? mm Shelf Edge

r=3.5 r 3 5 mm Std. Pencil

r=7.5 mm

r=8.95 mm

Sharp p 1/4 / turns around fiber shelves can have very y small radius Optical reliability can be maintained Long term mechanical reliability is at risk 7.5mm is a reasonable radius to maintain, without additional risk of mechanical failure

Bend optimized p fiber is not a substitute for good g cable management!!! g

Reliability of bare optical fiber with bends and tension

single turn calculation for 250 m fiber
Fiber bend radius 6.5 mm 6 5 mm 6.5 Pull force on fiber (pounds force) 0 4 Fiber strain kpsi ~100 ~300 20 year failure probability < 1 ppm > 50% 4 lbs pulling force (~200 kpsi)

Maximum stress at corner (~100 kpsi b di bending strain)

Combined bends and tension is now part of new revision to: IEC TR 62048 Optical fibers Reliability - Power law theory

Mechanical Reliability Criteria

What is the criteria for a data center?
MDU Users per fiber Accepted Reliability Criteria Cost of failure 1 Less than 10 ppm in 20 years 1 customer down, truck roll 5 mm Radius FTTx 10-100 Less than 1 ppm in 30 years Trunks and Metro Applications pp 100-1000 Less than 1 ppm in 30 years Long haul Submarine 100-10,000 0 ppm in 40 years target

Several customers down, need for immediate repair 10 mm Radius

FCC reportable incident, need for immediate repair 15 mm Radius

FCC reportable incident. Failure extremely costly (Ship must be used) 30 mm Radius

Minimum design bend

Summary of bend insensitive multimode fiber

It is prudent to balance macrobend performance with i h other h fib fiber parameters i including l di b bandwidth d id h and d interoperability to optimize systems performance Bend insensitive multimode fibers are a new and unstandardized product. Efforts are currently underway in the standards bodies to study this new fiber fiber. Wide spread adoption of this fiber should be limited until technical issues are resolved in standards bodies

Agenda g
Multimode Fiber Market Drivers Multimode Standards and Applications Value Proposition p Future of Multimode Fiber Conclusions

Why y the higher g speeds? p

More interconnects M i t t and switches required

High speed connections simplify the network

Pictures presented by Adam Bechtel Yahoo! Chief Architect IEEE 802.3 Plenary March 2007

Where is Multimode Going?

Towards 4x25 Gb/s transmission!
Data centers are overflowing g with connections The current Ethernet evolution path
10 Gb/s is a single fiber (Duplex cable) solution 40 Gb/s is a 4X10 solution sol tion with ith a 12 fiber cable 100 Gb/s is a 10X10 solution using a 24 fiber cable

On this path, transition from 40 G to 100 G will require additional fiber 100 Gb/s 4X25 solution is an easier upgrade path!
OPPORTUNITY: A new fiber (OM5) may be required for the longest links

Agenda g
Multimode Fiber Market Drivers Multimode Standards and Applications Value Proposition p Future of Multimode Fiber Conclusions

Conclusions
Data centers are shifting from copper to fiber
Copper is more costly and has limited transmission distance Fiber has much lower power consumption and can support longer links Multimode fiber has key advantages over both copper and single-mode fiber

OM3 and d OM4 Multimode M lti d fiber fib is i th the media di of f choice h i f for 40 40-100 100 Gb/ Gb/s transmission rates in data centers
OM1 and OM2 not recognized media in latest TIA-942A Data Center Cabling draft

OM4 is well accepted and has been incorporated in system standards

Recommended fiber in latest TIA-942A Data Center Cabling draft!

There will Th ill b be a shift hif to parallel ll l transmission i i over multimode l i d fib fiber with ih MPO connections as network speeds exceed 10 Gb/s 4x25Gb/s transmission is on the horizon

Th k You! Thank Y !
Visit OFS at the Exhibit Hall