DataCenterCablingStandardsKeepingPace

withTechnologyChanges
Thursday 9May
JonathanJew
J&MConsultants,Inc.
CoeditorTIA942ADataCentersTelecomInfrastructure
CochairBICSIDataCenterSubcommittee
VicechairTIATR42.6TelecomAdministration
VicechairTIATR42.1CommercialBuildings
www.jandm.com

UpdatesinTIA942A
Weareupdatingdatacentercablingstandards
tokeeppacewithtechnologychanges
TIA942wasrecentlyrevisedwithpublication
ofTIA942A
RecentpublicationofaddendumtoTIA942A
dealingwithdatacenterfabrics
BICSI002isundergoingrevisionwith2014or
2015publication

FitTIA942intonewTIAcabling
standardsstructure
InTIA568Cseries,cablingstandardswere
reorganizedtopermitpremisesspecific
standardstobedevelopedwithout
duplicationofcontent.
TIA942Awasrewrittentofitintothenew
structure

ReorganizationofTIAstandards

Movecontenttoproperstandards
Referencegenericcablingtopology,terms,andMICE
(mechanical,ingress,climatic,electromagnetic)
environmentalclassificationsfromTIA568C.0
Movebonding&groundingcontenttoTIA607B
Moveadministration&labelingtoTIA606B
Moveracks&cabinets,powerandtelecom
separation,andtemperature/humidityrequirements
toTIA569C
MoveoutsideplantpathwaystoTIA758B
ISO/IEC11801seriesalsobeingreorganized

ReorganizationofISO/IECstandards

TIAtomatchwithISO
LCconnectorforupto2fibers
(ISO/IECspecifiesangledLCforSMatthe
ExternalNetworkInterface)
MPOconnectorformorethantwofibers

LC

Removed100mchannellengthlimitation
forhorizontalcablingoffiber
Nowlimitedbyapplicationlength
restrictionsofthetypeoffiberused
Matchsometerminology(ENI,EO)

MPO

Higherbandwidths
Specifyhigherbandwidthcablingtypesto
support
Higherperformancesystemsandapplications
Higherperformancenetworks(switchfabrics)
LAN&SANconvergence

BenefitsofLAN/SANConvergence
Reducenumberofserverconnections
Allowuseofsmallservers(blade&1U)
thatcantsupportlargenumberof
adapters
Reducecostandadministration(fewer
adapters,fewerswitches,lesscabling)
Simplifysupport Ethernetonly(no
separateFibreChannelinfrastructure)
Butitrequireshighbandwidthandlow
latency datacenterLAN

TIA942AHigherBandwidthCopper
Retained734/735coaxcableforT3/E3
RemovedsupportforCategory3and5efor
horizontalcabling,butkeptforbackbone
cabling(WAN,voice,console)
Category6minforhorizontalcabling
Category6Aisrecommended
Category6AistheminimuminISO/IEC24764

734 coax

Cat 7A

Cat 6A

10GBaseT
Tosupport10GBaseT,Category6Aorbetter
recommendedinTIA942Aandminimumin
ISO/IEC24754
10GBaseTwillbewidelyadoptedin2013and
ispredictedtobethemostwidelyshipped
versionof10GEthernetin2014

Factorsinadoptionof10BaseT
Rapidlydecliningcost
Improvedpowerefficiency,now<4W,less
withshortreachmode(e.g.,<2Wat10m)
Autonegotiation backwardcompatiblewith
100M&1GEthernet
Familiarbalancedtwistedpaircabling&RJ45
thattechniciansknowhowtotroubleshoot
andterminate
Flexibleconfigurationandlengths:2to4
connectors,upto100m

NextGenerationCabling
IEEE40GBaseTlengthtarget30m(~20
cabinets) andwilluse2connectorchannels
(w/autonegotiationtolowerspeed)
TIA568C.21Category8 newbalancedTP
cablebackwardcompatiblew/Category6A
ISO/IECTR11801991guidanceforcablingin
supportof40Gbpstransmissionspecifies
30mchannelsforCat6A,7,7Athatcansupport40Gbps
30mchannelsusingCat8.1(backwardcompatiblew/Cat6A)
orCat8.2components(backwardcompatiblewithCat7A)

Higherbandwidthopticalfiber
RemovedOM1andOM2(62.5mandnon
laseroptimized50/125mmultimodefiber)

OM3 (50/125mlaseroptimized)isthe
minimum requirement
OM4 isrecommendedformorebandwidth
andlongerlengths

Ethernetchannellengthsovermultimodefiber
Fiber
Type

1G

10G

40G

100G

100G

#fibers

20

OM1
OM2
OM3
OM4

275m

26m

550m

82m

800m

300m

100 m

100 m

20m*

1040m

550 m

150m

150m

106m*

Distancesinred arespecifiedbymanufacturersbutnotinIEEE
standards.
*IEEE802.3bmStandardfor4lane100Gtarget1Q2015

40G/100GoverOpticalFiber
Multimodefibermorecosteffectivethan
singlemodeforlengths<=150m
40Guses8MMfibersandMPO
100Guses20MMfibersandeithertwo12
fiberMPOsorone24fiberMPO
4lane/8fiber100Gwillcostlessandis
expectedtosupportatleast20moverOM3,
100moverOM4,andatleast500moverSMF.

Singlemodefiber
Singlemodeneededfor:
Deliveryofcircuitsfromcarriers
Campuscabling
Backboneforlargedatacenters

Backbonecablelengths>150mwillneedsingle
modefiberfor40/100GEthernet
2SMfibersforcurrentstandard(10/40km)
4lane100GSMFstandardindevelopment
(500m 2km,2or8fibers tobedetermined)
Nextstepis400GEthernet(inprogress)

TIA942AEnergyEfficiency
Newsectiononenergyefficiency
Widerrangeoftemperatures
andhumidity(seeTIA568C)
basedonnewASHRAETC9.9
guidelines
Otherguidelinesforenergy
efficiencyrelatingtocabling,
pathways,andspaces

EnergyEfficiency
2011/2012ASHRAEguidelineswereadopted
(ANSI/TIA569C1)
Temperature:15 35oC (59 95oF)
Relativehumidity(RH):20 80%

ESDcouldbeaproblemwithlowhumidity
(<44%at18 oC,<25%at27 oC)
Thesearetemperaturesinthecoldaisles.In
hotaislestemperaturescouldbe55oC
Attenuationofbalancedpaircablesincreases
athightemperatures(~14%at55 oC)

EnergyEfficiency
Overheadcabling
ifunderfloor
spaceisusedfor
cooling
Routecablesand
pathwaysto
minimize
interferingwith
properairflow

EnergyEfficiency
Avoidexteriorwindowsandskylightsbecause
theycreateadditionalheatload
Greatbuilding,butnotgoodforadatacenter

EnergyEfficiency
Cableroutingshouldnot
compromiseefficiencyof
enclosures blanking
panels,brushes&
grommetsforcable
openings
Equipmentshouldmatch
airflowdesignofcabinets
(fronttoback,usebaffles
forsidetosidecooled
equipment)

EnergyEfficiency
Enclosuresorenclosuresystems:
cabinetswithisolatedsupplyorreturn(e.g.,chimneys)
cabinetcoolingsystem
hotorcoldaislecontainment

Cabinets with
Isolated return
Aisle-Containment

EnergyEfficiency
Separateareasforequipmentwithhighpower
densityneeds
Buildinphases(modularconstruction)

EnergyEfficiency Lighting
EnergyefficientlightingsuchasLED
3Levellightingprotocolbyzonefordatacenters
dependingonhumanoccupancy
Level1 nooccupants enoughlightforcameras
Level2 motiontoinitiatehigherlevelaccessto
providesafepassage&IDbycameras
Level3 occupiedforwork fulllighting500lux

Largerandmodulardatacenters
NewTIA942spaceIntermediateDistribution
Area(IDA) containingtheIntermediateCross
Connect(IC)
AdoptedinISO/IEC24764Addendum1as
IntermediateDistributor(ID)
EliminatedTIArequirementthatcentralized
opticalfibertopologiesbelimitedtoone
buildingtoaccommodatemodulardata
centersusingoutdoorcontainers&modules

ISO/IECJTC1SC39
NewISO/IECJTC1Subcommittee39
Sustainabilityfor&byInformationTechnology
Newprojects:
BestPracticesforGreenDataCenters
PowerUsageEffectiveness(PUE)&Data
CenterinfrastructureEfficiency(DCiE)metrics
SecretariatforSC39
SallySeitz
SSeitz@ansi.org

IDAExample

Modulardatacenters
Outdoorusingcontainers/modules
Indoorusingmanufacturedmodulesorby
simplybuildingrooms/areasinphases
Reducecapitalexpenditures(Capex)and
operatingexpenditures(Opex)bybuildingonly
whatyouneedwhenyouneedit.
Donotneedtopaymaintenanceand
power/coolequipmentandroomsthatarent
neededyet.
Modularityreducestheriskcreatedby
uncertaintyaboutthefuture

Datacenterswitchfabrics
TIA942Aaddendum1(approvedfor
publicationFeb2013)providesguidance
regardingcablingfordatacenterswitchfabrics
Nostandarddefinitionorimplementation
Generally,lowlatencyswitchfabricthatis
nonblocking,permittinganyportto
communicatewithanyotherportatthefull
capacityoftheports
Cloudcomputing,virtualizationandLAN/SAN
convergencearemajordrivingfactors

Traditionalswitcharchitecture

Backupconnectionsinactiveduetospanningtreeprotocols
Connectionsbetweenswitchesareoversubscribed
Asmanyas3hopsbetweenservers

FullMesh

Allswitchesconnectedtoallotherswitchesinfullmesh
Fornonblockingsumofbackbone>=sumofswitchbandwidth
Allconnectionsactive
Notsuitableforlarge#ofswitches

FatTree/Leaf&Spine

Eachaccessswitchconnectedtoeveryinterconnectionswitch
Nonblockingifsumbackbonebandwidth>=sumserverportbandwidth
Allconnectionsactive
Morescalablethanfullmesh
Anotherlayerofinterconnectionswitchescanbeaddedfor
verylargedatacenters

FatTreewithPortExtenders

Sameasfattree,butwithportextendersattopofracks
Maybenonblockingifbandwidthfromaccessswitchto
portextender>=sumofbandwidthofserverports

Datacenterfabrics
Muchmorecablingandhigherbandwidth
neededfordatacenterbackbones
Possibleneedforlongercablerunstoconnect
switches
Fabricscanbebuiltusingthecabling
topologyinANSI/TIA942A
MayneedIDAtoIDA&HDAtoHDAcabling
PointtopointcablinginEDAsshouldbeless
thanorequal10mandwithinacabinetrow

Otherconvergence
Structuredcablingisbeingusedtosupport
otherbuildingsystemsandshouldbe
consideredwhenplanningdatacenters:
IPcameras
Securitysystems
Buildingautomation/monitoring
Possiblyevenlighting

SeeBICSIElectronicsSafety&SecurityDesign
ReferenceManual&TIA862A

Summary
Standardsarebeingupdatedtosupportindustry
changesandnewtechnologies
Considerfutureneedstosupport10/40/100Gto
supportLAN/SANconvergence,datacenterfabrics,
virtualization
Cat6Aorbetter(e.g.Cat7/7A)forbalancedpaircable
OM4formultimodefiber(withLC&MPOconnectors)

Considerenergyefficiencyrecommendations
Buildmodularly (inphases)
Considertheimpactofdatacenterfabrics
Considerothersystems(e.g.cameras&
security)thatcanusestructuredcabling

ISO/IECJTC1SC25WG3Updates
AutomatedInfrastructureManagement(AIM)
addendumto147632(core&optionalAIM
functions)andnewstandard(requirementsfordevice
discovery,securitymanagement,processmgmt &modelfor
dataexchange)

ISO/IEC147633Edition2testingofoptical
fibre cabling
Improvedtestmethods
shouldtestingat1300nmbeoptional?Yes,fornowbut
wemayamendfuturestandardstomakeitoptional

ISO/IECJTC1SC25WG3Updates
ISO/IECTR11801991 40Gbpschannels
40GbpsusingCat6A,7,7A,andnewCat8.1andCat8.2

Newstandardindevelopmentfor
equipotentialbonding basedonANSI/TIA
607BandCENELECEN50310
Twinax cabling(forshortIEEE802.3connections)
3rd EditionofISO/IEC11801&reorganization
ofrelatedstandards(1General,2Offices,3
Industrial,4Homes,5DataCentres)

JonathanJew
PresidentJ&MConsultants,Inc.
Website:www.jandm.com
Email:jew@jandm.com
CochairBICSIdatacentersubcommittee
CoeditorTIA942A,EditorTIA942A1
ViceChairTIATR42.6telecomadministrationsubcommittee
ViceChairTIATR42.1commercialbuildingcabling
EditorISO/IECTR1476321telecomadministrationidentifiers
USNationalCommitteeProjectManagerISO/IEC24764data
centerstandard
DataCenter&AdministrationSectionEditor
ISO/IEC147632cablingplanning&installation