Beruflich Dokumente
Kultur Dokumente
Standards and
Cabling Options for
Data Center Networks
Data centers present the most opportunity and
greatest challenges for designers, installers and
managers of networks and structured cabling
systems today. Industry standards, combined
with new developments by vendors across the
networking spectrum, provide a technology
roadmap for those who must plan and implement
structured cabling systems in data centers of all
sizes. This Cabling Installation & Maintenance
editorial guide provides information on the latest
standards developments as well as cabling-media
sponsored by:
options, to guide professionals planning longterm support for their data center applications.
approves
10GBase-T
2 TIA
8
942-A data
equipment
center standard
availability
17 Smarter,
greener
infrastructure
for
23 Prospects
all-optical
cabling
The revised standard was several years in development. In May 2010, Cabling
Installation & Maintenance published an article authored by TIA-942-A co-editor
Jonathan Jew, president of J&M Consultants. The article outlined the changes from
the original TIA-942 standard that were anticipated to be included in TIA-942-A.
Recently, Jew summarized for us the changes in the new standard as follows.
Incorporate Addenda to TIA-942:
:: Addendum 1 - Additional coaxial cabling specifications and revised distances
:: A
ddendum 2 - General updates including revised tiering, the addition of
Category 6A, 3-level lighting protocol, revised temperature and humidity limits
Restructuring standards - Integrate into the TIA-568-C series:
:: R
eference generic cabling topology (horizontal = cabling subsystem 1,
distributor C = MC, B = IC, A = HC), terms, and MICE (mechanical ingress,
climatic, electromagnetic) environmental classifications
Cabling Installation & Maintenance :: EDITORIAL GUIDE
Energy efficiency:
:: New section on energy efficiency
:: W
ider range of temperature and humidity (see TIA-568-C) based on new
ASHRAE TC 9.9 guidelines
:: U
se of enclosures or enclosure systems to improve energy efficiency (cabinets
with isolated supply or return, cabinet cooling systems, hot or cold aisle
containment, panels and cable management that reduce air bypass, brushes/
Cabling Installation & Maintenance :: EDITORIAL GUIDE
Recently, Jonathan Jew and TIAs associate vice president of technology and
standards, Herb Congdon, discussed 942-A in a video produced by TIA. Their
Cabling Installation & Maintenance :: EDITORIAL GUIDE
Jew and Congdon also discussed energy efficiency as it relates to data center
Cabling Installation & Maintenance :: EDITORIAL GUIDE
operation. Jew discussed some of the specifications in 942-A that are also listed in
this article, which are aimed at achieving energy efficiency.
Congdon spent some time discussing the Sustainable Technology Environments
Program (STEP), of which TIA is a member.
The video can be viewed at www.tianow.org.
Not a Cabling Installation & Maintenance magazine subscriber? Click here to request a free subscription.
6
Cabling Installation & Maintenance :: EDITORIAL GUIDE
10GBase-T equipment
availability and the future
of copper media
Summarizing 10GBase-Ts ABCs of
availability, benefits and cost.
by Ron Cates, PLX Technology and Valerie Maguire, Siemon
10GBase-T network
equipment is available
and deployment
rates are increasing.
Although initially
hampered by powerhungry implementations,
Cabling Installation & Maintenance :: EDITORIAL GUIDE
todays chip technology that delivers the 10GBase-T bit stream (also called a
PHY) capitalizes on an advanced 40-nm lithography manufacturing process,
which cuts power use, board size and cost. As a result, significant adoption of
10GBase-T technology is expected to begin in 2012. During this year, at least
20 new platforms (e.g. switches, servers and network interface cards [NICs])
using 10GBase-T PHY devices are expected to have broad market availability. In
addition, a new market research report issued by The Linley Group forecasts more
than 2.7 million ports of 10GBase-T PHYs to ship in 2012a sharp rise from the
182,000 ports counted as shipped in 2011. The trend lines shown in the Figure
below depict The Linley Groups forecast for several different types of 10-Gbit/sec
Ethernet applications over the next few years. Note that 10GBase-T is expected to
achieve a dominant market position in 2014. The adoption rates forecasted by The
Linley Group are consistent with the historical Ethernet adoption profile, whereby
optical networking interfaces precede copper interfaces but copper port counts
greatly outnumber optical port counts soon thereafter.
10GBase-T and copper balanced twisted-pair cabling offer unique benefits
compared to other 10-Gbit/sec Ethernet solutions. With cost and power
dissipation significantly reduced with the newer 40-nm PHY devices, and further
reductions enabled by 28-nm devices expected in 2013, data center managers can
now capitalize on the fundamental advantages offered by 10GBase-T technology,
which include the following.
:: T
he ability to interoperate with legacy slower-speed Ethernet technologies
through the function of autoForecasted adoption rate of 10-Gbit/sec
negotiation
Ethernet applications
Port count 1,000
:: T
he ease of deploying a copper
balanced twisted-pair cabling
system and the use of familiar
cabling and connector interfaces
35,000
10GBASE-CR (direct attach assemblies)
30,000
25,000
20,000
15,000
10,000
5,000
0
2010
2011
2012
2013
Adoption year
1203CIMstandardsF1
Cabling Installation & Maintenance :: EDITORIAL GUIDE
2014
2015
:: T
he flexibility of 100-meter, 4-connector structured cabling topologies to
support additions, moves, and changes in local area network (LAN) and data
center environments
:: The ability to deliver Power over Ethernet (PoE and PoE Plus)
10
130nm
65nm
40nm
28nm
Lithography manufacturing process type
The
manufacturing process used to develop
1203CIMstandardsF2
PHYs has an impact on the ultimate power
dissipation of those PHYs. This chart shows
that as the 10GBase-T manufacturing
process is refined (from an original 130-nm
lithography process to a second-generation
62-nm process, then the current 40-nm and
anticipated future 28-nm processes), the Watt/
port power-dissipation level decreases.
11
WoL can take advantage of these demand fluctuations with startling results;
putting even a single server with a typical power dissipation of 500W to sleep
gains much more benefit than the difference in power of hundreds of transceiver
devices. Equally important, 10-Gbit/sec Ethernet deployed over optical media or
Cabling Installation & Maintenance :: EDITORIAL GUIDE
SFP+ direct-attach assemblies is not designed to support the WoL protocol at this
time and, as a result, these systems always dissipate their full power. WoL is
an important strategy employed uniquely by 10GBase-T to reduce overall power
consumption in the data center.
While WoL is designed for lengthy idle periods, another technology called Energy
Efficient Ethernet (EEE) is specifically designed to take advantage of the bursty
nature of computer traffic. It is the case that typical Ethernet traffic contains
many gaps, which can range in duration from microseconds to milliseconds.
Heretofore, these gaps have been filled with so-called idle patterns or
waveforms containing no real computer information, but whose transitions can
be used for maintaining clock synchronization between transceivers. The EEE
algorithm exchanges those idle patterns for Low Power Idle (LPI) mode where very
little power is dissipated.
The LPI mode used during idle periods requires a new signaling scheme
composed of alerts over the line and alerts to and from station management.
When in the LPI mode, a refresh signal is used to keep receiver parameters, such
as timing lock, equalizer coefficients and canceller coefficients, current. These are
also critical to enable fast transitions from LPI to active modes. Typical transition
times from active to LPI mode and back are in the three-microsecond range, so
implementation of EEE introduces minimal latency into the network.
The bottom line is that transceiver power savings using the EEE algorithm
can range from 50 percent to 90 percent depending on actual data patterns.
For example, a 28-nm 10GBase-T transceiver with a typical active mode power
dissipation of 1.5W for 30-meter reach will dissipate only 750W when using the
EEE algorithm with typical computer data patterns. Even better, system-level
EEE optimizations implemented in switches and Ethernet controller silicon are
expected to save far more power than EEE optimizations in the transceiver
because the energy consumption of the entire switch or server (which is more
than double the power per port of even the previous generation of transceivers)
can be leveraged.
12
Equipment
Maintenance
Channel/assembly
0
8,
00
0
7,
00
6,
00
5,
00
4,
00
3,
00
0
1,
00
0
2,
00
0
in
the reduction of overall power dissipation, is the ability to automatically
1203CIMstandardsF3
detect channel length between compliant transceivers. When the channel
length is less than 100 meters, 10GBase-T transceivers are able to reduce their
power dissipation while still maintaining fully compliant bit error rate (BER)
performance. This so-called short-reach mode takes advantage of the larger
signal-to-noise ratios (SNR) present due to lower signal attenuation in short
channels, and the power-dissipation reductions can be dramatic. For example,
because the signal strength at the transceiver is significantly larger if it is
attenuated by only 10 meters of cabling as opposed to 100 meters of cabling,
transmit power can be significantly reduced without adversely affecting BER. It
is a common misperception that short-reach mode is an on/off condition that is
directly tied to a specific link length (e.g. 30 meters). In fact, the short-reach mode
power-dissipation profile is contiguous and scalable versus length.
In short-reach mode, not only can transmit power be reduced, but the number of
filter taps used for echo cancellation and line equalization also can be curtailed
and powered down internally in the device. As an example, a transceiver typically
with 3.5W of power dissipation when connected to a 100-meter channel can
exhibit power dissipation of only 2.5W when connected to a 30-meter channel, or
less than 2W when connected to a 10-meter channel. Because many newer data
center configurations rely on shorter cable lengths than the maximum length of
100 meters, exploiting this feature is growing in importance.
13
selecting media, most designers will assert that cost significantly influences the
decision-making process. The truth is that 10GBase-T offers more benefits and
flexibility than other 10-Gbit/sec applications at the most favorable price point.
The figure on page 13 shows the equipment (server port and NIC), media and
annual maintenance costs for one channel and its corresponding 10-Gbit/sec
port connections, which are representative of the types and lengths of media
commonly deployed in data centers.
The most economical choice for 10-Gbit/sec transmission is 10GBase-T network
equipment in conjunction with Category 6A UTP, Category 6A F/UTP, or Category
7A S/FTP balanced copper twisted-pair cabling. The same conclusion is reached
when this analysis is repeated for channels and their corresponding port
connections that represent the types and lengths of media commonly deployed in
horizontal LAN cabling. It is this cost advantage that will drive the rapid adoption
of 10GBase-T in 2012.
Interest in speeds beyond 10 Gbits/sec over copper balanced twisted-pair
cabling is growing. The most significant confirmation that Base-T Ethernet
applications have a strong future is the growing interest in next-generation
cabling. This media will be targeted to support the copper balanced twisted-pair
application that comes after 10GBase-T. Because Ethernet applications in the LAN
backbone and data center core have always preceded Ethernet specifications for
the LAN horizontal and data center edge, it is a good bet that the next Ethernet
over balanced twisted-pair speed will be 40 Gbits/sec to supplement IEEE
802.3ba-compliant 40-Gbit/sec Ethernet computer backplanes and optical-fiber
network gear. At this time, the biggest driver demonstrating the great industry
commitment to, interest in and investment in the future of copper-based Ethernet
is the work being done by the ISO/IEC and TIA to develop next-generation cabling
specifications to support such an application.
14
15
Cabling Installation & Maintenance :: EDITORIAL GUIDE
RHINO4200LabelPrinter
17
improve power management and facilitate space savings. Among these are the
use of Energy Star servers and high-efficiency power distribution units (PDUs).
Too often overlooked, however, are the physical-layer components within the data
centers networking infrastructure and, arguably, these are equally important.
Essentially, there are three efficiencies to strive for in greening the data center:
cooling, power management and density. This article explains why building owners
and data center managers need to select the information and communication
networks physical layer components with optimal performance, efficiency and
sustainability in mind.
Cooling and power-management challenges
Respondents to a recent survey from Gartner revealed that they believe power
and cooling to be the top data center challenges. Todays active equipment not
only consumes more operating power, but also must be kept cool. In fact, it has
been estimated that cooling accounts for up to 30 percent of a data centers
energy load.
However, the cooling process itself is typically accompanied by excessive energy
waste as computer room air conditioner (CRAC) units attempt to compensate
for hot spots. Unfortunately, the result of this compensation means that an
enormous amount of energyand moneyis wasted.
Most data centers use a hot-aisle/cold-aisle configuration to help contain
hot air generated by the equipment. However, to take full advantage of this
configuration, you need to find solutions that work with equipment enclosures to
help separate the hot air coming out of the back of the equipment from the cold
air entering the front. Such heat-containment solutions significantly reduce the
mixing of hot and cold air, which increases the differential between the inlet and
outlet temperatures of a CRAC unit for better efficiency and energy savings.
18
For example, there are heat-containment devices that mount atop enclosures
and automatically monitor heat loads, and regulate fan speeds accordingly to
pull the right amount of air from the enclosure for return to the CRAC unit via
plenum spaces. This type of system has been shown to deliver up to 50 percent
energy savings when compared to hot-aisle/cold-aisle configurations. And they
Cabling Installation & Maintenance :: EDITORIAL GUIDE
This figure provided by Belden estimates the airflow-capacity savings that can be
achieved using its AEHC Airflow Management System.
19
Obviously, the more square footage you have, the more materials required
and the higher the energy requirements, so conserving space in the data
center wherever practical is desirable. Areas such as the main and horizontal
distribution rooms/areas house power-consuming core switches, as well as
hundreds of interconnects and crossconnects for connecting to equipment and
Cabling Installation & Maintenance :: EDITORIAL GUIDE
Going hand-in-hand with airflow-capacity savings, Belden says, are annual coolingcost savings when using its airflow management system. Of course actual savings
will vary depending on the data centers specific characteristics.
20
bundled, factory-terminated
cable assemblies provide
a smaller diameter for
pathway-space conservation,
and are also faster and easier
to install and maintain, thus
saving on labor expense.
Furthermore, preterminated
solutions allow for use of
short links, which for some
data centers can result in
a material reduction of up
to 40 percent in the overall
cabling topology. And
preterminated cabling is a
cost-saver over the long term
in that cables can be reused
and quickly repurposed to
accommodate moves, adds
and changes required in the
future.
Sustainable
infrastructure design
21
more bandwidth. Going green and becoming more sustainable goes far beyond
selecting and using environmentally friendly materials and components. Its
about working smarter and more efficiently across all enterprise systems and
operations, which brings us to the concept of Internet Protocol (IP) convergence.
In recent years, Ethernet and IP have advanced to where the technology can now
be used to transmit voice, video, security, and industrial control and buildingmanagement information as data signals across the enterprise, using a shared,
common network protocol.
In some enterprises, Ethernet IP convergence is being adopted to replace a
number of legacy proprietary systems. The beauty of the converged-systems
approach is that it allows multiple systems to run as an open network, using
the same infrastructure and cabling media, with security and authentication
measures built in to protect and secure confidential data. IP convergence is a
trend that likely will grow over time because it serves to increase efficiency,
reduce costs and waste, and maximize resource utilization. And, in the end, isnt
that what sustainability is all about?
Not a Cabling Installation & Maintenance magazine subscriber? Click here to request a free subscription.
22
Cabling Installation & Maintenance :: EDITORIAL GUIDE
23
One of them
problem status,
Building automation
among other
Healthcare
things, due to
Residential
24
It goes without saying that, providing variable conditions of actual projects one
can observe in practice, there is no perfect cable type for every occasion. The
evidence of this simple statement emanates from the fact that there are several
competing standard types of products in the market. In this regard the cabletype selection problem is getting noticeably complicated and actually comes
down to answers to the two largely complementary questions: 1) Does a certain
structure have any advantages under the given specific conditions? and 2) Do its
disadvantages under the given conditions have any significance?
25
Specific computations show that the viability border for transition from
multimode to singlemode technology lies within the range of approximately 600
meters (2,000 feet) to 1,200 meters (4,000 feet); while with the transmission rate
Cabling Installation & Maintenance :: EDITORIAL GUIDE
increases, the critical length shows a reducing trend. The latter results from both
the higher values of higher-data-rate electronics and noticeably growing adoption
of Om3 and Om4 multimode optical fibers.
Data centers impact
Other factors may have a significant impact on component-type selection,
and it is worth taking account of these factors. As an example, data center
environments have grown rapidly in recent years, and top the list of factors
that determine technology directions and may significantly impact component
selection.
The following objective factors have a significant impact on the application scope
of optical technologies in data centers.
A) A data center as a technical object does not provide services to end users, i.e.
it lacks well-known man-machine system restrictions, primarily conditioning the
ultimate data transmission lines.
B) Provision of operational support for the enterprise telephone network is not
the top-priority objective in a data center, and thus, there is little or no demand
for PoE technology. In this case the task of supply voltage delivery to the active
networking equipment is resolved by completely different means.
C) Optical-fiber cables have much better weight-dimensional parameters when
compared to balanced twisted-pair cables, facilitating the creation of a cooling
system of active networking equipment.
D) When operated on long links, optical-fiber interfaces provide for much lower
power consumption, i.e. building of green or energy-efficient data centers
becomes easier.
26
Advantages
Disadvantages
Further analysis, however, causes us also to take into consideration the several
additional factors. First, based on statistical data it should be concluded that
owing to the focal application of a data center and its purpose, an expected
market size of this type of structured cabling shall not exceed one-third of the
office structured cabling market.
Second, in an earlier-published article (see Considerations to make before
investing in Category 6A cabling) we provided statistical data on some completed
projects, showing that the average horizontal subsystem link length in a data
center does not exceed 30 meters (100 feet), which is explained by the small
size of this object owing to the natural intention to save an expensive area with
complex engineering stuffing (uninterruptible power supply systems, precision
air-conditioning/cooling systems, and others). The network interface based on
balanced twisted-pair cable functions in a short-reach mode at such distances.
It means that the length of most links is such that the optical technologies have
no significant power-consumption advantages.
Third, 40-Gbit/sec transmission data rate is highly demanded in the data
center, with the focal application area of server-to-server links. Currently such
transmission data rates may be confidently implemented on modern twisted-pair
equipment, which still is less expensive than the optical equipment. Actually,
until recently the lack of official standards was the basic restraining factor for its
practical application.
Thus, despite a larger possible application niche for optical solutions in a data
center, the classical office structured cabling shall remain unsurpassed either
now or in the foreseeable future in terms of the application scope.
Access networks
Access networks must be considered within the scope of this discussion, because
they seemingly link up with the enterprise IT system owing to the current
predominance of Ethernet technology.
27
28
29
Cabling Installation & Maintenance :: EDITORIAL GUIDE
links:
30