Sie sind auf Seite 1von 14

White Paper

Modernizing the Core With OTN Switching & 40/100G Transport

Prepared by

Sterling Perrin Senior Analyst, Heavy Reading

Prepared by Sterling Perrin Senior Analyst, Heavy Reading On behalf of December 2010

On behalf of

Prepared by Sterling Perrin Senior Analyst, Heavy Reading On behalf of December 2010

December 2010


















Switched Sonet/SDH






Switched OTN



Transport Rate Options


Continue to Add 10G Capacity


Upgrade Infrastructure to 40/100G






40G & 100G Core Transport



OTN Switching






Figure 1: Global Mobile Broadband Connections by Technology


Figure 2: Power Consumption by Element Type


Figure 3: Drivers for 100G Deployments


Figure 4: Expected Client Interface Mix for 100G Core Networks in 2013


Figure 5: The OTN Universal Transport Network


Figure 6: Core Modernization Network Diagram




Faced with continued high growth in IP traffic, large installed bases of TDM services and ele- ments that are not going away anytime soon, and slow revenue growth overall, operators are wrestling with how best to upgrade their networks to handle their network traffic and service de- mands, while remaining viable and competitive business entities. The core network, in particular, is becoming a key focus area for many operators worldwide, following several years of invest- ments in their access and metro networks.

This white paper assesses the key pain points operators face in the core and provides an over- view of the main architectural options available to them. Specifically, we discuss switched Sonet/ SDH networks (the PMO), integrated optics on core routers (IP over DWDM), and switched OTN networking. Heavy Reading makes the case for switched OTN networking, combined with 40- Gbit/s and 100-Gbit/s transport, as the best architectural approach for operator core network modernization.


Market Trends in the Core

Network operators in all regions of the world face a fundamental dilemma. On the one hand, global IP traffic continues to grow rapidly driven by both consumer and business applications. On the other hand, operator revenue is growing at single-digit rates at best and is not keeping pace with network traffic growth.

On the consumer side, demand continues for bandwidth-intensive third-party content, such as over-the-top (OTT) video, peer-to-peer applications, gaming, and HD video. On the enterprise side, customers continue to migrate from TDM private lines, Frame Relay, and ATM to higher- capacity Ethernet services (both private lines and VPNs), wavelength services, and IP VPNs.

Tier 1 ISP NTT Communications reports that IP traffic on its global network has increased at a 75 percent CAGR from 2002 through 2009, with no signs of abating. Other operators that report traf- fic growth statistics, such as AT&T, consistently report their IP traffic growth in the 50+ percent range annually. Meanwhile, these historical statistics do not fully reflect the future growth in ma- chine-to-machine traffic, driven by distributed data centers processing cloud-based applications. Nor do these historical growth rates adequately capture the mobile broadband surge that is just beginning. Today, mobile data's contribution to IP traffic volume is small, but the rapid rise of mo- bile broadband subscribers coupled with the migration from low-speed technologies to advanced 3G (such as HSPA+) and 4G technologies promises to change this picture dramatically.

Pyramid Research forecasts that the number of mobile broadband connections worldwide will reach 1 billion during 2012, with the growth driven by advanced 3G and 4G (see Figure 1). As a result, mobile broadband traffic is set to boom over the next five years, growing three times faster than wireline traffic.

Figure 1: Global Mobile Broadband Connections by Technology

Figure 1: Global Mobile Broadband Connections by Technology Source: Pyramid Research, part of the Light Reading

Source: Pyramid Research, part of the Light Reading Communications Group


Pain Points Requiring the Core to "Modernize"

Operators are faced with several pain points in the core. Growth in IP traffic has led to prolifera- tion of core routers and core router ports processing IP traffic. Many operators have found that a large percentage of traffic moving through core routers is transit traffic that does not need to be processed by the core IP/MPLS router. This traffic is bound for destinations further along in the network, and the transit routers are simply performing a regeneration or grooming role at those intermediate locations.

IP/MPLS router processing of transit traffic is significant for three reasons:

Operators are reporting that as much as 60 percent of traffic passing through core routers is transit traffic bound for other destinations, so the amount of transit traffic in routed net- works is far from trivial.

Core router ports are tremendously expensive, particularly when compared to lower-layer ports, such as Ethernet or Layer 1 OTN.

Aside from capex issues, core routers are tremendous opex consumers in terms of phys- ical rack space, power, and management complexity. Some large operators have re- ported to Heavy Reading that they are simply running out of physical space to house their routers. On power consumption, a core router consumes more power, by far, than any lower-layer network element, giving operators a strong incentive to transport their bits at the lowest layer possible. Figure 2, from Verizon, illustrates this point.

Figure 2: Power Consumption by Element Type

this point. Figure 2: Power Consumption by Element Type Source: Verizon, 2009 • IP traffic continues

Source: Verizon, 2009

IP traffic continues to grow at high rates, most likely around 50 percent per year, based on several industry research sources and anecdotal figures gathered in Heavy Reading network operator interviews. This means that operators are hard pressed to figure out better ways to scale their IP networks more efficiently.

New Metro Ethernet Forum (MEF) Ethernet services are growing in popularity globally. These Ethernet services are currently supported over IP/MPLS infrastructure across the core but it isn't cost-effective to use sophisticated IP/MPLS routers to handle these Layer 2/2.5 services.

Operators are seeing increasing demand for wavelength services and high-capacity pri- vate line services supporting applications, such as video transport, data center connectiv- ity, and low-latency transport. Routed networks cannot effectively support these services.

At the same time, the installed base of Sonet/SDH transport equipment is not up to the task of scaling the packet network. Sonet/SDH was sufficient when network traffic was primarily low- speed TDM with some Ethernet/IP, but that network model has now been reversed. Networks of today – and certainly tomorrow – are primarily packets with some TDM. TDM may still be growing in incumbent operator networks (particularly high-speed TDM), but the growth in packet traffic is far greater.

Forcing packets into Sonet/SDH framing is an inefficient means of transport; yet, until recently, operators have had no other choice but to transport packets over Sonet/SDH to use its manage- ment, resiliency, and reliability functions. Early implementations of carrier Ethernet, MPLS, and pure DWDM transport lacked these abilities.

In addition, there is tremendous operator momentum around 100G transport as the next-gen core transport rate. Yet, the Sonet/SDH standards have been capped at 40G rates (OC768/STM256). Operators migrating to 100G transport must use Ethernet and OTN.


Operator Requirements for the Next-Generation Core

Operators have different approaches to managing their networks. Regardless of whether operators keep packet and transport operations teams separate or converge them into a single group, Heavy Reading believes that operators require the following for their next-generation core networks.

Significant increase in packet capacity (on the order of tenfold improvements every four to five years)

Efficient switching and grooming for both TDM and packet traffic

Greater automation in service and network provisioning and management

Resiliency and reliability to match (or exceed) existing Sonet/SDH network norms

Efficient support of new high-capacity service types

Ability to retain existing Sonet/SDH and core router networks as long as needed

Greater packet intelligence to handle traffic that is predominantly packet


Architectural Approaches Being Evaluated

In order to address the pain points discussed above and to meet their future requirements, opera- tors are evaluating several different architectures for core transport and switching. Often, core transport and switching are being evaluated simultaneously. This section provides an overview of the primary architecture options for the core.

5.1 Switched Sonet/SDH

Standalone optical crossconnects deployed widely in operator networks today are based on Sonet/SDH switching fabrics. These optical crossconnects are effective in networks that are pri- marily Sonet/SDH with some Ethernet, but were not designed for networks that are primarily based on Ethernet and IP. Limited Ethernet functionality has been added to optical crossconnects over time, allowing for Ethernet over Sonet/SDH or a small amount of Ethernet switching, but they fundamentally remain Sonet/SDH network devices. Faced with capacity constraints, operators can upgrade these early-generation Sonet/SDH-based optical crossconnects to newer, high- capacity versions, but Heavy Reading views this more as an interim solution to the packet traffic problem for networks dominated by packet traffic and packet traffic growth.

5.2 IP over DWDM (IPoDWDM)

The integration of DWDM transponders onto routers has been touted by major IP router suppliers as a means of saving capex spend in the core by eliminating separate DWDM equipment sitting next to core routers in central offices. The integration of DWDM long reach optics on routers is typically called IP over DWDM (IPoDWDM). While there have been some IPoDWDM deploy- ments over the last few years, Heavy Reading sees this type of architecture as falling short in solving the operator pain points discussed in Section III.

Although IPoDWDM architecture does reduce transponder capex by eliminating the short-reach optics connection from the router to the DWDM system, disadvantages on capex, opex, and or- ganization levels often outweigh the short-reach optics savings. The major disadvantages of IPoDWDM architectures are discussed below.

As noted earlier, router ports are expensive when compared to switch or transmission ports. Historically, the decline in cost per bit for router ports has lagged in comparison with other networking technologies, notably Ethernet switching and optical transport. This high cost makes the router an excessive cost component for forwarding tandem (transit) traffic across intermediate locations, for example. Thus, for many operators, IPoDWDM solves the wrong capex problem.

In addition, the IPoDWDM architecture ties the operator to the router vendor for both the core router and DWDM transport network. Yet DWDM interfaces on routers have not kept pace with overall DWDM transmission advances over the past couple of years. Technical progress related to increased data rates and new modulation formats have resulted in expensive 10G or 40G router interfaces becoming outdated before their investment has depreciated to an acceptable level. Future wavelength capacity growth beyond 100G will only exacerbate this problem. Several operators have told Heavy Reading that they need to keep their DWDM layer and the core router layers separate specifically for this reason.

Further, the use of alien wavelengths over third-party optical transmission equipment is possible but it is not recommended for high-performance operation. Alien wavelengths typically perform at reduced performance levels because transmitter/receiver and line system configurations are not optimized. Network locations, such as metros, where long distance or high capacities are not required, may be suitable for alien wavelengths; but it must be pointed out that such solutions are not well integrated into a host system's fault management system, and consequently will be more complicated to operate.

Finally, IPoDWDM poses organizational challenges that are not trivial. Historically, opera- tor organizations have been siloed between transport and data. While there is a clear trend toward melding these organizations together, we have seen greater acceptance toward uniting Ethernet and optical transport groups but still keeping IP routing separate. Thus, IPoDWDM forces an organization change that most operators are not yet willing or able to make. In the absence of a unified transport and IP organization, IPoDWDM quick- ly causes internal conflicts. Who is managing the IPoDWDM devices – the transport group or the routing group?

5.3 Switched OTN

OTN is an optical transport standard developed by the ITU-T and first standardized in 1998. It is also known as ITU G.709 and "digital wrapper." In OTN, the ITU defined a payload encapsulation, OAM overhead, forward error correction (FEC), and multiplexing hierarchy. The result is a trans- port standard that includes the benefits of Sonet/SDH (such as resiliency and manageability) but with improvements for transporting data payloads. OTN was specifically designed to be multiser- vice transport container for any type of service from TDM to packet, and this is one of the tech- nology's strongest value propositions.

OTN has been widely deployed for transport within long-haul networks, particularly because the inherent FEC features enable optical transmission over longer distances. A newer trend we are seeing globally is high operator interest in adding large-scale OTN switching – in addition to exist- ing OTN transport – to their next-generation networks. In the next section, we explore the benefits of OTN-switched networks in detail.

5.4 Transport Rate Options

Coupled with the transport and switching technology decisions discussed above, operators must also make key decisions about their future transport rates when building out their next-generation core networks. The options are to continue to base their core networks on 10G or to add 40G and/or 100G to the network. We discuss these transport rate options briefly below.

Continue to Add 10G Capacity

Core networks have standardized on 10G transport. We believe that 10G will continue to play a significant role in networks for many years to come, in no small part because they have come down in price and will continue to decline for many years. However, 10G has been the network standard for more than a decade. Many networks that were well-served by 10G capacities at the beginning of the decade are now straining from years of 50+ percent annual traffic growth. While these operators can continue to add 10G waves to these routes, it becomes operationally com- plex and inefficient to keep doing so (if higher-rate options are available). In addition, growth in 10GE enterprise connectivity drives operators that serve these enterprises to look beyond 10G for transport of these services.

Upgrade Infrastructure to 40/100G

Operators looking to upgrade their core network capacities now have both 40G and 100G as via- ble options. During the last couple of years, 40G transport has moved from the early-adoption phase to mass-deployment phase. Heavy Reading counts more than 110 40G deployments worldwide as of mid 2010, with operators including AT&T, Verizon, Deutsche Telekom, Reliance Globalcom, China Telecom, and many other Tier 1 and 2 operators. In July 2010, the IEEE rati- fied the 40G/100G Ethernet standard for client interfaces. Coupled with the ITU-T's recommenda- tion for 100G transport completed at the end of 2009, this means that 40GE and 100GE transport over 40G and 100G DWDM networks is fully standardized. In fact, initial commercial 100G dep- loyments have begun.


OTN & 40/100G Transport: The Right Approach

Among the options for core network modernization described above, Heavy Reading sees grow- ing operator momentum globally around the combination of 40G and/or 100G transport coupled with a large-scale OTN-switched core. Furthermore, we see operators combining these 40/100G transport and OTN-switching functions within a single core network element. Transport is no longer simple point-to-point static capacity augmentation. It is now a switched networking layer with the flexibility to provide capacity from anywhere to anywhere.

6.1 40G & 100G Core Transport

Although 10G is now a low-cost technology, driven by its maturity and high volumes, for the core in particular many operators around the world have seen an immediate need to move beyond 10G. Figure 3 ranks the primary drivers for deploying 100G, based on a Heavy Reading operator survey of operators around the world. Note that network capacity exhaust ranked second on the list – a key indicator that operators are running out of network capacity.

Figure 3: Drivers for 100G Deployments





Connect 100GE interfaces on core routers



Network capacity exhaust



Better economics compared with 10G



Better economics compared with 40G



Wholesale customer services



Enterprise customer services



OTU4/ODU4 transport services



Source: The Road to 100 Gbit/s Transport Networks: A Heavy Reading Multi-Client Study, June 2009

Significantly, the operator migration is not just about 40G but also about 100G transport, along- side (and sometimes instead of) 40G transport. Verizon, which became the first operator to com- mercially deploy 100G systems in 2009, is just one example of an operator that is pursuing both transport rates.

Given the timing of 40G and 100G and the restraints on carrier spending, a rip-and-replace ap- proach to 40G and/or 100G is not an option. Not surprisingly, in the same Heavy Reading opera- tor survey referenced above, operators listed the need to interoperate with existing DWDM line systems as the top operational requirement in choosing a 100G system. 100G (and 40G) must be able to work on the existing fiber plant and the existing 10G line system.

Modulation techniques based on dual-polarization quadrature phase-shift keying (DPQPSK), combined with digital manipulation of bits through coherent receivers and digital signal proces- sors (DSPs), are enabling 40G and 100G transmission on existing networks built for 10G trans- mission with performance that closely matches that of 10G. For example, at the 2010 OFC confe- rence, Ciena demonstrated 40G ultra-long-haul transmission at 3,100 km and 100G transmission at 1,500 km, all using commercially available products. In addition, DSPs combined with electron- ic dispersion compensation eliminates the need for dispersion compensating fibers, which yield tremendous cost savings, particularly in ultra-long-haul and undersea networks. The technology also reduces the number of amplifiers in the network and allows 40G/100G to work with existing amplifiers deployed. Eliminating dispersion compensating fibers is also important for reducing latency, an increasingly important consideration for enterprises, particularly in finance.

A final key point on higher-speed transport: Line-side migration (i.e., the DWDM equipment) does not mandate a client-side migration (i.e., the router equipment). In other words, operators will mi- grate their core DWDM networks to 40G and 100G but will continue to offer 10G services to cus- tomers and interface to 10G ports on routers for a long time to come. Here, switched OTN, with its ability to groom traffic to efficiently pack pipes, will be a critical requirement. Muxponders (such as 4x10G) will also be used for point-to-point connections, though such non-switched connections are far less efficient at using available bandwidth when compared to mesh-switched OTN.

Figure 4 shows operator expectations for client interfaces in the long-haul core for 2013, based on Heavy Reading's operator survey.

Figure 4: Expected Client Interface Mix for 100G Core Networks in 2013

Expected Client Interface Mix for 100G Core Networks in 2013 N=94 Source: The Road to 100


Source: The Road to 100 Gbit/s Transport Networks: A Heavy Reading Multi-Client Study, June 2009

6.2 OTN Switching

OTN in the core makes sense for several reasons:

Operators envision OTN as a universal transport protocol for all service client types

OTN is designed to transmit 10GE, 40GE, and 100GE transparently

Newer ODU0 and ODUflex standards extend OTN to further sub-wavelength granularity for Gigabit Ethernet and to non-traditional data rates, such as arbitrary-rate Ethernet payloads and Fibre Channel variants

OTN inherently incorporates FEC technology, which is key for transmitting at long-haul and ultra-long-haul distances, as needed in core networks

OTN provides the required OAM functionality that many operators require for transporting GE and 10GE traffic over long distances.

(Note that several of these arguments also apply to metro networks, and Heavy Reading is see- ing growing interest among operators to add OTN to the metro. However, we limit the scope of this paper to core network modernization only.)

Traditionally, operators have relied on Sonet/SDH to provide electrical layer functions of sub- wavelength level grooming, performance monitoring, management, maintenance, protection, and restoration. All of these functions are key requirements for delivering true carrier-class services.

DWDM does not have these capabilities, as these functions can only be done electrically or, at least, are best done electrically. To date, operators have relied primarily on Sonet/SDH to perform these functions, even when the traffic being carried is packet. Thus, operators will transport pack- et over Sonet/SDH over a wavelength, with the Sonet/SDH providing the grooming (assuming optical crossconnects are used), OAM, protection, and restoration. For one, OTN is asynchron- ous, and so does not require the costly and complex timing that Sonet/SDH requires. Yet, OTN is a transparent protocol that can carry asynchronous traffic, such as GE and synchronous traffic, such as Sonet/SDH, and even a mixture of synchronous and asynchronous, without interfering with the OAM of the client traffic itself.

Finally, compared to Sonet/SDH, OTN is better matched to WDM rates, as well as high-speed Ethernet rates, from GE through 100GE. OTN's current defined rates are:

OTU1: 2.7 Gbit/s, for OC48/STM16 transport

OTU2: 10.7 Gbit/s, for OC192/STM64 and 10GE transport

OTU3: 43 Gbit/s, for OC768/STM256 and 40GE transport

OTU4: 112 Gbit/s, for 100GE transport

ODU0: 1.25 Gbit/s, designed to combining multiple GE streams within the OTU-N con- tainers listed above.

Furthermore, newer OTN standard functionality called ODUflex matches OTN containers to any payloads in 1.25-Gbit/s increments (i.e., n x 1.25 Gbit/s), including fitting unusual rates such as 3G HD-SDI, 4G Fibre Channel, 5G InfiniBand, variable rate Ethernet, and 16G Fibre Channel (in the future).

Thus, with OTN operators are finally able to replace Sonet/SDH with a new universal transport protocol that is better tuned to packet data and DWDM. Figure 5 illustrates the concept of OTN as the new universal transport protocol.

Figure 5: The OTN Universal Transport Network

protocol. Figure 5: The OTN Universal Transport Network Source: Ciena and Heavy Reading, 2009 Offloading routers

Source: Ciena and Heavy Reading, 2009

Offloading routers of transit traffic by using WDM and OTN enables operators to reduce capex while freeing up router capacity to handle future growth in IP traffic. It also reduces overall net-

work delay, which is added every time traffic has to pass through an intermediate router. The added delay can be a significant problem for operators, particularly in delay-sensitive video and gaming networks. Finally, router offload brings benefits in reducing packet loss and increased network availability. All router networks struggle to match the five nines (sometimes six nines) availability of an optical transport network.

Figure 6 illustrates in simple terms the new architecture and interconnectivity of a converged Layer 0-2 (40/100G) infrastructure providing connectivity to Layer 3 network (10G evolving to 100GE), and including non-routed services.

Figure 6: Core Modernization Network Diagram

services. Figure 6: Core Modernization Network Diagram © HEAVY READING | DECEMBER 2010 | WHITE PAPER



The core network has become a focal point for network modernization as continued growth in IP traffic has led to proliferation of core routers and core router ports processing IP traffic – routers that are expensive in terms of both capex and opex.

The primary core modernization options available to operators are: to continue to invest in their switched Sonet/SDH networks by adding more capacity; to build out IPoDWDM networks using integrated optics on routers; or to deploy switched packet-optical/OTN networks. Of these op- tions, Heavy Reading sees switched OTN as the best network upgrade approach, particularly for network operators that have large installed bases of Sonet/SDH equipment and services that will need to be accommodated for many years to come, even as they invest in Ethernet and IP.

Switched OTN uniquely combines the "carrier-class" transport qualities of Sonet/SDH with the flexibility to efficiently handle Ethernet transport from GE through 100GE when combined with 40G and/or 100G transport. With switched OTN and 40G/100G transport in the core, operators will finally be able to replace Sonet/SDH with a new transport protocol that is tuned to Ethernet and high-speed DWDM, while simultaneously supporting the legacy Sonet/SDH network and ser- vices for as long as needed.