White Paper

Understanding Intelligent Carrier Ethernet

Bringing the Advantages of Ethernet to the Service Provider
Abstract

Ethernet has become the most successful and widely deployed LAN transport technology
in the world. While other technologies such as Token Ring and Fiber Distributed Data
Interface (FDDI) have become obsolete, Ethernet has more than 100 million clients deployed
today, making it the standard interface for most network-capable devices.
The last 20 years have seen significant development of Ethernet technology from 10-Mbps
shared operation over thick yellow cable, to switched operation over unshielded twisted
pair (UTP) at 10, 100, and 1000 Mbps. However, the most significant development from a
wide-area networking perspective has been fiber optic transmission at 100 Mbps, 1 Gbps,
and 10 Gbps at transmission distances from 2 kilometers (km) up to 2000 km using
long-haul dense wavelength-division multiplexing (DWDM) systems.

In parallel to these developments, Ethernet switching technologies have evolved from simple
2-port devices with forwarding rates typically in the thousands of packets per second to
today’s modern switching systems that support high port densities, forwarding rates in the
millions of packets per second, and sophisticated wire-rate services. Additionally, the
economics of Ethernet make it an attractive proposition as a WAN transport for service
providers.

Another advantage of Ethernet as a transport is its support of class of service (CoS) that
allows up to eight classes of service to be defined. From an enterprise perspective, the
ubiquitous nature and sympathetic CoS characteristics of Ethernet make Ethernet as a WAN
technology very attractive because the Ethernet WAN can be seen as an extension of the
campus LAN.

Ethernet uniquely supports true multipoint communications. Most WAN technologies such
as Frame Relay or ATM offer point-to-point connectivity only, which are complex to
configure as more connections are added. If a Virtual Private LAN Service (VPLS) is used, its
inherent broadcast nature makes newly added locations ready to use because the routing
protocol will automatically detect neighboring devices and form routing adjacencies.
Although Metro Ethernet is often thought to be analogous to Transparent LAN Services,
Ethernet can be used as a broadband access technology to point-to-point Layer 2 VPN,
Layer 3 VPN, and Internet services.

For many of these same reasons, service providers are beginning to consider Ethernet a core
technology, if not a core service, for their WAN and metropolitan-area network (metro)
service offerings as well as a mechanism for offering numerous other services. When coupled

Cisco Systems, Inc.

All contents are Copyright © 1992–2003 Cisco Systems, Inc. All rights reserved. Important Notices and Privacy Statement.
Page 1 of 21
 with existing, well-known technologies such as SONET/SDH, IP, and Multiprotocol Label Switching (MPLS),
Ethernet has tremendous potential for providing high-bandwidth connectivity and service across the range of service
provider offerings.

This paper will introduce Ethernet as a metro networking technology and describe the various technologies and
services that can be enabled over an Ethernet infrastructure, including integration with established or emerging
technologies such as Resilient Packet Ring (RPR) and SONET/SDH.

Demands for Ethernet in the Service Provider Network

Most, if not all, service providers are looking closely at Ethernet as a technology within their service networks.
For example, incumbent local exchange carriers (ILECs); post, telephone, and telegraphs (PTTs); and interexchange
carriers (IXCs) are considering Ethernet, either as a pure Layer 2 transport mechanism, or for offering IP VPN
services to complement their existing Frame Relay or ATMs services. A number of independent operating companies
(IOCs) are using Ethernet as a broadband technology for delivery of multiple services to the residential customer.
Additionally, there is an increasing trend involving the use of Ethernet as a backhaul instead of ATM for xDSL
applications. Cable companies themselves are using Ethernet as a means of aggregating cable headend systems. These
applications will be further discussed later in this paper.

But it is first important to understand what is encouraging the interest and demand for Ethernet in the service
provider space. These factors can be summarized as:
• Incremental services
• Bandwidth
• Availability of fiber
• The enabling applications

Incremental Services

Ethernet is of interest to the service provider only if it provides a means for incremental revenue. In the past, Ethernet
was offered only as a custom service to large customers who had the requirement and were willing to pay for it. Now,
service providers are looking to Ethernet as a core transition medium for three main reasons: to enable new services,
to align business objectives and network infrastructure, and to customize the network for the end customer.

Ethernet first gained traction in the service provider space in 2001 due to the increased demand for transparent LAN
service. This service, while originally viewed as a niche service, has recently seen an upswing. Ethernet, by its nature,
allows a provider to offer this service, incremental to existing leased line or Frame Relay service, for additional
revenue. Another benefit of Ethernet, particularly in tandem with MPLS, is the ability to converge multiple services
onto a common transport medium. This is one of the primary benefits of a connectionless service interface as opposed
to the current connection-based, point-to-point services offered today.

The speed and bandwidth characteristics of Ethernet (discussed in the next section) allow the service provider to offer
incremental and customized services more easily than previously possible. For instance, a particular end customer
might be looking for a voice service as well as a point-to-multipoint capability for video. The provider could offer a
voice service, while at the same time taking advantage of the broadcast nature of video and the intelligence of IP to
offer multicast video. All of this is run over a high-bandwidth infrastructure, helping mitigate the need for very
granular quality of service (QoS).

Cisco Systems, Inc.

All contents are Copyright © 1992–2003 Cisco Systems, Inc. All rights reserved. Important Notices and Privacy Statement.
Page 2 of 21
 Bandwidth-Scaling Characteristics

One of the clear advantages of Ethernet is its ability to scale in bandwidth and speed. Whereas Ethernet began its life
as a shared 10-Mbps medium over thick coaxial cable, it can now scale to switched 10-Gbps per link over fiber, with
100 Mbps and 1 Gbps as interim steps. As Internet and Web-hosting traffic continue to grow, the ability to scale
bandwidth on a flexible platform, such as the Cisco® Catalyst® 6500 Series switches or Cisco 12000 Series routers,
becomes extremely attractive. For example, a 100-Mbps connection to a data center server can be easily upgraded
to a 1-Gbps connection, should that bandwidth be required. This is important because large networking servers can
now manage 100 Mbps to 1 Gbps of sustained throughput. Similarly, a network architect might also consider
upgrading the backbone links to 10 Gigabits Ethernet, something that can be done easily without costly equipment
upgrades or SONET ring resizing.
In addition, the costs for Ethernet interfaces are often considerably less than those for more traditional technologies
such as packet over SONET (POS). Ethernet lacks some POS characteristics, such as 50-milliseconds (ms) link
restoration, that make POS so attractive to a service provider. However, many providers are willing to consider
Ethernet as a lower-cost means of connectivity, particularly if a SONET/SDH infrastructure is not already in place.

Product Availability

Another trend that is clearly helping both Ethernet and IP is the availability of Ethernet interfaces on a wider variety
of equipment. Whereas five years ago, Ethernet could only be found on network interface cards (NICs), hubs, and
switches, it is now found, in relatively high densities, across most switching and routing platforms. What is even more
interesting are the other devices that can now connect to an Ethernet and/or IP network. These include storage arrays,
WiFi access points, cable headend equipment, gaming equipment, and cameras.

The cost of Ethernet interfaces is dropping dramatically as they become more integrated into the end-user devices.
Most PCs now come with an onboard 100-Mbps or 10/100/1000 interface. NICs can be purchased inexpensively.
An Ethernet interface on anything from a Sony Playstation to a video-on-demand quadrature amplitude
modulation (QAM) modulator can be included easily and at little cost to the equipment manufacturer. The sheer
availability of Ethernet is increasing bandwidth for the enterprise and home user, thereby requiring the service
provider to increase its own bandwidth.

Availability of Fiber

Ethernet is designed to run over two media types: copper and fiber. Ethernet is usually sent over copper cable because
the vast majority of Ethernet connections are from a switch to an end station. Longer-distance Ethernet transmissions
at gigabit or tens-of-gigabits rates require fiber-optic cable. Fortunately for the service provider, much of that fiber
already exists. In the 1990s many service providers, utilities, and government entities began trenching and laying fiber
into the ground. While the expected demand for that fiber has not grown as quickly as expected, the infrastructure,
to a large extent, exists and is waiting to be used.

While much of this fiber has been “lit up” with time-division multiplexing (TDM) or private-line access to their
customer base, many service providers have a large portion of dark fiber waiting to be used. Many providers are
looking to this fiber to carry incremental service—this is where Ethernet broadband, often referred to as Ethernet to
the X (where “X” is “business” or “residential”), and Metro Ethernet can be deployed to offer new and innovative
high-bandwidth services. Many providers have fiber access to many large business parks and some residential areas
(though a residential offering is more common in high-density residential areas featuring many apartments). There

Cisco Systems, Inc.

All contents are Copyright © 1992–2003 Cisco Systems, Inc. All rights reserved. Important Notices and Privacy Statement.
Page 3 of 21
 is still a challenge, though, because fiber is not yet universal and offering fiber directly to the business can often
involve additional trenching. However, the availability of fiber in the collector rings, backbone rings, and long haul
has made Ethernet an attractive transmission medium.

The use of copper for Ethernet service delivery is becoming increasingly attractive, particularly considering the sheer
volume of copper already in the ground. Copper today provides the majority of last-mile technology for telephone
and serial WAN connections, both for residential and business services. Fortunately, innovations in Ethernet, as well
its usage in retrofitting existing technologies, have helped enable copper access to Ethernet networks. Technology
such as Ethernet over very-high-bit-rate DSL (VDSL) is one mechanism that has been deployed in Europe and in
parts of the United States today. In addition, other technologies such as Digital Subscriber Line Access
Multiplexers (DSLAMs) with Ethernet uplinks and asymmetric DSL2 (ADSL2) are also aiding in the adoption of
Ethernet in the last mile.

The Enabling Applications

Many enterprise network managers are seeing their data volume dramatically increase. This is due to a number of
reasons, such as more peer-to-peer traffic, larger file transfers (either FTP or e-mail), and simply more use of the
Internet as a business tool. Another, arguably more relevant factor that contributes to higher bandwidth, even though
the data volumes may be low, is application transaction delay. Some modern applications actually require transaction
delays in the 50-ms range that can only be delivered using high-speed transport. However, these applications will
typically not compel a provider to dramatically rearchitect its network; instead, they will change the way in which
services including bandwidth are delivered. The providers, understandably, are looking for the “next big thing.” The
emerging applications may tip the balance in the service provider networks toward higher bandwidth and Ethernet.

Storage
Enterprises are now more dependant than ever on their data, both transactional and backup. This has made storage
networking a critical business requirement. While many transactional storage applications require synchronous
storage using dark fiber or DWDM, many backup, network-attached storage (NAS) and disaster-recovery solutions
can make excellent use of Ethernet as a transmission medium in the metro between data centers. Service providers
are beginning to notice this opportunity because it not only requires bandwidth, but a guaranteed service-level
agreement (SLA), something that they can charge back to their customers.

Video
Video providers, such as cable companies, are looking for less expensive ways of delivering data to their end users.
As many of these cable headend devices become Ethernet-attached to the aggregation layer of the network, more
bandwidth is required to transfer these now-digital files. In addition, multiple video streams per link can dramatically
increase bandwidth requirements, thereby generating more interest in cost-effective transmission. Ethernet’s
simplicity and affordability is appealing to video and cable providers.

Triple Play (Voice, Video, and Data)

Many service providers, particularly cable companies and IOCs, are seeking entry into existing markets by offering
more innovative services. Cable companies and IOCs typically have access to large amounts of fiber, enabling them
to offer broadband Ethernet service. By providing 10 Mbps to each residence, they can use that connection to offer
voice services, cable television (video) services, and Internet access. With bandwidth once again a factor, Ethernet
becomes a straightforward technology choice for deployment.

Cisco Systems, Inc.

All contents are Copyright © 1992–2003 Cisco Systems, Inc. All rights reserved. Important Notices and Privacy Statement.
Page 4 of 21
 Additionally, the advent of high-bandwidth ADSL2 that can support voice, video, and data will likely demand a
high-bandwidth Ethernet backhaul instead of the more traditional ATM backhaul networks. Given the copper
footprint in many markets, ADSL2 and Ethernet provide a complimentary solution to deliver high-bandwidth
services over copper and fiber.

The Cisco metro solution delivers the most comprehensive multilayer service portfolio for providers to quickly scale
customers and revenue. This portfolio can enable high-value services at any layer while providing a growth path to
a broader and more profitable service offering. Cisco Systems® has developed its metro solution to tightly link into
enterprise environments that require integrated voice, video, and content applications. This integration allows service
providers to link to enterprise applications based on Cisco AVVID (Architecture for Voice, Video and Integrated
Data). The perspectives provided in this paper come from the considerable experience and success Cisco has achieved
in helping enterprise companies design and deploy their converged data networks.

Standards Development

Another important reason for Ethernet in the service provider space has been the activity within the standards bodies.
In 2001, the Metro Ethernet Forum (MEF) group was formed with the express mission of furthering the adoption
of service provider Ethernet, and Ethernet technologies in general, in the WAN space. Specifically, MEF has been
looking at the Ethernet services and service definitions, gathering agreement between Ethernet vendors and service
providers about how a particular service should behave. This allows Ethernet vendors and service providers to
best understand what the service will look like to the end customer. Finally, MEF has been evaluating carrier-class
Ethernet. This initiative seeks to specify the architecture, protocols, and management for Ethernet-based
transport networks.

MEF has been an excellent forum for discussion of these services and capabilities. It is the standards bodies, however,
such as the IETF, the International Telecommunication Union (ITU), and the IEEE, that have begun work on the
standardization of these particular protocols. The IETF has been specifically involved in the development of a
multipoint Ethernet service architecture called Virtual Private LAN Service (VPLS) and Virtual Private Wire
Service (VPWS). The IEEE, which governs Ethernet and bridging standards with the 802.3 and 802.1 committees,
respectively, has been working on standards such as Resilient Packet Ring (RPR) (with the 802.17 committee) and
Provider Bridges (802.1ad).

Cisco has also been heavily involved in the “Ethernet over Transport Architecture” initiative. The original initiative
created the “G.etna” draft, which is now further segmented into G.etoa, G.ethna, G.esm, and G.ethsrv. The
following work items are under study group:
• G.eota (Ethernet over transport architecture)
• G.ethna (Ethernet-layer network architecture)
• G.esm (Ethernet over transport Ethernet service multiplexing)
• G.ethsrv (Ethernet over transport service characteristics)
• G.smc (service management channel private line)
• G.enni (Ethernet over transport network interface)
• G.euni (Ethernet over transport user interface)
• G.eequ (Ethernet equipment)

Cisco Systems, Inc.

All contents are Copyright © 1992–2003 Cisco Systems, Inc. All rights reserved. Important Notices and Privacy Statement.
Page 5 of 21
 The standards effort, and the participation and leadership of Cisco, are accelerating the adoption of Ethernet as a
metro and WAN technology. As service providers see the commitment to making Ethernet “carrier-class” and its
availability to the enterprise, the interest in and use of Ethernet is expected to significantly increase.

Transporting Ethernet in a Service Provider Network

The service provider environment is very diverse; there are numerous technologies and protocols available for use in
service creation and service delivery. The first step to understanding how Ethernet as a technology plays in the metro
is to understand how it fits into the general technology scheme. Figure 1 shows the hierarchy and interrelation
between the metro technologies.

Figure 1
Ethernet Related to Service Provider Technologies

IP / MPLS Control

Resilient
Packet SONET / SDH Switched
Ring Ethernet

CWDM / DWDM

Fiber

As stated earlier, the critical enabler for Ethernet as a transmission media is the availability of fiber. For that reason,
fiber is shown as the foundation for all the technologies in Figure 1. How that fiber is used, meaning what
transmission technology is implemented, depends on a number of factors, such as installed base, how the fiber is laid
out, density of customers in a particular area, and the service being offered.

At this juncture, it is very important to point out that “metro Ethernet” or “service provider Ethernet” does not
necessarily imply that Ethernet is used end-to-end. As Figure 1 shows, Ethernet is one possible option as a transport
medium; however, Ethernet can also be run over SONET or RPR. This is a critical distinction in the discussion of
service provider Ethernet: Ethernet as the customer User-Network Interface (UNI) versus Ethernet as the transport.
As the Figure 1 demonstrates, both are possible.

Ethernet in the First Mile and Access into the Network

Access into the service provider network can also be provided via Ethernet, either from a business or a residence.
This technology is generally referred to as Ethernet in the First Mile (EFM). The media choices are straightforward:
There is either copper access, such as existing telephone wire or Category 5 unshielded twisted pair, or fiber.
Many multi-dwelling units have copper wiring that has been in existence for quite some time. The ability to make
use of this for Ethernet is one of the main attractions of Long-Reach Ethernet (LRE). LRE, which is offered on the
Cisco Catalyst 2950 LRE Series switches, supports between 5 and 15 Mbps of Ethernet transmission over existing
voice-grade cabling with a drive distance of up to 5000 feet, making it an ideal solution for in-building connectivity

Cisco Systems, Inc.

All contents are Copyright © 1992–2003 Cisco Systems, Inc. All rights reserved. Important Notices and Privacy Statement.
Page 6 of 21
 over traditional copper. It is important to point out, though, that the current distance supported makes LRE
problematic for exterior copper access from the central office (CO) to the residence. Cisco is working with the
standards committees, including the 802.3ah Ethernet First Mile group, to develop solutions for longer-range
Ethernet.

Fiber cabling to the residence or multi-dwelling unit simplifies the distance and bandwidth limitations that are found
in LRE and xDSL. As stated earlier, the amount of fiber deployed every year by service providers, utilities, and local
governments continues to increase year after year. High-density fiber aggregation devices are required to support such
technologies as 1000Base-LX and 100Base-LX10. Cisco offers solutions for both “ends” of the fiber. The Cisco
ONT 1031 Media Converter provides customer-premises fiber termination. This serves as a demarcation between
the service provider network and customer network with environmental hardening for outside placement, if required.
It provides one 1000BASE-LX Gigabit Ethernet port for the service provider and one 10/100/1000BASE-T copper
port for connectivity to the end-customer network.

Cisco Catalyst switches serve as the “end point” for customer aggregation, either residential or business. For
example, the Cisco Catalyst 2950 and Catalyst 3550 series switches provide a low-cost customer premises
equipment (CPE) device to aggregate lower-density (less than 48 customers) 100Base-TX or LRE-connected users
within a building. The Cisco Catalyst 4500 Series switches are optimized for residential aggregation, featuring
high-density 1000Base-LX and 100Base-LX10 line cards that provide aggregation for many customers onto a single
platform, where intelligent services can then be applied. Many providers offering business services use Cisco Catalyst
6500 Series switches and Cisco 7600 Series routers as the aggregation and/or POP device to provide VPN service,
QoS, and advanced services.

Beyond the Access—Understanding the Physical Topology

The existing fiber plant a provider has access to will determine almost everything about the physical transmission
medium used. Unlike enterprise fiber plants, a service provider fiber plant cannot be installed, removed, or altered
without a major effort, one that usually involves pulling fiber and the associated trenching that may also incur
substantial costs. For that reason, a fiber plant is installed with the intention to make no alterations other than to
provide access to the customer site itself. It is important to understand that the logical topology and the physical
topology will differ significantly because traditional transmission technologies such as SONET/SDH rings rely on the
physical layout of the fiber plant.

Cisco Systems, Inc.

All contents are Copyright © 1992–2003 Cisco Systems, Inc. All rights reserved. Important Notices and Privacy Statement.
Page 7 of 21
 Figure 2
Sample Service Provider Fiber and SONET/SDH Layout

Main HQ

Branch
Office

Figure 2 shows the typical physical topology of a service provider’s network. The topology is laid out in a ring
configuration with three distinct tiers. The access ring provides access out to the customer premises. In many cases,
an access ring will support only a single customer, although there are cases where many customers could be supported
per ring, depending on the bandwidth allocated per access ring. The access rings are terminated at ring collector
nodes, which are interconnected via the access-collector ring. The access-collector ring aggregates multiple access
rings into a single fiber distribution frame. Finally, these access-collector rings are aggregated at a CO point of
presence (POP). Each CO POP is typically interconnected over a high-speed (OC-48 or 192 ring) inter-office and/or
regional ring.

The physical topology determines a number of important factors, each of which affects the design of the Ethernet
service network. These factors include:
• Amount of fiber available—In some cases, a provider has installed plenty of fiber cores. This means that some of
the cores can be used for traditional TDM traffic while others can be “sold” to the data portion of the provider
as dark fiber.
• Section of the business—The transmission group in a service provider may be the group offering the Ethernet
service. In that case, they almost always want to use the infrastructure already there, namely SONET/SDH. The
service offered will most likely be an Ethernet private line with an end-to-end guaranteed SLA.
• Customer density—If there are few customers, it might make more sense to multiplex Ethernet over an OC-3
circuit than to run Gigabit Ethernet to each customer. Conversely, if there are numerous customers, using Ethernet
switching over dark fiber might make the most sense.

The question now arises as to what technologies are available to offer Ethernet over the transmission medium. Once
the provider has decided on the service offering and understands the implications on the transmission network, the
options can be considered.

Cisco Systems, Inc.

All contents are Copyright © 1992–2003 Cisco Systems, Inc. All rights reserved. Important Notices and Privacy Statement.
Page 8 of 21
 Scaling Fiber with Wavelength-Division Multiplexing

Wavelength-division multiplexing (WDM) technology uses the physical characteristics of light to multiplex
wavelengths (lambdas), or streams of light, at different frequencies on the same fiber. There are two main types
of WDM technology: coarse and dense. Coarse WDM, or CWDM, uses wider spacing between frequency bands,
allowing for fewer wavelengths, but at a lower cost because the granularity of the lasers doesn’t have to be tightly
controlled. Dense WDM, or DWDM, is more sophisticated and, with a smaller spacing between wavelengths, allows
for more lambdas on the same fiber.

CWDM and DWDM Gigabit Interface Converters

Cisco supports CWDM and DWDM technology in a gigabit interface converter (GBIC) form factor. This technology
brings many of the benefits of WDM technology, such as fiber savings and bandwidth multiplexing, together with
the simplicity of Ethernet. The CWDM GBICs can be inserted into any standard GBIC slot on a Cisco Catalyst
switching platform or GBIC-enabled router (such as the Cisco 7600 Series). Each GBIC is tuned to a wavelength on
the C-band (1550 nanometers spectrum); because eight wavelengths are supported, there are eight individual GBICs,
each tuned to a specific wavelength. Each wavelength is capable of sending a full 1 Gbps in each direction. For the
DWDM GBICs, up to 32 wavelengths are supported, using the 100-gigahertz grid. Unlike the CWDM GBICs, these
wavelengths can be amplified.

The second component of the CWDM solution is the optical add/drop multiplexer (OADM). These OADMs are
designed to take in the “colored” light from the GBIC and multiplex it with the other wavelengths over the trunk.
Each site would have an OADM, either single lambda or multilambda, depending on the number of wavelengths
added or dropped. Figure 3 shows a typical CWDM Ethernet ring configuration. For the DWDM GBICs, the
Cisco ONS 15216 Metropolitan/Regional DWDM System can be used for the add/drop component. Either way, the
capability and architecture of the network is the same.

Figure 3
CWDM/DWDM Ethernet Ring

Multiple l
Passive Single l
CWDM Multiplexer Passive
(Headend) CWDM Multiplexer

0/1
2/1
0/2

2/2

East-Facing GBIC
West-Facing GBIC

Because each wavelength is essentially a point-to-point link, the logical topology is a point-to-point network across
a physical ring. This is much like an Ethernet over SONET deployment in that the physical and logical topologies
vary. Redundancy within CWDM is handled by the edge devices because CWDM simply provides virtual
point-to-point Ethernet connections across the physical-ring topology. The edge devices may use 802.1 Spanning Tree

Cisco Systems, Inc.

All contents are Copyright © 1992–2003 Cisco Systems, Inc. All rights reserved. Important Notices and Privacy Statement.
Page 9 of 21
 Protocol, 802.3ad Cisco EtherChannel® technology, or an IP routing protocol to provide redundancy. Each port
in the EtherChannel device is configured with a CWDM GBIC on the same wavelength; one port on the channel is
heading in the “east” direction on the ring, the other in the “west” direction. Thus, if there is a fiber cut somewhere
on the ring, EtherChannel technology will simply continue forwarding on the active port in the direction where
connectivity still exists.
EtherChannel failover has been consistently measured at 200 milliseconds, considerably better than the 50
seconds with Spanning Tree Protocol [802.1d] and even the 2 seconds measured with Rapid Spanning Tree
Protocol [802.1w].

DWDM
DWDM is a far more robust technology, providing more wavelengths than a CWDM solution. This enables a service
provider to better use the fiber plant by increasing the bandwidth potential of the installed fiber. DWDM is also
amplifiable to provide greater drive distances between nodes. Most DWDM solutions, including the Cisco ONS
15530 DWDM Multiservice Aggregation Platform and Cisco ONS 15540 ESP and ESPx extended services
platforms, provide support for up to 32 wavelengths, with each wavelength capable of supporting a single 10-Gbps
stream, multiplexed 1-Gbps stream, or Fibre Channel traffic. Figure 4 shows how DWDM can be used in a service
provider network to support Ethernet.

Figure 4
DWDM in a Service Provider Environment

GRID Computing
Facility
Service
Provider
POP
xDSL

ATM

Frame
Relay
Si

Super Computing
Metro
Facility
DWDM
32 x 10 GE
Wavelengths

Systems that support DWDM are typically more complex. To design a scalable DWDM network, the network
architect must consider the dispersion and amplification characteristics of the network due to fiber as well as distance.
While CWDM has similar issues, only a few frequencies in the C-band can be amplified. In DWDM, amplification
and dispersion compensation must be strategically placed throughout the network to ensure signal integrity.

Cisco Systems, Inc.

All contents are Copyright © 1992–2003 Cisco Systems, Inc. All rights reserved. Important Notices and Privacy Statement.
Page 10 of 21
 Unlike CWDM, which uses the attached devices to provide the failure-recovery mechanism, DWDM provides these
mechanisms natively within the system. Using mechanisms such as splitter protection, which provides line card
protection with a special Y-cable, a DWDM system can reroute around a ring failure in 50 ms, the same as in a
SONET/SDH network.

Due to the large amount of fiber in the ground today, most service providers are not using WDM strictly for fiber
savings. However, they are using WDM technology to efficiently scale bandwidth and provide rapid service
deployment. This is seen with the Cisco ONS 15540, which is often deployed in storage applications for data center
disaster recovery (both in enterprises and service providers). The Cisco ONS 15540 uses 10-Gigabit Ethernet
technology that, coupled with DWDM technology, delivers up to 320 Gbps of bandwidth over a single pair of fiber.
This capacity is then available for the service provider to offer LAN and storage area network (SAN) extension
between sites while making efficient use of the available fiber.

Another emerging application for DWDM and Ethernet is video on demand (VoD), which not only has large
bandwidth requirements, but also needs to be distributed to multiple distributed hubs located around the ring. In this
application, video traffic is sent from an Ethernet-connected VoD server over WDM and then distributed to the
headends serving customers. This gives the VoD provider the price advantage of Ethernet within their infrastructure
and the high bandwidth over single fiber gained from WDM.

Data-Optimized SONET/SDH
In many cases, particularly with an incumbent provider, the installed fiber is carrying TDM traffic. This traffic is
essential a service provider’s business. These service providers already have SONET/SDH networks deployed and
have been offering services off them for a number of years. As providers look to offer incremental data services with
Ethernet over their existing infrastructure, they require the flexible technology that is found with a multiservice
provisioning platform (MSPP).

An MSPP is a platform that can transport traditional circuit-based TDM traffic such as voice, but also supports
native data services (such as switched Ethernet) that offer efficient and cost-effective services over SONET/SDH
infrastructure. The ability to integrate both capabilities allows the service provider to deploy a flexible, cost-effective
architecture that makes the best use of its existing investment while providing incremental services.

There are two ways of implementing data over this type of an infrastructure. First, the Ethernet-connected customer
could be mapped into a STS/VC circuit in a point-to-point fashion. This is often called “book-ending” because the
circuit originates on an Ethernet card in one location, such as a corporate headquarters, and terminates on another
Ethernet card at a satellite office. This one-to-one mapping does not, however, make the best use of bandwidth and
can be very inefficient. Consequently, Ethernet deployments over SONET/SDH infrastructure have traditionally been
expensive.

Technology supported on the Cisco ONS 15454 MSPP provides data optimization via two main mechanisms. First,
the Cisco ONS 15454 ML Series line card provides integrated Ethernet switching in the SONET/SDH platform.
Ethernet switching can then be used to multiplex customers into the SONET infrastructure. For example, say a
provider wants to offer service to 15 customers in a multitenant unit. Using the Cisco ONS 15454, the provider can
connect each customer via a 100-Mbps or 1-Gbps interface on the Cisco ONS 15454 ML Series line card. By using
the rate limiting and policing capabilities of the ML Ethernet switch, the provider can give a committed access rate
to each customer. That traffic is then mapped on the back end to STS-1, STS-3, or STS-12 circuits on the SONET/
SDH ring. This traffic can then be terminated at a service platform, which will be discussed later.

Cisco Systems, Inc.

All contents are Copyright © 1992–2003 Cisco Systems, Inc. All rights reserved. Important Notices and Privacy Statement.
Page 11 of 21
 Another important technology that makes better use of SONET/SDH is Resilient Packet Ring (RPR). RPR is defined
by the IEEE 802.17 committee, in which Cisco has been a major contributor. (RPR is based, in part, on the Spatial
Reuse Protocol [SRP] developed by Cisco, and on Dynamic Packet Transport [DPT].) Cisco provides a scalable
solution for both Layer 2 and Layer 3 RPR. The implementation discussed here represents a Layer 2 implementation
over SONET or SDH. However, the Layer 3 version, traditionally called DPT, is available on high-end Cisco routers
for POP interconnect and other high-bandwidth, ring-based topologies. Figure 5 shows the basic operation of RPR.

Figure 5
Resilient Packet Ring Operation

Data and Control Packets Physical Layer Initially

are Carried in Ring in SONET/SDH Framing
Opposite Directions (Ethernet and Optical Transport Network Possible)

RPR Ring

Inner Ring Cisco

Outer Ring ONS15454

In a multiservice provisioning platform-enabled SONET/SDH ring, RPR is an overlay and is processed by the
multiservice component of the platform on a common circuit. Therefore, a single 1-Gbps circuit can support many
customers connected on ML Series cards on numerous Cisco ONS 15454 platforms around the ring. This provides
excellent bandwidth utilization while still providing fair access to all nodes on the ring. To gain further bandwidth
and ring efficiencies, an oversubscription factor can be applied to the ring, which is determined by the service defined
by the provider as well as the SLA offered to the end customer.
There are a number of other advances in data-optimized SONET/SDH. One technology, Virtual Concatenation,
provides the ability to group several noncontiguous STS or virtual circuit fragments into a single larger virtual STS.
This virtual STS is referred to as a Virtual Concatenated Group (VCG) because it is made up of a group of smaller
STS, or VT, levels. Virtual Concatenation is used in conjunction with another scheme, Link Capacity Adjustment
Scheme (LCAS), which allow members of a VCG to be dynamically added or subtracted to provide additional
bandwidth as required. These schemes have now been finalized by the ITU (G.707 defines Virtual Concatenation and
G.7042 defines LCAS).

Ethernet over the Transmission Mechanisms

It is important at this juncture to point out that, in most cases, the Ethernet topology (Figure 6), or the view from
the spanning tree or IP and MPLS perspective, may look considerably different than the physical SONET/SDH
network. Review Figure 2 and then evaluate Figure 6 for comparison.

Cisco Systems, Inc.

All contents are Copyright © 1992–2003 Cisco Systems, Inc. All rights reserved. Important Notices and Privacy Statement.
Page 12 of 21
 Figure 6
Ethernet Overlay on a Service Provider’s Physical Infrastructure

One of the most obvious differences between the physical infrastructure and logical one is the fact that, although the
physical network is laid out in a ring, the Ethernet network is a full mesh of point-to-point circuits. To access a
customer, for example, a single Ethernet connection will be mapped to a circuit, such as a 10-Mbps port to a DS-3
circuit. Using the SONET/SDH ring protection mechanism means that there does not necessarily need to be dual
physical connections on the Ethernet side, because redundancy can be handled by SONET/SDH. Another important
consideration is the routing protocol. Its topology depends on the routers in the network and what adjacencies and
access exists. Therefore, a network administrator has to be familiar with the underlying transport as well as the
Ethernet and routed network.

Native Ethernet as a Transmission Medium

Although Ethernet can be overlaid on top of an existing transport infrastructure, such as data-optimized SONET/
SDH or DWDM, Ethernet switching can also be run over dark fiber end to end. There are some clear advantages to
this, namely the simplicity of Ethernet and the cost associated with Ethernet switching equipment. It is, however,
important to consider the unique characteristics of Ethernet when using it as a transmission medium. By using
SONET/SDH, for example, resiliency is provided by the transport mechanisms, such as bidirectional line switched
ring (BLSR) or unidirectional-path switched ring (UPSR) in SONET. Ethernet, over dark fiber, must use the schemes
inherent in Ethernet.

It is important to understand at this juncture that Ethernet at Layer 2 is radically different than traditional Layer 2
WAN and MAN protocols such as Frame Relay and ATM. One of the main differences is that Frame Relay and ATM
both offer an intelligent forwarding mechanism (essentially a routing protocol) at Layer 2. For ATM, this protocol
is Private Network Node Interface (PNNI). Frame Relay uses a proprietary routing protocol that uses PNNI,
although most Frame Relay cores are ATM-based. Switched Ethernet at Layer 2 has no such intelligence. It follows

Cisco Systems, Inc.

All contents are Copyright © 1992–2003 Cisco Systems, Inc. All rights reserved. Important Notices and Privacy Statement.
Page 13 of 21
 simple rules: forward the packet if the source and destination are known, flood the packet if the destination is
unknown, and learn the address and forward the packet if the source was previously unknown. There is little
intelligence in this scenario. For this reason, IP or MPLS is typically used to scale the network to support thousands
of end customers.

Native Ethernet uses two main topologies: point-to-point and ring (Figure 7). The choice between the two is often
predetermined by what exists in the ground (SONET/SDH or fiber layout) and cost. If a SONET/SDH network exists
already, then the Ethernet overlay network could be either hub-and-spoke over a physical SONET/SDH ring or could
use a shared ring mechanism like RPR. The hub would reside at the POP and provide aggregation for the spokes,
which are customer premises equipment (CPE).

Figure 7
Ethernet Topologies

Metro
POP

If fiber rings exist, there are two methods of connecting Ethernet devices together: either “daisy chaining” devices
together in a ring, as shown in Figure 7, or using CWDM to provide virtual point-to-point Gigabit Ethernet circuits
over the physical ring as described earlier. The first method, used in many networks today, deploys native Ethernet
rings using the IEEE 802.1 Spanning Tree protocols. Ethernet rings are usually considered for deployment if the
following conditions exist: there is no SONET or SDH already installed, and multiple customer premises are
physically connected over the same fiber ring. Ethernet rings are particularly attractive to startup Ethernet providers
who have installed their own fiber and are cost-conscious, and may consider Ethernet a low-cost alternative to a
transport infrastructure (SONET, SDH, or DPT). However, there are a number of issues to consider when dealing
with Ethernet ring scenarios.

Figure 7 shows a sample network architecture using Ethernet rings. From a technology perspective, Ethernet
switching at Layer 2 is not optimized in a ring configuration. This is in part due to the operation of the Spanning
Tree Protocol that is required by Ethernet to prevent loops in the network. Convergence times may be as short as 2
seconds with Rapid Spanning Tree Protocol [802.1w] and as long as 50 seconds with Spanning Tree Protocol
[802.1d].

Cisco Systems, Inc.

All contents are Copyright © 1992–2003 Cisco Systems, Inc. All rights reserved. Important Notices and Privacy Statement.
Page 14 of 21
 Cisco, by working within the standards committees, has proposed numerous mechanisms to scale the capabilities of
Ethernet. These standards culminated in the recently standardized Rapid Spanning Tree Protocol [802.1w], which
can reduce convergence time considerably. This protocol is supported in the Cisco Catalyst 3550, Catalyst, 4500,
and Catalyst 6500 series switches as well as the Cisco ONS 15454 ML Series line card. By using both Rapid Spanning
Tree Protocol [802.1w] and by managing bandwidth via the policing capabilities in the Cisco Catalyst switches, it is
possible to deploy a spanning tree ring in a standards-based, high-performance manner. Convergence has been
measured in the subsecond range for a ring of 10 switches.

The Service Application Layer—Bringing Intelligence to Ethernet

So far, this paper has focused on how Ethernet can be used as a transmission medium from a customer site into the
service provider’s network. However, intelligence is a critical component of a carrier Ethernet network, bringing
availability, QoS for SLA enforcement, and network security to the service provider network.

The intelligence provided, typically in the POP or data center, allows the service provider to offer not just connectivity
with an Ethernet User-Network Interface (UNI), but ISP peering, data center connectivity and aggregation, and
intelligent inter-POP connectivity.

Intelligence for Creating Carrier Ethernet

Carrier Ethernet must be built into a robust infrastructure capable of providing either the services a provider can
offer to customers or a backbone infrastructure to support those services. Although the applications may vary, the
services themselves are the same. The primary services are:
• Availability and redundancy
• Quality of service
• Security

Availability and Redundancy

Whether Ethernet is provided as a service or used as a backbone media, the availability of the network is essential
to meeting customer SLAs or protecting the service provider’s backbone network from failure. Optical media, such
as SONET/SDH, have built-in protection schemes to reroute around failures. The Cisco ONS 15454, for instance,
supports UPSR and BLSR, which provide 50-ms convergence in the event of a line or path failure. Ethernet also
provides data-plane resiliency schemes required for path protection in the event of a failure. Cisco EtherChannel
technology, which was the foundation for IEEE 802.3ad, bundles Ethernet links to create both a higher-bandwidth
point-to-point physical connection as well as rapid failure (about 200 ms) in the event of a loss of a link within an
EtherChannel device. The Cisco Catalyst Family of switches supports Spanning Tree Protocol [802.1d], Rapid
Spanning Tree Protocol [802.1w], and Shared Spanning Tree Protocol [802.1s]. Unlike some proprietary Ethernet
resiliency schemes, Cisco implementations of Spanning Tree Protocol provide standards-based protection
mechanisms, which can be measured from subsecond convergence in the event of a failure.

Many service providers today use IP or MPLS to scale their networks, either to offer Internet access or to provide an
MPLS VPN service to their customers. Those providers not using IP or MPLS today almost always have a roadmap
for how they will get there. The use of IP or MPLS (which requires an IP foundation) brings another set of resiliency
mechanisms that may be used to augment the availability of the network. Cisco IOS® Software, which has been
developed over the past decade and deployed in most every service provider network in the world, brings a robust

Cisco Systems, Inc.

All contents are Copyright © 1992–2003 Cisco Systems, Inc. All rights reserved. Important Notices and Privacy Statement.
Page 15 of 21
 set of routing protocols, such as Border Gateway Protocol (BGP), Open Shortest Path First (OSPF) Protocol, and
Intermediate System-to-Intermediate System (IS-IS) Protocol. These routing protocols, critical to determining reach
ability in the IP network, offer equal-cost load balancing and subsecond convergence. This is important for any
service, whether a VPN service or just Internet access.

Cisco pioneered Tag Switching in 1997, which became the foundation for today’s MPLS. Cisco also innovated
numerous MPLS features, including Fast Reroute and Traffic Engineering, to better manage bandwidth and
availability, and MPLS VPN to provide highly available and efficient use of network resources as well as sophisticated
services. Cisco supports IP routing on all of its platforms based on Cisco IOS Software, ranging from the Cisco 800
and 1700 series routers to the Cisco 7600 and 12000 series routers , and also including switching platforms such as
the Cisco Catalyst 3550, Catalyst 4500, and Catalyst 6500 series and the Cisco ONS 15454 ML Series. MPLS is
supported across most Cisco IOS routers, from the Cisco 3600 Series to the Cisco 12000 Series routers, and is also
available on the Cisco Catalyst 6500 Series switches.

Quality of Service
QoS provides a means of guaranteeing bandwidth and service to one type of traffic over another. It allows service
providers to effectively manage their bandwidth usage and offer tiered traffic service levels. As service providers look
to differentiate themselves and offer SLAs to their customers, QoS becomes an important component in the network.
Many providers may choose to use a packet-based network as the transport medium. Cisco switches and routers
provide a number of mechanisms to manage how QoS is applied to a packet as it traverses a network. First, the
service provider may provide a certain data rate out to the customer. The provider can use the policing capability of
Cisco routers and Cisco Catalyst switches to limit the customer’s traffic and provide burst capabilities. These
functions are analogous to Committed Information Rate (CIR) and Peak Information Rate (PIR) capabilities within
Frame Relay and ATM networks. Cisco provides several mechanisms that support granular policing, from kilobits
per second to megabits per second. Once the traffic is in the provider’s network, a number of mechanisms can be
employed. First, some traffic types, such as voice, have specific latency and jitter requirements. Strict priority queuing,
when used in conjunction with Weighted Round Robin (in the switching platforms) and Class-Based Weighted Fair
Queuing (in the routers), provides for bandwidth guarantees while also providing traffic differentiation based on IP
precedence or differentiated services code point (DSCP). QoS marking, reclassification, and queuing is handled in the
hardware of Cisco Catalyst switches and Cisco 12000 and 7600 series routers without incurring performance
degradation.

Perhaps the easiest way to guarantee a customer’s traffic is to provide an end-to-end physical connection, such as an
Ethernet-based private line. The Cisco ONS 15454 allows mapping of an Ethernet UNI directly to an STS on a
SONET network that guarantees end-to-end bandwidth within the network. However, many providers may be
interested in multiplexing multiple customers’ traffic streams onto a single STS to realize greater bandwidth
efficiencies within the network. By using the Cisco ONS 15454 ML Series line card to an STS circuit, several customer
traffic streams can be aggregated and QoS policies applied such that if congestion is experienced, QoS can manage
the congestion and help ensure that SLAs are met. By using Resilient Packet Ring (RPR) on the Cisco ONS 15454
ML Series line card, fairness around the ring can be provided via RPR ring access fairness algorithms.

Cisco Systems, Inc.

All contents are Copyright © 1992–2003 Cisco Systems, Inc. All rights reserved. Important Notices and Privacy Statement.
Page 16 of 21
 Network Security
Whether the network is providing an Ethernet UNI or merely using Ethernet as a transport, the network must be
secure from internal and external attacks. Not only are unauthorized network-access cases on the rise, but the
sophistication of the attacks has increased tremendously. Service providers need to be concerned about not only
protecting their own networks from attack, but also protecting the integrity of their customers’ data. Unauthorized
access to a customer’s data or systems as a result of poor network security can hurt the provider’s business and may
also now have legal repercussions. For that reason, Cisco has developed a number of network security mechanisms
to protect Ethernet-based connectivity and service.

It is important to remember that Ethernet in itself, due to its simple forward and filter mechanisms, is not inherently
secure. A number of mechanisms in Cisco switches and routers have been put in place to ensure that the network is
as secure as possible. One simple mechanism, port security, can be enabled to limit the number of MAC addresses
that are learned per port. A static entry of a MAC address per port is also possible. This prevents unauthorized end
stations from “appearing” on the provider’s network. Cisco has also been helping lead the adoption of IEEE 802.1x
port-based user authentication. Once the MAC address has been learned by the switch, 802.1x will authenticate the
user attempting network access. This provides an additional layer of network security by ensuring that only allowed
and authenticated devices are learned on the service provider network.

At Layer 3, Cisco provides a rich set of access control lists (ACLs), all supported in hardware, that enforces policies
on Layer 3 and 4 information. The switching mechanism used by Cisco switches and routers is called Cisco Express
Forwarding. While this in itself is not a network security mechanism, the fact that it uses topology-based switching
means that certain Internet worms that generate thousands of IP flows will not adversely affect network performance.
Cisco uses Cisco Express Forwarding across its entire line of routing and switching products. At Layer 2, Cisco also
provides sophisticated protection mechanisms to protect Spanning Tree Protocol, such as BPDUGuard and
RootGuard, which enhance the overall security and availability of the network.

Ethernet VPN Services

If Ethernet is being offered as the UNI, the transmission medium, whether SONET, WDM, or Ethernet, gets the end
customer’s traffic into the provider’s network. When offering a VPN service to the enterprise, service providers need
to put mechanisms in place to create the VPN and scale it across the service provider’s infrastructure. To accomplish
this, a number of mechanisms can be used.

Tag Stacking
Tag Stacking, also known as Q-in-Q, provides a means of tunneling one set of VLAN tags by inserting a second
VLAN tag into the frame. This allows a service provider who is running a native Ethernet switched network (in other
words, a network build entirely on Ethernet and Spanning Tree Protocol without IP or MPLS) to transparently tunnel
the customer’s VLANs from one site, through the service provider “cloud,” to the second site. There are a few benefits
to this type of approach. The first benefit is that the service provider and customer VLAN tags do not need to match.
This eases the VLAN management and distribution for the service provider. Another benefit is that this approach is
inherently multipoint and provides a service that, to the enterprise customer, looks like an Ethernet segment. Because
Ethernet is inherently a broadcast-based medium, many providers consider offering an Ethernet service that uses
these Ethernet capabilities. It should be noted that Tag Stacking is also less expensive and complex than MPLS
solutions and is very attractive from that perspective, although Spanning Tree Protocol limits the extent to which
a Tag Stacking network can grow.

Cisco Systems, Inc.

All contents are Copyright © 1992–2003 Cisco Systems, Inc. All rights reserved. Important Notices and Privacy Statement.
Page 17 of 21
 However, Tag Stacking, when used with technologies such as Ethernet over MPLS (EoMPLS), offers the best of both
worlds: scaling to very large networks as well as providing transparent multipoint LAN services. Available on the
Cisco 7600 Series, Cisco ONS 15454 ML Series, and Cisco Catalyst 3550 and Catalyst 6500 series, this capability
allows VLAN transparency between the enterprise and service provider, between two enterprise end points. This
addresses one of the major enterprise applications, namely data mirroring and backup, which often requires the
servers at both sites to use the same subnet or VLAN.

Cisco is working within the IEEE 802.1 committee to standardize a mechanism of scaling bridging in the service
provider space. This draft, called Provider Bridges, is currently being evaluated in the IEEE and is also being
coordinated with efforts in other standards bodies, including ITU and IETF.

MPLS and MPLS VPN

As service providers worldwide have looked to scale their networks, MPLS has emerged as a highly scalable and
highly beneficial technology. Many Internet service providers (ISPs), particularly in Europe and Asia, have deployed
MPLS to scale their networks. In addition, many interexchange carriers (IXCs) and post, telephone, and
telegraphs (PTTs) have MPLS networks deployed today. This enables service providers to both scale their networks
and offer MPLS VPNs to enterprise customers. Unlike the Layer 2 mechanisms discussed in the following sections,
MPLS VPN is a Layer 3 mechanism that provides enhanced network scalability, simplicity, and control. How do
MPLS VPN and Ethernet map together? Very simply: Ethernet can be used as a UNI to access MPLS VPN services.

In this case, the “spigot” delivered to the customer is a provisioned or managed Ethernet connection. Using Cisco IOS
routers such as the Cisco 2600, 3600, or 3700 series, the customer-edge device is mapped into an Ethernet VLAN
that defines membership of a particular MPLS VPN. The Ethernet tagged frames will be transported over the
transmission infrastructure to a Cisco Catalyst 6500 Series Supervisor Engine 720, or Cisco 7600 Series or 12000
Series provider-edge router where the Ethernet VLAN is subsequently mapped to a particular MPLS VPN. At this
point, advanced MPLS features such as traffic engineering and MPLS fast reroute can be used to better engineer the
customer’s network, adding potential service revenue to the provider.

Virtual Private Wire Service

With an MPLS network already deployed, either for integrating existing Layer 2 services such as ATM or Frame
Relay or for Layer 3 services, incremental services supporting Layer 2 Ethernet technologies can be deployed.
MPLS is beginning to be viewed as highly advantageous to Ethernet local exchange carriers (LECs) and incumbent
LECS (ILECs) who are looking to provide Ethernet transport services to enterprise customers at Layer 2, while still
being able to scale their core architectures. The IETF has defined Virtual Private Wire Service (VPWS) as a mechanism
to provide Layer 2 connectivity, on a point-to-point basis, within a service provider’s MPLS cloud. Each
point-to-point circuit, or pseudo-wire, uses MPLS or Layer 2 Tunneling Protocol (L2TP) Version 3 to encapsulate
the native header and send it across the MPLS cloud. For Ethernet, that mechanism is called Ethernet over
MPLS (EoMPLS).

The Cisco EoMPLS solution, based on an IETF draft standard, is an extension of MPLS, which naturally
complements the VLAN capability inherent in Layer 2 architectures. In its simplest form, EoMPLS provides an
emulated wire that is used to transport Layer 2 traffic across an MPLS-enabled Layer 3 core. This allows the service
provider the best of both worlds: the scalability of an MPLS core without having to worry about Spanning Tree
Protocol, and a Layer 2 transparent service offering.

Cisco Systems, Inc.

All contents are Copyright © 1992–2003 Cisco Systems, Inc. All rights reserved. Important Notices and Privacy Statement.
Page 18 of 21
 VPWS capabilities are offered on a variety of platforms, including the Cisco Catalyst 6500 Series and Cisco 7600,
10700, and 12000 series.

Virtual Private LAN Service

The lack of inherent multipoint capability in MPLS and the VPWS architecture has been brought to the forefront
by a number of service providers looking to offer an Ethernet Multipoint Service (EMS), also referred to as a
Transparent LAN Service. Although it is widely acknowledged that the majority of Ethernet-based services offered
by the provider will be point to point (the way Frame Relay, SONET, and ATM work today), this capability is
still of tremendous interest. For that reason, Cisco has been working within the IETF on the Virtual Private LAN
Service (VPLS) drafts and within the IEEE to accelerate standards-based solutions for multipoint service delivery that
will allow for multivendor interoperability within the industry.

A VPLS is an architecture that describes how Virtual Switch Instances may be interconnected using pseudo-wires to
form an emulated LAN service. A Virtual Switch Instance behaves, functionally at least, much the way a LAN switch
would. This architecture functions as an overlay onto the MPLS cloud. Cisco is committed to supporting VPLS as a
service offering on its switch and router platforms.

Residential Ethernet

In many places around the world, construction of new multi-dwelling units and turnover of infrastructure, such as
laying new fiber, is allowing Ethernet to become an alternative for broadband access. Instead of using telephone lines
for DSL or the coaxial cable for cable modems, fiber is run to the multi-dwelling units or, in some cases, directly to
the residence. Many providers, such as some PTTs and competitive LECs in Europe and Asia or Independent
Operating Companies (IOCs) in the United States, are looking to offer a “triple play” service to the residential
customer, bundling voice services, video (cable TV and VoD), and data services (Internet access and Internet gaming).
Although voice and Internet access typically require little bandwidth of the network, video often requires significant
bandwidth, hence the interest in Ethernet to the home.

Multicast
Internet access is handled by high-speed connectivity coupled with intelligent routing protocols such as OSPF and
BGP. Voice, which is not bandwidth-intensive, requires QoS to meet its latency and jitter needs. The third service in
the residential market, particularly one over Ethernet, is video. And to scale delivery of video and ensure the best uses
of the available bandwidth, the service provider’s network must support multicast.

Multicast, in its simplest definition, provides intelligent forwarding of IP video stream to the destinations that have
requested that stream. Unlike bridged broadcasts, which flood to all users in a VLAN or bridge domain, multicast
uses Layer 3 to request streams that may be available across an IP network to be forwarded to the requester.
Additional Layer 2 mechanisms, such as Internet Group Management Protocol (IGMP) snooping, in the Cisco
switches then ensure that the multicasts (which at Layer 2, by default, are treated as broadcasts) are forwarded
intelligently to their destination.

Cisco has worked over the past 10 years to develop numerous innovations related to multicast forwarding and
routing protocols. Multicast replication and forwarding is performed in hardware at data rates exceeding tens to
hundreds of millions of packets per second in Cisco switches and high-end routers (Cisco 7600 and 12000 series).
Forwarding, though, is only part of the equation. The other is the complex task of determining how to send multicast
data from the source to only those destinations that have requested the multicast stream.

Cisco Systems, Inc.

All contents are Copyright © 1992–2003 Cisco Systems, Inc. All rights reserved. Important Notices and Privacy Statement.
Page 19 of 21
 At Layer 2, IGMP snooping is a feature used to “scan” end-station IGMP Join messages and ensure that the return
data returns to only those ports that requested the stream. At Layer 3, multicast routing protocols are required to
build the multicast “tree” in the optimal fashion across the network. To accomplish this, Cisco innovated Protocol
Independent Multicast (PIM), a multicast routing protocol designed to operate over an IP network. PIM has proven
itself in numerous enterprise networks and is now deployed in some of the largest residential Ethernet deployments
in the world.

Multicast VLAN Registration (MVR) is another Cisco innovation that increases the efficiency of multicast transport
and is important for residential providers. MVR involves the creation of separate, dedicated VLANs constructed
specifically for multicast traffic distribution. Each Cisco Catalyst switch that receives an MVR stream will examine
each multicast group and internally bridge the multicast VLAN traffic to a particular subscriber that has requested
the multicast stream. This is yet another feature Cisco has developed to help providers offer new and incremental
services to their customers.

Service Selection
As Ethernet moves from a niche residential application to a more mainstream broadband-access mechanism, the
ability of the provider to create and deploy services and bill customers accordingly is of tremendous interest to the
provider. Today, many providers are aggregating Ethernet UNIs at the customer premises back to an aggregation
switch or router in the POP. It is now possible to add more system intelligence by including Cisco Service Selection
Gateway (SSG) software on the Cisco Catalyst 6500 Series and Cisco 7600 Series.

Both the Cisco Catalyst 6500 Series and the Cisco 7600 Series are optimized for aggregating Ethernet traffic that has
been deployed in numerous Ethernet service networks and service provider backbones. By coupling the Cisco SSG
software with the Multiprocessor WAN Application Module (MWAM), a line card dedicated to broadband service
selection in the Cisco Catalyst 6500 Series and Cisco 7600 Series, the service provider can support such service
deployments as PPP Terminated, L2TP Access Concentrator, and L2TP Network Server. This allows for termination
of the session and then application of the appropriate service.

The Cisco MWAM is a complete subscriber-management platform, providing basic subscriber services as well as
complete service-profile assignment and network routing. It enables the service provider to offer a number of services
to their subscribers. These include integrated voice and data, IP television and VoD, differentiated Internet access
with QoS, and VPN access. This allows for the “triple play” to be offered to residential users. The provider can also
offer more advanced services to the residence, such as online gaming and multimedia services.

The service-selection capability also allows for additional capabilities, including:

• Captive portal
• Web-based subscriber authentication
• Walled Garden Service creation
• Micro billing services
• Content filtering
• Bandwidth on demand
• E-learning

Cisco Systems, Inc.

All contents are Copyright © 1992–2003 Cisco Systems, Inc. All rights reserved. Important Notices and Privacy Statement.
Page 20 of 21
Conclusion
Ethernet, the mainstay technology in the LAN, is now making its
way into the service provider networks, due to its high bandwidth,
price points, and flexibility. Ethernet deployment, both within the
service provider networks and as a service offering to the end
customer, is growing rapidly. Ethernet as a service-access technology
is also attractive; its sympathetic operation with enterprise networks
makes a service provider’s network appear as an extension of the
enterprise’s network. Ethernet has the added flexibility of being
transportable over the wide variety of technologies, such as SONET/
SDH, RPR, DPT, IP, and MPLS.
Cisco Systems is the only vendor that is able to integrate these
technologies into an architecture that allows service providers to
offer a rich service to their end customers. Cisco is uniquely
positioned to offer these capabilities by addressing both the
transport and control plane options. From the Cisco ONS transport
platforms to the Cisco Catalyst switches to the Cisco routers, Cisco
addresses the integration of transport technologies (SONET,
Ethernet, and DPT) and offers years of expertise in Ethernet
switching, MPLS, and IP. As a result, Cisco delivers a single cohesive
architecture to address service provider and enterprise requirements.

Corporate Headquarters European Headquarters Americas Headquarters Asia Pacific Headquarters

Cisco Systems, Inc. Cisco Systems International BV Cisco Systems, Inc. Cisco Systems, Inc.
170 West Tasman Drive Haarlerbergpark 170 West Tasman Drive Capital Tower
San Jose, CA 95134-1706 Haarlerbergweg 13-19 San Jose, CA 95134-1706 168 Robinson Road
USA 1101 CH Amsterdam USA #22-01 to #29-01
www.cisco.com The Netherlands www.cisco.com Singapore 068912
Tel: 408 526-4000 www-europe.cisco.com Tel: 408 526-7660 www.cisco.com
800 553-NETS (6387) Tel: 31 0 20 357 1000 Fax: 408 527-0883 Tel: +65 6317 7777
Fax: 408 526-4100 Fax: 31 0 20 357 1100 Fax: +65 6317 7799

Cisco Systems has more than 200 offices in the following countries and regions. Addresses, phone numbers, and fax numbers are listed on the
Cisco Web site at www.cisco.com/go/offices
Argentina • Australia • Austria • Belgium • Brazil • Bulgaria • Canada • Chile • China PRC • Colombia • Costa Rica • Croatia
Czech Republic • Denmark • Dubai, UAE • Finland • France • Germany • Greece • Hong Kong SAR • Hungary • India • Indonesia • Ireland
Israel • Italy • Japan • Korea • Luxembourg • Malaysia • Mexico • The Netherlands • New Zealand • Norway • Peru • Philippines • Poland
Portugal • Puerto Rico • Romania • Russia • Saudi Arabia • Scotland • Singapore • Slovakia • Slovenia • South Africa • Spain • Sweden
S w i t z e r l a n d • Ta i w a n • T h a i l a n d • Tu r k e y • U k r a i n e • U n i t e d K i n g d o m • U n i t e d S t a t e s • Ve n e z u e l a • Vi e t n a m • Z i m b a b w e

All contents are Copyright © 1992–2003 Cisco Systems, Inc. All rights reserved. CCIP, CCSP, the Cisco Arrow logo, the Cisco Powered Network mark, Cisco Unity, Follow Me Browsing, FormShare, and StackWise are
trademarks of Cisco Systems, Inc.; Changing the Way We Work, Live, Play, and Learn, and iQuick Study are service marks of Cisco Systems, Inc.; and Aironet, ASIST, BPX, Catalyst, CCDA, CCDP, CCIE, CCNA, CCNP,
Cisco, the Cisco Certified Internetwork Expert logo, Cisco IOS, the Cisco IOS logo, Cisco Press, Cisco Systems, Cisco Systems Capital, the Cisco Systems logo, Empowering the Internet Generation, Enterprise/Solver,
EtherChannel, EtherSwitch, Fast Step, GigaStack, Internet Quotient, IOS, IP/TV, iQ Expertise, the iQ logo, iQ Net Readiness Scorecard, LightStream, MGX, MICA, the Networkers logo, Networking Academy, Network
Registrar, Packet, PIX, Post-Routing, Pre-Routing, RateMUX, Registrar, ScriptShare, SlideCast, SMARTnet, StrataView Plus, Stratm, SwitchProbe, TeleRouter, The Fastest Way to Increase Your Internet Quotient,
TransPath, and VCO are registered trademarks of Cisco Systems, Inc. and/or its affiliates in the U.S. and certain other countries.
All other trademarks mentioned in this document or Web site are the property of their respective owners. The use of the word partner does not imply a partnership relationship between Cisco and any other company.
(0304R) ETMG 203150—JR 10/03