Beruflich Dokumente
Kultur Dokumente
Support Contacts
U.S. Europe
E-mail: support@spirentcom.com E-mail: support.europe@spirentcom.com
Web: http://support.spirentcom.com Web: http://support.spirentcom.com
Toll Free: +1 800-SPIRENT (+1 800-774-7368) Phone: +33 (0) 1 61 37 22 70
Phone: +1 818-676-2616 Fax: +33 (0) 1 61 37 22 51
Fax: +1 818-880-9154
China
E-mail: supportchina@spirentcom.com
Web: http://support.spirentcom.com.cn
Toll Free: +1 800-810-9529 (mainland China only)
Phone: +86 10 8233 0033 (rest of the world)
Fax: +86 10 8233 0022
Copyright
© 2006 Spirent Communications, Inc. All Rights Reserved.
All of the company names and/or brand names and/or product names referred to in this document, in particular, the
name “Spirent” and its logo device, are either registered trademarks or trademarks of Spirent plc and its subsidiaries,
pending registration in accordance with relevant national laws. All other registered trademarks or trademarks are the
property of their respective owners. The information contained in this document is subject to change without notice
and does not represent a commitment on the part of Spirent Communications. The information in this document is
believed to be accurate and reliable, however, Spirent Communications assumes no responsibility or liability for any
errors or inaccuracies that may appear in the document.
Limited Warranty
Spirent Communications, Inc. (“Spirent”) warrants that its Products will conform to the description on the face of
order, that it will convey good title thereto, and that the Product will be delivered free from any lawful security interest
or other lien or encumbrance.
Spirent further warrants to Customer that hardware which it supplies and the tangible media on which it supplies
software will be free from significant defects in materials and workmanship for a period of twelve (12) months, except
as otherwise noted, from the date of delivery (the “Hardware Warranty Period”), under normal use and conditions.
To the extent the Product is or contains software (“Software”), Spirent also warrants that, if properly used by Customer
in accordance with the Software License Agreement, the Software which it supplies will operate in material
conformity with the specifications supplied by Spirent for such Software for a period of ninety (90) days from the date
of delivery (the “Software Warranty Period”). The “Product Warranty Period” shall mean the Hardware Warranty
Period or the Software Warranty Period, as applicable. Spirent does not warrant that the functions contained in the
Software will meet a specific requirement or that the operation will be uninterrupted or error free. Spirent shall have no
warranty obligations whatsoever with respect to any Software which has been modified in any manner by Customer or
any third party.
Defective Products and Software under warranty shall be, at Spirent's discretion, repaired or replaced or a credit issued
to Customer's account for an amount equal to the price paid for such Product provided that: (a) such Product is
returned to Spirent after first obtaining a return authorization number and shipping instructions, freight prepaid, to
Spirent's location in the United States; (b) Customer provides a written explanation of the defect or Software failure
claimed by Customer; and (c) the claimed defect actually exists and was not caused by neglect, accident, misuse,
improper installation, improper repair, fire, flood, lightning, power surges, earthquake, or alteration. Spirent will ship
repaired Products to Customer, freight prepaid, within ten (10) working days after the receipt of defective Products.
Except as otherwise stated, any claim on account of defective materials or for any other cause whatsoever will
conclusively be deemed waived by Customer unless written notice thereof is given to Spirent within the Warranty
Period. Spirent reserves the right to change the warranty and service policy set forth above at any time, after
reasonable notice and without liability to Customer.
TO THE EXTENT PERMITTED BY APPLICABLE LAW, ALL IMPLIED WARRANTIES, INCLUDING BUT
NOT LIMITED TO IMPLIED WARRANTIES OF MERCHANTABILITY, NONINFRINGEMENT AND FITNESS
FOR A PARTICULAR PURPOSE, ARE HEREBY EXCLUDED, AND THE LIABILITY OF SPIRENT, IF ANY,
FOR DAMAGE RELATING TO ANY ALLEGEDLY DEFECTIVE PRODUCT SHALL BE LIMITED TO THE
ACTUAL PRICE PAID BY THE CUSTOMER FOR SUCH PRODUCT. THE PROVISIONS SET FORTH ABOVE
STATE SPIRENT'S ENTIRE RESPONSIBILITY AND CUSTOMER'S SOLE AND EXCLUSIVE REMEDY WITH
RESPECT TO ANY BREACH OF ANY WARRANTY.
10-Gigabit Ethernet Testing
The development of high throughput and low latency 10-Gigabit network switches and routers
is challenging. Network equipment manufacturers of switches and routers with 10-Gigabit
Ethernet network interfaces are challenged to provide switch fabrics that have high availability
10 Gigabits-per-second line rate forwarding that meets the requirements of converged, multi-
service networks that demand low latency performance. This application note introduces
Spirent Communications’ new test tool platform, Spirent TestCenter™, how it addresses the
critical aspects of 10-Gigabit Ethernet performance testing, and the network protocols that
affect 10-Gigabit Ethernet layer 2 and layer 3 switch performance. Spirent TestCenter is
uniquely suited to assist network equipment manufacturers to meet the challenges of
successfully delivering 10GbE network devices.
In this document...
• Overview of 10-Gigabit Ethernet Network Devices . . . . 4
• Conclusion . . . . 18
Table 1. Network Interface Specifications for 10gbe in Enterprise Network Applications Driven by the
IEEE802.32ae Standard
Table 1. Network Interface Specifications for 10gbe in Enterprise Network Applications Driven by the
IEEE802.32ae Standard (continued)
Table 2. Network Interface Specifications for 10GbE in Metropolitan Network Applications Driven by the
IEEE802.32ae Standard
There are some significant differences between 10GBASE-R and 10GBASE-W. The
10GBASE-W protocol operates at a 9.95328Gbps line rate that conforms to the
requirements of SONET STS-192c and SDH VC-4-64c frame rates. It matches the
9.58464 Gb/s payload rate of SONET. The IEEE802.3ae standard states, “The purpose of
the WIS is to allow 10GBASE-W equipment to generate Ethernet data streams that may
be mapped directly to STS-192c or VC-4-64c streams at the PHY level, without requiring
MAC or higher-layer processing.” This means that its design is meant for a direct PHY-to-
PHY interface, a point-to-point connection. A 10GBSAE-W port should be connected to
another 10GBASE-W port.
The 10GBASE-W implements the minimum framing, scrambling, and error detection for
SONET. It implements the sufficient Path, Section and Line overhead fields that are
required to maintain a SONET link. 10GBASE-W emulates a SONET link using SONET
encapsulation of the Ethernet frame. On the network, it looks like a SONET frame, but it
does not have the jitter, synchronous operation, electrical interface, or various
performance and line controls that a standard fully SONET compliant link would possess.
10GBASE-W is not directly interoperable with OC-192c SONET/SDH; one cannot attach
a 10GBASE-W interface to an OC-192c SONET/SDH interface. The IEEE8023.ae
standard requires that one 10GBASE-W interface be connected to another 10GBASE-W
interface.
An example of this is the connection of two routers that have 10GbE Ethernet and OC-
192c SONET interfaces. When the two routers are connected by two 10GBASE-SW links,
the router’s switching fabric can easily move the SONET encapsulated Ethernet frames
seamlessly to the OC-192c interface that is connected to a core optical network. Rates,
framing, and error detection are aligned between the two different PHY. Within a service
provider network 10GbE is used in LAN or WAN modes depending on the requirement.
The WAN protocol can be used to link to a core network at 10Gb/s rates. The WAN
protocol encapsulates the Ethernet frame in a fixed SONET header to allow SONET
devices to forward it over OC-192c SONET/SDH links.
The operator of the router, from a maintenance viewpoint, would see two SONET links,
not an Ethernet link and a SONET link. This is intended to make links easier to maintain
in a SONET environment.
In metropolitan network applications this translates into the ability to attach Ethernet
switches and routers to core network devices that are attached to the SONET cloud. A
service provider point-of-presence can connect a 10GBASE-W link to a carrier’s central
office network equipment that is using SONET/SDH or optical transport equipment.
Now – and here is the catch – the whole scheme of the IEEE802.3as standard is to enable
Ethernet to participate in true end-to-end applications that makes use of compatible
infrastructure in the LAN and allow Ethernet to be readily adapted to MAN environments
with facility to interconnect with a network device that is attached to the optical core
network. This is made possible because the underlying rate at the MAC layer is 10Gb/s,
the MAC layer rate is constant and is common to both 10GbE LAN and WAN protocols.
The WIS layer adapts the MAC’s 10Gb/s rate to be compatible with SONET/SDH. By
doing this, a 10GBASE-W link inherits the lower cost and ease of maintenance inherent in
10GASE-R links. Due to the constant rate MAC, a switch or router fabric can handle
switching 10GBASE-R to 10GBASE-W across the fabric in the same box.
This flavor of 10GbE, 10GBASE-W, adds new complexities to performance testing of
switches and routers, depending upon where the equipment is deployed in the network.
throughput, forwarding rates, and latency with these various protocols, turned on or off,
with various combinations, is critical to the design of the network. There are more GE and
10GbE links present in the network than ever before, so it is important to know that
network devices can handle planned traffic loads under segmented and prioritized
conditions. The switch must maintain the quality of the user’s experience on the network.
Table 3. Layer 2 and Layer 3 Protocols that Affect Forwarding Performance in Switches
IEEE 802.1Q, Virtual LAN (VLAN) Layer 2 Increase the length of packet headers by 4 bytes for each
(MAC layer) packet. At high 10Gb/s speeds switches must forward
larger packets with very low latency.
VLAN control field data must be inspected for each
packet and this increases packet-processing time.
Multiple VLANs per port force switches to be tested for
throughput per VLAN, VLAN leakage (mis-forwarding),
and over subscription that can flood ports
In Ethernet end-to-end applications over the WAN it
requires Layer 2 to Layer 3 mapping to maintain CoS.
Switches spend more time tracking and forwarding.
IEEE 802.1D Quality of Service Layer 2 More segregation of bandwidth and network domains
management, MAC bridges, support (MAC layer) over individual LANs, that cause more packet inspection,
for multicast networks in increase in forwarding table sizes, and buffer memory
multimedia applications (grouped management. This increases potential for higher latency.
MAC addresses) Switch must maintain throughput in the presence of mis-
ordered and lost frames per VLAN.
Switch must inspect VLAN priorities and maintain
throughput (forwarding rate) over all separate VLANs
Table 3. Layer 2 and Layer 3 Protocols that Affect Forwarding Performance in Switches (continued)
IEEE802.3-Clause 43 Link Layer 2 Traffic on a single port or across multiple ports must
Aggregation and IEEE 802.1ad (MAC layer) support different customer LANs on the service provider
Bridged Local Area Networks, network. Links can be aggregates with up to 8 GE links
Virtual Bridged Local Area aggregating to 1 10GbE link, and 10GbE links to 40Gbps
Networks, links.
Forwarding performance and maintenance of QoS over
segregated LANs combined with the dedication of
bandwidth and network domains per customer, because
more packet inspection, increased packet lengths due to
keys and tags and increased complexity of forwarding
tables, and buffer memory management.
This has potential for higher latency, incorrect forwarding,
and VLAN leakage must be measured. Both Port and per
stream QoS and forwarding rates should be measured on
aggregated links.
Other measurements include port and aggregated stream
failover.
Internet Protocol version 6 (IPv6) Layer 3 Longer standard packet header that supports header
• 40-byte header length extensions. This requires additional packet inspection of a
(IP Layer)
• Extension headers larger packet header with more fields.
• Dual Stack operations IPv6 and IPv4 protocol must be supported in a dual-stack
• IPv6 tunneling mode on a per port basis. This requires additional packet
inspection, which increases the possibility for higher
latency and stresses memory buffers.
RFC 2474 Definition of the Layer 3 Uses the DS field of the IP header to mark packets with
Differentiated Services Field (DS values (called code points) that prioritize one type of
(IP Layer)
Field) in the IPv4 and IPv6 Headers traffic over another based on class and cost. It is used in
QoS, CoS and layer 2 to Layer 3 mapping applications
(MPLS).
This, although not a guaranteed service, requires switches
to perform additional header inspection and can cost
processing time and increase latency.
Table 3. Layer 2 and Layer 3 Protocols that Affect Forwarding Performance in Switches (continued)
Multi-Protocol Label Switching Layer 2 and Service provider networks have two types of routers that
(MPLS) Layer 3 interact. Label Edge Routers (LER) at the provider edge
• MPLS VPN and Label Switching Routers (LSR) within the provider
• VPLS core. MPLS provides a mechanism for IP flows from
many LAN customers to enter a large service provider IP/
MPLS network and to separately maintain QoS for each
customer over the backbone links between the routers in
the provider network.
LERs must inspect and keep track of many different
network domains, and perform deep and complex, packet
inspection. This increases processing time and forwarding
latency.
LSRs label switch the packets and forward them. The
LSRs must demonstrate low Layer 2 latency.
Time sensitive applications like IP telephony and video
are affected by MPLS services.
IP Multicast Layer 2 and A one-to-many protocol used for switching and routing of
Layer 3 audio, video, and conferencing of web casts. Switches
must keep track of grouped MAC addresses and VLANs,
where there can be many users per group and thousands of
groups per port. The switch must be able to allow users to
join and leave these groups without affecting QoS and
throughput. Poor latency performance may disrupt audio
and video transmissions.
Switch fabrics must be able to handle IPv4, IPv6, VLANs,
Unicast, and Multicast across the entire fabric.
The test tools built by Spirent Communications are designed with 10GbE testing in mind.
Spirent TestCenter has capabilities and features that are critical to the testing of 10GbE.
Spirent TestCenter does account for the differences in GE and 10GbE testing. The tool
accommodates the nuances in testing the 10GbE interface. This section discussed what is
required in a test tool to accurately test 10GbE and its common interaction with GE.
The sum of these considerations is that: (1) 10GbE requires additional scalability testing,
(2) 10GbE testing requires higher resolution test tools with improved accuracy, and (3) the
test tool must be flexible enough to allow the test engineer to discretely test and measure
the different implementations of 10GbE (i.e 10GBASE-R and 10GBASE-W interfaces).
The net result is that the actual IFG, which affects the final transmission rate on the line,
will be slower than the theoretical 10GbE line rate. Certain packet sizes are affected by the
average IFG generating mechanism in 10GbE. A 10GbE interface will transmit between
12 and 15 bytes of interfame gap at a given frame size. What this means is that in some
frame sizes, the theoretical 10GbE line rate will not be achieved. So, how does one
perform a 10GbE line rate test on a switch port with so many different frame sizes that
almost ensure that line rate will be never be achieved?
To test at the 10GbE line rate, the test tool must have Deficit Idle Count (DIC)
compensation. The test tool must allow the test to be run with DIC enabled or disabled.
Fundamentally, DIC adds or subtracts up to 3 bytes to or from the nominal average 12-
byte interframe gap in order to maintain the 10GbE frame rate. The range of IFGs
transmitted by the 10GbE interface can be from 9 to 15 bytes, but the average is 12 bytes.
In order to achieve the maximum 10GbE throughput, the test tools and devices under test
must have DIC enabled. The pattern in Table 4 on page 15, will faithfully repeat. There
will be less difference in frames per second rates between DIC enabled and DIC disabled
as the frames increase in size. If the switch is not tested with DIC enabled, and the device
is placed into a DIC-enabled network, the probability increases that the DIC-enabled
network interface will overrun the receive frame buffer of the device with DIC
compensation disabled. This problem occurs with smaller frame lengths.
DIC provides a higher gradient for frames per second rates, so in the event of packet loss,
a more accurate line rate can be measured and reported with DIC enabled.
Table 4. Transmit Rate Differences at the 10Gb/s Line Rate with Deficit Idle Count
64 14,880,952 14,880,952 0
68 14,204,545 14,204,545 0
72 13,586,956 13,586,956 0
The first recommendation that comes from this is to initially test the switch with DIC
enabled. This will stress the switch far more than with DIC disabled by subjecting it to
many varying frame rates. The second recommendation is to run through a considerable
range and number of small frame sizes, for both odd and even frame sizes. Throughput,
packet loss, and forwarding rates must be evaluated until one is satisfied that the switch is
forwarding at the 10GbE line rate over many frame sizes. The third recommendation is to
make sure the range of packet sizes that are test is wide. The switch should be tested in
small frame size steps (increments) from 64 bytes to 16Kb frame lengths.
In Table 5 on page 16, the standard RFC frame sizes and frames per second rates are
shown with DIC enabled and disabled with a 12-byte IFG. Notice that with frame sizes
that are modulo 4, that no impact from DIC is seen until the 1518 frame size is
encountered (reference blue highlighted table cells). This may appear like a subtle
anomaly of the RFC test, or 10GbE, but it is not. When all possible frames sizes are
considered and the differences in frame rates between DIC enabled and DIC disabled
(Table 4) is considered there will be many frame sizes that will pass line rates tests with
DIC disabled but may fail line rate tests with DIC enabled. Spirent recommends that
throughput, frame loss and forwarding rate tests be conducted with DIC enabled.
Table 5. 10-Gigabit Ethernet Line Rates With Standard RFC 2544 and 2889 Frame Sizes
Frame Size Frames / second rates Frames / second rates Interframe Gap
(bytes) (DIC enabled) (DIC disabled) (100% line rate)
For example, if a routine run of the standard RFC frame sizes performed over a 10GbE
interface without DIC, the DUT may pass 100%. Trouble may arise if that same port
interconnects with another port with DIC enabled running off-standard frame sizes. The
DUT may have been tested at less than line rate for non modulo 4 frame sizes. It is easy to
be caught off one’s guard, even in a well known test like RFC 2544 or 2889.
As shown in Table 3 on page 9, the newer network protocols create frame sizes that are
not part of RFC 2889 and 2544, as they were originally written. Table 5 on page 16,
selects a few fundamental frame sizes that exceed the RFC test requirements. These frame
sizes should be tested across the entire switch fabric with their header fields populated
with realistic forwarding values seen in virtual and bridged networks. Today’s switches
must be tested beyond the traditional RFC, using protocols with headers that segment
traffic in support of QoS as well as in converged network applications.
Table 6 does not list all of the possible influences on critical Ethernet frame sizes that
stress switch performance with the many protocols listed in Table 3 on page 9. The point
illustrated is this: simply running a vanilla-flavored RFC 2889 and RFC 2544 switch test
and claiming that one’s device is ready for the world of 10GbE is not advisable.
Table 6. 10-Gigabit Ethernet Frame Rates with Non-standard RFC Frame Sizes
• Latency measurements are dramatically different between 10GbE LAN and 10GbE
WAN. The 10GbE WAN will have a larger average latency and range compared to
10GbE LAN. Average, minimum and maximum latency for both 10GbE WAN and
10GbE LAN should be measured.
• For latency measurements, the test equipment should have 10-nanosecond
measurement resolution and an accuracy rating of 40 nanoseconds.
• 10GbE supports several types of optical and copper transceivers. Latency test results
may vary according to the technology of the transceivers that are used in the test. A
low quality transceiver can add latency to the performance results.
Conclusion
What 10GbE deployment means for NEMS, service providers, carriers and their
customers, is that new test methodology is required to test switches with 10GbE interfaces
in a comprehensive manner. The RFC tests should be run, as well as tests that include
Virtual LANs (VLANs), IPv6 addressing and tunneling, IP multicasting, and Quality of
Service. Another influence that stresses switch fabrics is subnets per VLAN for IPv4 and
IPv6. The summary point is that switches and routers are being asked to simultaneously
perform more tasks on a single port, with more ports, at higher speeds, at lower cost.
The primary challenges for switch and router vendors are:
• Deeper frame and packet header inspections due to more active controls in frame
headers
• High speed, tagging, mapping, and tracking of many more header field data to
network segments
• Perform the functions in bullets 1 and 2 for thousands of clients on a single port,
across multiple ports, over an entire switch fabric
• To maintain large memory buffers for forwarding tables. This is intermixed between
physical and virtual networks.
• Switches and routers with 10GbE network interfaces must be able to forward layer 2
traffic at line rate at all packet sizes, not just the standard packet sizes, and with jumbo
frames.
Table 7 on page 19 and Table 8 on page 19 provide product information for your
reference.
For more details on Switch Test Methodologies, please refer to Spirent’s Test
Methodology Journals for RFC 2544 and RFC 2889.
XFP-1001A Standard scalability and performance, Traffic generator and XFP transceivers: 850,
1-port, 10GbE test module statistical measurement 1310nm, and 1550nm
analyzer
XFP-2001A High scalability and performance, Traffic generator and XFP transceivers: 850,
1-port, 10GbE test module statistical measurement 1310nm, and 1550nm
analyzer
MSA-1001A Standard scalability and performance, High port density traffic XENPAK LAN
2-port Multi-MSA test module generator and statistical transceivers: 850nm,
measurement analyzer 1310nm, and 1550nm
XENPAK LAN/WAN
transceiver: 1310nm
CX-4 Copper, LAN
transceiver
BPK-1001A Packet generator and analyzer base Traffic generator and IPv4, IPv6, layer 2 and layer 3
package A statistical traffic generation, QoS,
measurement analyzer Diffserv, error injection,
custom packet generator,
capture, statistics and real-time
measurements/charts
TPK-1000 RFC-2544 with VLAN network device Layer 3 RFC switch Deficit Idle Count
benchmark test package and router testing and
beyond with IPv6,
VLAN support
TPK-1001 RFC-2889 with VLAN switching Layer 2 RFC switch Deficit Idle Count
benchmark test package and router testing and
beyond with IPv6,
VLAN support
BPK-1002A STP/RSTP/PVST+ base package A Key metropolitan and STP/RSTP/PVST+ and per-
enterprise protocols VLAN Spanning Tree Protocol
state machines
BPK-1014A Multiple Spanning Tree base package A Key metropolitan and Multiple Spanning Tree
enterprise protocols Protocol
BPK-1003A IGMP/MLD host IP multicast base Multicast routing Multicast registration IGMPv1/
package A v2/v3 and MLDv1/v2
BPK-1004A Unicast routing base package A Unicast routing IPv4 and IPv6 interior and
exterior gateway routing
protocols: RIPv1/v2, RIPng,
OSPFv2/v3, IS-IS, IS-ISv6,
BGP-4 and BGP+
BPK-1005A Multicast routing base package A Multicast routing Multicast registration IGMPv1/
v2/v3 and MLDv1/v2
Deficit Idle Count
Table 8. Spirent TestCenter Software Packages for 10-Gigabit Ethernet Testing (continued)
BPK-1006A MPLS/LDP/RSVP-TE base package A MPLS support RFC 2547bis Layer 3 VPNs,
Martini-draft Layer 2 VPNs
(PWE emulation), Virtual
Private LAN Service - LDP
(VPLS – LDP), Virtual Private
LAN Service - BGP (VPLS –
LDP), Layer 3 IPv6 VPNs
Deficit Idle Count