Sie sind auf Seite 1von 88

#CLUS

Cisco Nexus 3000


Switch Architecture
Faraz Taifehesmatian, Technical Marketing
Engineer
CCIE R&S, DC
BRKDCN-3734

#CLUS
Cisco Webex Teams
Questions?
Use Cisco Webex Teams (formerly Cisco Spark)
to chat with the speaker after the session

How
1 Find this session in the Cisco Live Mobile App
2 Click “Join the Discussion”
3 Install Webex Teams or go directly to the team space
4 Enter messages/questions in the team space

Webex Teams will be moderated cs.co/ciscolivebot#BRKDCN-3734


by the speaker until June 18, 2018.

#CLUS BRKDCN-3734 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 3
Session Abstract
This session presents an in-depth study of the architecture of the latest
generation of Nexus 3000 top-of-rack data center switches. Topics
include merchant silicon architecture and capabilities (Broadcom
Trident3, Tomahawk 2, Jericho+ and Barefoot Tofino) , forwarding
hardware, and other physical design elements, as well as a discussion
of key hardware-enabled features and capabilities that combine to
provide high-performance, low latency data center network services.

#CLUS BRKDCN-3734 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 4
What This Session Covers
• Overview of Merchant Silicon with Cisco Nexus Switches
• Latest generation of Nexus 3000 switches
• System and hardware architecture, key forwarding functions,
packet walks

Not covered:
• Nexus 9000 ASIC/platform architectures
• Nexus 9500 merchant-silicon based architectures
• Other Nexus platforms

#CLUS BRKDCN-3734 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 5
Agenda
• Merchant Silicon Overview
• Nexus 3000 Portfolio
• ASIC overview
• Forwarding Pipeline
• Platform Specific Details
• Key Takeaways

#CLUS BRKDCN-3734 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 6
Merchant Silicon
Overview
Merchant Silicon in Cisco DC switches
N3000 N3200, N3100V,9500-R N3100Z,N3200E,N3400

StrataXGS StrataXGS StrataXGS


Trident +, 480 Gbps Tomahawk, 3.2 Tbps Tomahawk II, 6.4 Tbps
Trident II+, 1.28 Tbps Trident III, 3.2 Tbps

StrataDNX StrataDNX
Jericho, 600 Gbps Jericho II, 2.4 Tbps

Barefoot
Tofino, 1.8, 6.4 Tbps

N3100,N9300 N3600R

StrataXGS StrataDNX
Trident II, 1280 Gbps Jericho+, 900 Gbps

#CLUS BRKDCN-3734 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 8
Nexus 3000 Portfolio
Nexus 3000 Series Switch Portfolio

Nexus 3100 Nexus 3200 Nexus 3400


• ToR Leaf • Fixed High Density • Programmable pipeline
• Full-featured DC access • High throughput & performance • Support for P4-INT
• Broad switch portfolio • Flexible connectivity options • Enable custom use cases
• Based on Trident ASIC family • Based on Tomahawk ASIC family • Based on Tofino ASIC

Nexus 3500 Nexus 3600


• Ultra Low Latency • Deep Buffer
• Financial/HFT workloads • High route scale
• Based on Cisco Monticello ASICs • Video & Drop sensitive deployments
• Based on Jericho+ ASIC family

#CLUS BRKDCN-3734 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 10
• Nexus 3100 Switch Family
• Trident 3 ASIC Architecture
• ASIC Single Pipeline Block
Nexus 3100 • N3K-C3132C-Z Switch
Architecture
• N3K-C3132C-Z ASIC Port-
map

BRKDCN-3734 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 11
Nexus 3100 Switch Family

• Based on Trident 2
3100 • 1.28Tbps
• 12.2 MB Buffer

• Based on Trident (2+)


3100-V • 1.28Tbps
• 16MB Buffer

• Based on Trident (3)


3100-Z • 3.2Tbps
• 32MB Buffer

#CLUS BRKDCN-3734 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 12
Trident 3 ASIC Architecture
• BCM56870 from StrataXGS family (BCM56870)
Dynamic Memory Manager Fully Shared
• 3.2Tbps Single Chip Ethernet Switch (8 CoS, SP/WRR/WDRR) 32MB Buffer

• 2 Pipes @1.6 Tbps Multistage Content Aware Engine


(Classify, Filter, Modify, Parallel Lookup)
L2/L3
Host/ARP

• 32 MB of Buffer L2 MAC
L2/L3 Processing L3
Processing
VLAN, VRF, VSI, ECMP

Flex
Hashing/ Lead Balancing Engine Counters

CPU
128 x 25G SerDes MerlinCore
PCIe
…………
32 x 100GE, 10GE, 4 x Gen3
64 x 40GE/50GE 2.5GE,
128 x 10GE/25GE 1GE

#CLUS BRKDCN-3734 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 13
Trident 3 ASIC Single Pipeline Block
FleXGS Ingress Pipeline 0
Lookup Lookup Lookup Special
Port Macro Parser Engine(s) Engine(s) Functions,
Engine(s)
ECMP, Hash

Memory
Management
Unit (MMU)
FleXGS Egress Pipeline 0

Editor Lookup
Port Macro Editor
Control Engine(s)

#CLUS BRKDCN-3734 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 14
Trident 3 Cut-Through Vs Store-and-Forward
• Trident 3 MMU supports both • CT mode is used in latency-
store-and-forward (SF) and cut- sensitive applications
through (CT) modes
• In CT mode, the packet is
• In SF mode, an entire incoming scheduled through the cut-
packet is written into the buffer through path and dequeued to the
first. The packet is held in the EP before it has been completely
buffer until the scheduler selects received from the ingress pipeline.
that particular egress port's queue
• In CT mode, After one or more
packet cells are received by the
MMU, the packet becomes eligible
for dequeening

#CLUS BRKDCN-3734 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 15
Trident 3 Cut-Through switching Matrix

Destination port
Min Ingress Speed Max Egress Speed
Speed 10G ->10G
25G -> 10G
10G 50G 10G 40G -> 10G
25G 50G 25G 50G -> 10G

40G 100G 40G


50G 100G 50G
100G 100G 100G

#CLUS BRKDCN-3734 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 16
N3K-C3132C-Z

Lane-selected LED

Beacon LED
Status LED
Environment LED

32 40/100GE QSFP28 1 RU
2x10GE SPF+

Each QSFP28 port can operate at 10, 25, 40, 50, and 100 G
Up to a maximum of 128 x 25-Gbps ports

#CLUS BRKDCN-3734 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 17
N3K-C3132C-Z Switch Architecture
Fan 1 Chimay CPU Board
Management DIMM 650W
Console G1x1PCIe PSU
Fan 2 USB
CPU
Leffe Board
Fan 3 1.8 GHz Intel Xeon
650W
Fan Interface Board PSU
Fan 4
G2x4PCIe G1x1PCIe CTRL/Status

Haggan Board

x4PCIe MI FPGA CPLD


CPLD
CPLD
CPLD

Trident 3

MC FCs

2 x 10 G 32 x QSPF28
SFP

#CLUS BRKDCN-3734 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 18
N3K-C3132C-Z ASIC Port-map

Trident 3 FalconCores
M FC FC FC
C FC FC FC FC FC FC FC FC FC FC FC FC FC
1 1 5 9 13 17 21 25 29 2 6 10 14 18 22 26 30
M FC FC FC
C FC FC FC FC FC FC FC FC FC FC FC FC FC
2 3 7 11 15 19 23 27 31 4 8 12 16 20 24 28 32

Port Group 0
Pipe 0
Port Group 1
Port Group 2
Pipe 1
Port Group 3

3
3 1 3 5 7 9 11 13 15 17 19 21 23 25 27 29 31
3
4 2 4 6 8 10 12 14 16 18 20 22 24 26 28 30 32

C3132C-Z Front Panel Ports

#CLUS BRKDCN-3734 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 19
Nexus 3000 Series Switch Portfolio

Nexus 3100 Nexus 3200 Nexus 3400


• ToR Leaf • Fixed High Density • Programmable pipeline
• Full-featured DC access • High throughput & performance • Support for P4-INT
• Broad switch portfolio • Flexible connectivity options • Enable custom use cases
• Based on Trident ASIC family • Based on Tomahawk ASIC family • Based on Tofino ASIC

Nexus 3500 Nexus 3600


• Ultra Low Latency • Deep Buffer
• Financial/HFT workloads • High route scale
• Based on Cisco Monticello ASICs • Video & Drop sensitive deployments
• Based on Jericho+ ASIC family

#CLUS BRKDCN-3734 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 20
• Nexus 3200 Switch Family
• Tomahawk2 ASIC
Architecture

Nexus 3200 • ASIC Forwarding Pipeline


• N3K-C3264C-E Switch
Architecture
• N3K-C3264C-E ASIC Port-
map

BRKDCN-3734 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 21
Nexus 3200 Switch Family

• Based on Tomahawk
3200 • 3.2Tbps
• 16MB (4x4MB) Buffer

• Based on Tomahawk 2
3200-E • 6.4Tbps
• 42MB (4x10.5MB) Buffer

#CLUS BRKDCN-3734 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 22
Tomahawk2 ASIC Architecture
• BCM56970 from StrataXGS family (BCM56970)
Dynamic Memory Manager 42MB
• 6.4Tbps Single Chip Ethernet Switch (8 CoS, SP/WRR/WDRR) SmartBuffer

• 4 Pipes @1.6 Tbps Multistage Content Aware Engine


(Classify, Filter, Modify, Parallel Lookup)
L2/L3
Host/ARP

• 42MB (4x10.5MB) of Buffer L2 MAC


L2/L3 Processing L3
Processing
VLAN, VRF, VSI, ECMP

• Ingress & Egress Packet Time


Hashing/ Lead Balancing Engine Flex
Stamping Counters

CPU
256 x 25G SerDes MerlinCore
PCIe
…………
64 x 100GE, 1x100GE, Gen3
128 x 40GE/50GE 2x40GE/50GE x4/x2/x1
128 x 10GE/25GE 4x1/2.5/10/25GE

#CLUS BRKDCN-3734 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 23
Tomahawk2 ASIC Single Pipeline Block

Ingress Pipeline

Intelligent Tunnel Vlan L2 L3


ICAP
Parser Termination Processing Switching Switching

Memory
Management
Egress Pipeline Unit (MMU)
Packet Egress ACL Egress Vlan Egress
Modification Switching Processing Parsing

#CLUS BRKDCN-3734 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 24
Tomahawk2 Cut-Through switching Matrix

Destination port
Min Ingress Speed Max Egress Speed
Speed 10G ->10G
25G -> 10G
10G 50G 10G 40G -> 10G
25G 50G 25G 50G -> 10G

40G 100G 40G


50G 100G 50G
100G 100G 100G

#CLUS BRKDCN-3734 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 25
N3K-C3264C-E
16 QSFP28 +MACsec
Lane-selected LED

Beacon LED
Status LED
Environment LED

2 RU
64 40/100GE QSFP28

2x10GE SPF+

Each QSFP28 port can operate at 10, 25, 40, 50, and 100 G
Up to a maximum of 128 x 25-Gbps ports

#CLUS BRKDCN-3734 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 26
N3K-C3264C-E Switch Architecture
Tyskie CPU Board
Fan1
Management DIMM 1200W
Console
G1x1PCIe
PSU
USB Shocktop
Fan 2 CPU PSU
BlondeBock Board
Board 1.8 GHz Intel Xeon
1200W
Fan 3 PSU

G2x4PCIe G1x1PCIe G1x1PCIe

Tetley Board

x4PCIe MIFPGA MIFPGA CPLD0


0 1 CPLD1

Tomahawk2
FC

MC FC

8 X BearValley

2 x SFP 48 x QSFP28 16 x QSFP28


+ MACsec

64 x QSFP28

#CLUS BRKDCN-3734 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 27
N3K-C3264C-E ASIC Port-map
Tomahawk2 FalconCores
FC FC FC FC FC FC FC FC FC FC FC FC FC FC FC FC
9 13 18 21 6 25 36 60 39 44 52 56 32 47 64 4
FC FC FC FC FC FC FC FC FC FC FC FC FC FC FC FC
10 14 17 22 5 26 35 59 40 43 51 55 31 46 63 3

M
C FC FC FC FC FC FC FC FC FC FC FC FC FC FC FC FC
1 11 16 20 24 8 27 34 58 38 42 50 54 30 45 62 2
M
C FC FC FC FC FC FC FC FC FC FC FC FC FC FC FC FC
2 12 15 19 23 7 28 33 57 37 41 49 53 29 44 61 1

Pipe 0
Pipe 1
8 X BearValley
Pipe 2
Pipe 3

1 2 3 4 5 6 7 8 9 10 11 12 49 50 51 52
13 14 15 16 17 18 19 20 21 22 23 24 53 54 55 56
6
5 25 26 27 28 29 30 31 32 33 34 35 36 57 58 59 60
6
6 37 38 39 40 41 42 43 44 45 46 47 48 61 62 63 64

C3264C-E Front Panel Ports


#CLUS BRKDCN-3734 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 28
N3K-C3264C-E Breakout Port-mode
Following Port-modes are supported:

Default

• 96x50g + 16x100g
• 96x25g + 32x100g
• 128x25g

CLI commands for Port-mode change: System reload is required

Profile CLI : “hardware profile portmode <config>”


<config> is “96x50g+16x100g” or “96x25g+32x100g” or “128x25g”
Dynamic breakout CLI: “interface breakout module 1 port <front_port_num> map
<config>” <config> is “50g-2x” or “25g-4x” or “10g-4x”

#CLUS BRKDCN-3734 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 29
N3K-C3264C-E 96x50g+16x100g
• All ports are operational and first 48 front ports will support 2x50G dynamic breakout.
• 64x100G, 64x50G + 32x100G, 96x50G + 16x100G will be met with this h/w profile
• 49-64 MACSEC Ports will support 100G and 40G operational modes
• SLIC adaptor is not supported in this h/w profile

Breakout Capable port


No Breakout

1 2 3 4 5 6 7 8 9 10 11 12 49 50 51 52
13 14 15 16 17 18 19 20 21 22 23 24 53 54 55 56
6
5 25 26 27 28 29 30 31 32 33 34 35 36 57 58 59 60
6
6 37 38 39 40 41 42 43 44 45 46 47 48 61 62 63 64

#CLUS BRKDCN-3734 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 30
N3K-C3264C-E 96x25g+32x100g
• Front port 1-24, 29-32, 37-64 will be operational
• 1-24 front ports will support 2x50G, 4x25G, 4x10G dynamic breakout
• 29-32, 37-48, 49-64 MACSEC Ports will support 100G and 40G operational modes
• SLIC adaptor is supported on 1-24 Front port

Breakout Capable port


No Breakout
Disabled

1 2 3 4 5 6 7 8 9 10 11 12 49 50 51 52
13 14 15 16 17 18 19 20 21 22 23 24 53 54 55 56
6
5 25 26 27 28 29 30 31 32 33 34 35 36 57 58 59 60
6
6 37 38 39 40 41 42 43 44 45 46 47 48 61 62 63 64

#CLUS BRKDCN-3734 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 31
N3K-C3264C-E 128x25g
• Front port 1-28, 33-36 will be operational
• 1-28, 33-36 front ports will support 2x50G, 4x25G, 4x10G dynamic breakout
• SLIC adaptor is supported on 1-24 Front port

Breakout Capable port


Disabled

1 2 3 4 5 6 7 8 9 10 11 12 49 50 51 52
13 14 15 16 17 18 19 20 21 22 23 24 53 54 55 56
6
5 25 26 27 28 29 30 31 32 33 34 35 36 57 58 59 60
6
6 37 38 39 40 41 42 43 44 45 46 47 48 61 62 63 64

#CLUS BRKDCN-3734 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 32
Nexus 3000 Series Switch Portfolio

Nexus 3100 Nexus 3200 Nexus 3400


• ToR Leaf • Fixed High Density • Programmable pipeline
• Full-featured DC access • High throughput & performance • Support for P4-INT
• Broad switch portfolio • Flexible connectivity options • Enable custom use cases
• Based on Trident ASIC family • Based on Tomahawk ASIC family • Based on Tofino ASIC

Nexus 3500 Nexus 3600


• Ultra Low Latency • Deep Buffer
• Financial/HFT workloads • High route scale
• Based on Cisco Monticello ASICs • Video & Drop sensitive deployments
• Based on Jericho+ ASIC family

#CLUS BRKDCN-3734 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 33
• Barefoot Tofino ASIC Architecture
• Tofino Simplified Block Diagram
• Programmable Switch Approach
• Match-Action Packet Processing

Nexus 3400 • Match-Action Unit


• Hardware Telemetry
• Nexus 34180YC Switch
Architecture
• Nexus 34180YC ASIC Port-map

BRKDCN-3734 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 34
Barefoot Tofino ASIC Architecture
• BFN-T10-018D from Tofino family (BFN-T10-018D)
SRAM Memory
• 1.8Tbps Single Chip Ethernet Switch (Tables, Counters, Policers)

TCAM Table Memory


• 2 Pipes @0.9 Tbps (ACL, LPM)

• P4-programmable pipeline Match-Action Engine


(Exact Match, TCAM, range)

• Single 20 MB Unified Packet Buffer Packet Buffer


• Inband Network Telemetry (INT) CPU
CPU
72 x 25G SerDes
Eth. PCIe
…………
18 x 100GE, 1x100GE, Gen3
36 x 40GE/50GE 2x40GE/50GE x4/x2/x1
72 x 25GE/10GE 4x1/2.5/10/25GE

#CLUS BRKDCN-3734 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 35
Tofino Simplified Block Diagram
Reset and PCIe CPU DMA
Clocks MAC Engines

Control and Configurations

Pipe0
10/25/ Ingress Egress 10/25/
….

….
40/50/100G Match-Action Match-Action 40/50/100G
Rx MACs Pipeline Pipeline Tx MACs
Common
Queuing
Pipe1 and Packet
10/25/ Ingress Data Egress 10/25/

….
….

40/50/100G Match-Action Buffers Match-Action 40/50/100G


Rx MACs Pipeline Pipeline Tx MACs

#CLUS BRKDCN-3734 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 36
Tofino Programmable Switch Approach
Bottom-up Network element design Top-down Network element design

Network Demands Switch OS Network Demands NX-OS

Run Time API Auto Generated API

Driver Driver

ASIC
Tofino

Fixed-function Switch Programmable Switch

#CLUS BRKDCN-3734 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 37
What Is P4?
• P4 - Programming Protocol-Independent Packet Processors
• Programming language designed to allow the definition of data planes
• Open-source, permissively-licensed language
• Designed to be protocol-independent, implementation-independent
• Protocol independence and the abstract language model allow for re-
configurability, target-independence

#CLUS BRKDCN-3734 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 38
Tofino Match-Action Packet Processing
Programmable Ingress Match-Action Pipeline Programmable Egress Match-Action Pipeline
Buffer
Memory ALU Memory ALU Memory ALU Memory ALU Memory ALU Memory ALU
Programmable

Memory ALU Memory ALU Memory ALU Memory ALU Memory ALU Memory ALU

… …
Parser

Memory ALU Memory ALU Memory ALU Memory ALU Memory ALU Memory ALU

Memory ALU Memory ALU Memory ALU Memory ALU Memory ALU Memory ALU

Memory ALU Memory ALU Memory ALU Memory ALU Memory ALU Memory ALU

Memory ALU Memory ALU Memory ALU Memory ALU Memory ALU Memory ALU

Action Action
Match Match
(PHV in) (PHV out) (PHV in) (PHV out)
Table Table

PHV’
PHV
Parameters
PHV

Parameters
PHV’

ALUs ALUs

(Match Action Unit 0 ) (Match Action Unit N )

#CLUS BRKDCN-3734 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 39
PISA: Protocol Independent Switch Architecture
Multiple simultaneous lookups and actions can be supported

Memory ALU

Memory ALU

N lookups M actions
Memory ALU

Memory ALU

Memory ALU

Memory ALU

Match + Action Stage (Unit)

#CLUS BRKDCN-3734 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 40
PISA: Match and Action are Separate Phases
Sequential Execution (Match dependency)
Memory ALU Memory ALU Memory ALU

Memory ALU Memory ALU Memory ALU

Memory
Total Latency = 3
Memory ALU Memory ALU ALU

Memory ALU Memory ALU Memory ALU

Memory ALU Memory ALU Memory ALU

#CLUS BRKDCN-3734 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 41
PISA: Match and Action are Separate Phases
Staggered Execution (Action Dependency)
Memory ALU

Memory ALU

Memory

Memory
ALU

ALU
Total Latency = 2
Memory ALU

Memory ALU

Memory ALU

Memory ALU

Memory ALU

Memory ALU

Memory ALU

Memory ALU

Memory ALU

Memory ALU

Memory ALU

#CLUS BRKDCN-3734 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 42
PISA: Match and Action are Separate Phases
Parallel Execution (No Dependencies)
Memory ALU

Memory ALU

Memory

Memory
ALU

ALU
Total Latency =
Memory ALU
1.1
Memory ALU

Memory ALU

Memory ALU

Memory ALU

Memory ALU

Memory ALU

Memory ALU

Memory ALU

Memory ALU

Memory ALU

#CLUS BRKDCN-3734 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 43
Tofino Ingress Processing
• All packets processed by the ingress buffer & parser
• Parser splits packet header into separate PHV filed, TPHV files and packet body
• PHV files traverse through Ingress Match-Action Pipeline for table lookup and manipulation
• Deparser reassembles packets based on files in PHV

PHV PHV’ Common


Ingress Queuing and
10/ Ingress Buffer Match-Action Deparser
….

40/100G & Parser Packet Data


Pipeline Buffers
Rx MACs

Full Packet

#CLUS BRKDCN-3734 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 44
Tofino Programmable Parser
• Will receive the packet data from the Ethernet MACs
Ethernet MACs and then it would parse 10Gb/s
the packet stream according to the 10Gb/s
10Gb/s
pre-computed parse graph 10Gb/s

SerDes
40Gb/s
100Gb/s Parser
40Gb/s
Next, the fields from the parsed

Packet Header Vector


Ingress Pipeline Arbiter

100Gb/s
protocol headers are extracted into the
corresponding PHVs 10Gb/s
10Gb/s
• Once the Parser has assembled a PHV 10Gb/s
10Gb/s

SerDes
Parser
it can then insert that PHV into the
100Gb/s
40Gb/s
40Gb/s
Match-Action Pipeline
100Gb/s
A single Parser unit can process packets at
about 100Gb/s, it connects to either:
4 x10/25Gb/s MACs, or PCIe
Host PCIe
Packet Interface
100Gb/s
Parser
2 x40/50Gb/s MACs, or
1 x100Gb/s MAC

#CLUS BRKDCN-3734 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 45
Tofino Egress Pipeline
• Egress parser extracts metadata from ingress and packet header from the packet
• Egress Match-Action Pipeline performs additional processing
• Egress deparser assembles outgoing packet

PHV PHV’ 10/

….
Common
Egress 40/100G
Queuing and Egress Buffer Deparser
Match-Action Tx MACs
Packet Data & Parser Pipeline
Buffers

Full Packet

#CLUS BRKDCN-3734 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 46
Tofino Egress Match-Action Pipeline
• Additional lookups for packet header modifications (i.e. tunnel
encap, multicast replicated packets)
• Perform calculations (such as WRED) based on intrinsic
metadata from TM
• Additional stats and policing as specified by P4 program

#CLUS BRKDCN-3734 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 47
Tofino Egress Deparser
• PHV data is reassembled with packet
payload
• Unnecessary fields are omitted from
reassembled packet
Eth(L2) Fields
Vlan TCP UDP
• Final outgoing packet length fed back Vlan
IPv4
in
order
Eth(L2) IPv4

to TM for scheduling and shaping Fields


Dictionary TCP
feedback UDP Eth(L2) IPv4 UDP

All fields with holes removed


• Optionally send copy of packet to
mirror buffer for egress mirroring
• Optionally capture PHV data into digest
buffer for coalescing

#CLUS BRKDCN-3734 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 48
Tofino Combined Ingress/Egress Pipeline
Combined Ingress/Egress Match-Action Pipeline Egress Packet Body
MAU 0 MAU n
Egress MAC
Parser PHV PHV
0 MAC
Egress Egress


Deparser Packet


Constructor
Egress
MAC
Parser
m MAC


Ingress
MAC Parser
0 Queues
MAC Ingress
And
Packet

Ingress Ingress Packet


Deparser Constructor
Buffer Buffers
Ingress
MAC Parser
m
MAC

Recirculation
Buffer

Ingress Packet Body


#CLUS BRKDCN-3734 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 49
Telemetry Modes

Inband Network Telemetry


Postcard Mode
(INT)

• In the postcard mode, each network • Metadata is embedded in between


device generates its own telemetry the original headers of data packets
reports as they traverse the network
• The collector will receive reports • This is done by INT data plane
from different network devices, each specifications
describing the telemetry metadata

#CLUS BRKDCN-3734 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 50
Postcard Mode vs INT Mode
Host Host

Header Payload Header Payload


Switch A Switch A
Watchlist Watchlist
Event detection

Packet Header + Info A Endpoint (Source)

Header Payload Header Switch A info Payload


Switch B Switch B
Watchlist
Event detection
Transit
Packet Header + Info B

Header Payload Header Switch A info Switch B info Payload


Switch C Switch C
Watchlist
Event detection
Endpoint (Sink)
Event detection

Packet Header + Info C Packet Header +


Header Payload Collector
Info A, B, C
Header Payload

Host Host

#CLUS BRKDCN-3734 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 51
Inband Network Telemetry (INT)
• First Record (INT instruction
+metadata) will be inserted in data
INT Transit INT Transit
packet at INT Source node
• Second Record (INT metadata) will be
appended to same data packet at INT
Transit node INT Source INT Sink

• Third Record (INT metadata) will be


Host
appended to INT stack at INT Sink. Host

• INT Sink will remove INT record and


forward to INT Collector while original
packet will be forwarded to server
facing port
INT Collector

#CLUS BRKDCN-3734 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 52
INT Per-switch information captured
Flow Watch List (zoom-in view per 5- Flow Drop List (Drop due to various
tuple of flow + DSCP bits) – 1K drop reasons) - 256
Switch ID Switch ID
Hop latency Ingress Port ID
Queue ID + Queue occupancy Egress Port ID
Ingress timestamp Queue ID
Egress timestamp Drop Reason

• Node-to-Node: Reserved DSCP bit will be inserted temporarily in data packets to indicate that
packets also carry INT data
• Node-to-Collector: A UDP encapsulation is used to pack collected INT stack at INT Sink and send to
collector. Flow-affinity is maintained to send same flow-record to same collector for easy processing

#CLUS BRKDCN-3734 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 53
NX-OS INT Configuration Model
Exporter
Where to send?
(IP-Address, Source)

INT Record
What to capture?
“feature hw_telemetry”

(timestamp, Queue-id)

System Monitor
Watch List
Which flow?
Enable INT

(5-tuple, Permit/Deny) Monitor Session


<n>
Queue Profile (Role Transit/Endpoint,
What queue thresholds? Mode INT/Postcard)
(Depth, Latency)

Flow Profile(DSCP, Age,


How to identify the flow? Latency Quantization)

Which drops to report? Drop List

#CLUS BRKDCN-3734 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 54
NX-OS INT Configuration Example
feature hw_telemetry

inband-telemetry exporter E1
destination 10.200.20.2
source Ethernet1/10

inband-telemetry record R1
collect switch-id
collect port-id
collect queue-occupancy
INT Transit INT Transit
collect ingress-timestamp
collect egress-timestamp

inband-telemetry watchlist ip WL1


10 permit ip 1.1.1.1/24 10.10.10.10/24 interface ethernet x/x interface ethernet x/x
20 deny ip 2.2.2.2/24 4.4.4.4/24 port-type fabric port-type fabric
inband-telemetry monitor M1
record R1
exporter E1
watchlist WL1 INT Source INT Sink
inband-telemetry queue-profile QP1
depth 1000 Host Host
latency 1000

inband-telemetry flow-profile FP1


dscp 1
age 5
latency quantization 10

inband-telemetry system monitor M1

#CLUS BRKDCN-3734 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 55
N3K-C34180YC
Beacon LED 6 QSFP28 40/100GE
Status LED
Environment LED

1 RU
48 x 10/25GE SFP28

#CLUS BRKDCN-3734 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 56
Nexus 34180YC Switch Architecture
Fan 1 Chimay CPU Board
Management DIMM 650W
Console G1x1PCIe PSU
Fan 2 USB
CPU
Leffe Board
Fan 3 1.8 GHz Intel Xeon
650W
Fan Interface Board PSU
Fan 4
G2x4PCIe G1x1PCIe CTRL/Status

Boddington Board

x4PCIe MI FPGA CPLD


CPLD
CPLD
CPLD

Barefoot Tofino
ETH

ETH 3 X BearValley

6 x QSFP+
48 x SPF+ /QSFP28
10/25G
40/100G

#CLUS BRKDCN-3734 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 57
Nexus 34180YC ASIC Port-map

Tofino ETH ports


ETH ETH ETH
2 30 26
ETH ETH ETH ETH ETH ETH ETH ETH ETH ETH ETH ETH
22 23 24 20 18 16 14 12 10 8 7 6 ETH ETH ETH
4 0 28

Pipe 0
3 X BearValley
Pipe 1

1 1 1 1 1 2 2 2 2 2 3 3 3 3 3 4 4 4 4
1 3 5 7 9 1 3 5 7 9 1 3 5 7 9 1 3 5 7 9 1 3 5 7 49 51 53
2 4 6 8 1
0
1
2
1
4
1
6
1
8
2
0
2
2
2
4
2
6
2
8
3
0
3
2
3
4
3
6
3
8
4
0
4
2
4
4
4
6
4
8
50 52 54

48 SPF28 10/25Gig 6 QSPF28 40/100Gig

34180YC Front panel ports


#CLUS BRKDCN-3734 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 58
Nexus 34180YC Generic Profile
48x25G, 6x100G (Breakout 100/25, 40/10)

L3
L2 •
• Interfaces (L3, SVI (2k), L3 port-channels)
L3 Routing v4/v6 (v4 Host 32k and v4 LPM 4k, v6 hosts
• Interfaces (access, Trunk, Q-in-Q, Port-Channels (128)) 16k, v6 LPM 4k, Next hops 48k (shared) )

• 4k VLAN, STP, Storm Control • ECMP (32-way, 1k groups)

• Unicast Bridging (32k MAC), Multicast forwarding/ IGMP snooping • BGP, OSPF, BFD

• Peer-link less VPC • HSRP, VRRP

• LACP/UDLD • Multicast Routing, PIM-SM, SSM (SG 8k/2k, *.G 4k/1k.

Data plane Telemetry


ACL, QoS • In-band Telemetry (Flow reports, Queue reports, Drop reports) -1k
Flow watchlist, 256 Drop watchlist
• Ingress ACL (MACL, VACL, RACL) (7k)
• SPAN/E Data plane Telemetry RSPAN/Mirroring ( 64 sessions)
• Egress Policing
• Object Models
• Ingress QOS (Classification, Policing, Marking, Shaping, scheduling)
• PTP (Boundary mode)
• CoPP, Custom CoPP
• DHCP v4/v6 Relay
• PFC, LLFC, ECN

#CLUS BRKDCN-3734 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 59
N3K-C34180YC L3-Heavy Profile
48x25G, 6x100G (Breakout 100/25, 40/10)

L3
L2 • Interfaces (L3, SVI (2k), L3 port-channels)
• L3 Routing v4 (Host 64K and LPM 64K, next hops 64K)
• Interfaces (access, Trunk, Port-Channels (128))
• ECMP (32-way, 1k groups)
• 4k VLAN, STP, Storm Control
• BGP, OSPF, BFD
• Unicast Bridging (2k MAC)
• HSRP, VRRP
• Peer-link less VPC (FCS-only)
• LACP/UDLD

Data plane Telemetry


ACL, QoS • In-band Telemetry (Flow reports, Queue reports, Drop reports) -
• Ingress ACL (RACL) (1k), (+ Minimum Ingress ACL 1k Flow watchlist, 256 Drop watchlist
support for System ACL) • SPAN/ERSPAN/Mirroring (64 sessions)
• Egress Shaping • Object Models
• QOS (Classification, Policing, Marking, Shaping, scheduling) • PTP (Boundary mode)
• CoPP, Custom CoPP • DHCP v4 Relay
• ECN

#CLUS BRKDCN-3734 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 60
Nexus 3000 Series Switch Portfolio

Nexus 3100 Nexus 3200 Nexus 3400


• ToR Leaf • Fixed High Density • Programmable pipeline
• Full-featured DC access • High throughput & performance • Support for P4-INT
• Broad switch portfolio • Flexible connectivity options • Enable custom use cases
• Based on Trident ASIC family • Based on Tomahawk ASIC family • Based on Tofino ASIC

Nexus 3500 Nexus 3600


• Ultra Low Latency • Deep Buffer
• Financial/HFT workloads • High route scale
• Based on Cisco Monticello ASICs • Video & Drop sensitive deployments
• Based on Jericho+ ASIC family

#CLUS BRKDCN-3734 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 61
• Monticello Architecture
• ASIC Block Diagram
• ASIC Forwarding Paths
• ASIC Forwarding Pipeline
• Normal Mode
Nexus 3500 • Warp Mode
• Monticello Warp Span
• N3K-C3548P-XL Switch
Architecture
• N3K-C3548P-XL ASIC
Port-map

BRKDCN-3734 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 62
Monticello ASIC Architecture
• 480 Gbps Single Chip Ethernet Switch (MonticelloCR)

• 720 MPPS @ 64 Bytes Decision Engine

• 18 MB of Buffer (3 x 6MB) Admission Control

• UUL 250ns (~200ns in Warp Mode) Output Output Output


Buffer 1 Buffer 2 Buffer 3

ReWrite ReWrite ReWrite

CPU
48 x 10G XFI
PCIe

…………

#CLUS BRKDCN-3734 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 63
Monticello ASIC Block Diagram

Ports: Ports:
Output
9-12 9-12
Buffer
x16
….

x16

….
21-24 Parser ReWrite 21-24
0
33-36 33-36
6MB
45-48 45-48

Admission Control
Decision Engine
Ports: Ports:
5-8 Output
5-8
ReWrite
x16
….

Buffer
x16

….
17-20 Parser 17-20
29-32 1
29-32
41-44 6MB
41-44

Ports: Ports:
1-4 Output 1-4
Buffer
x16
….

….
x16 13-16
25-28
Parser
2
ReWrite 13-16
25-28
37-40 6MB
37-40

#CLUS BRKDCN-3734 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 64
Monticello ASIC Forwarding Paths

Normal

L2

L3
Classification

ACL

Egress Port
Incoming
Packet

L2 + L3
Warp

Warp Span

#CLUS BRKDCN-3734 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 65
Monticello ASIC Forwarding Pipeline
Normal Mode
Lookup Engine

Unicast LPM
Table 24K

Multicast
Parsed/ Route Table L3 UC
Classified 8K ECMP MAC Table Decision To Buffer
Traffic Table 16K 64K Engine

Unicast Host
Table 64K

ACL Table
4K Egress ACL Result

For Egress ACL lookup

#CLUS BRKDCN-3734 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 66
Monticello ASIC Forwarding Pipeline
WARP Mode
Lookup Engine

Unicast LPM 4K
Multicast Table
8K
MAC Table 8K

Host Table 8K
Parsed/ L3 UC
Classified Control 4K ECMP MAC Table Decision To Buffer
Traffic Table 16K 64K Engine

Unicast Host
Table 64K

ACL Table Ingress ACL Result


4K

#CLUS BRKDCN-3734 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 67
Normal vs. Warp Mode Forwarding
Feature Normal Warp
Latency 250ns 190ns

NAT Yes Yes

Ingress RACL/VACL Yes Yes

Egress RACL/VACL Yes No

Unicast Route 24K 4K

Multicast Route 8K 8K

L3 ECMP Yes No

#CLUS BRKDCN-3734 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 68
Monticello Warp Span
Exchange Feed
• WARP SPAN can be enabled both in
normal and WARP mode 1/36

25-28

1-4
• Latency ~50 ns
WARP SPAN source has to be port

29-32

5-8
1/36
Destination ports would be group of

33-35

9-12

4 ports Shared Buffer

13-16
37-40

17-20
41-44
45-48

21-24
#CLUS BRKDCN-3734 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 69
N3K-C3548P-XL
48 1/10GE SPF+
10/100/1000 management port

RS-232 serial console port

Same Physical device as 3524-XL


1 RU

40 GE by combining
four sequential SFP+
Two USB ports

#CLUS BRKDCN-3734 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 70
N3K-C3548P-XL Switch Architecture
16GB Bootflash Fan1
400W DIMM
PSU
Fan Fan2
CPU Board
Fan3
400W 2.50GHz GHz Intel
PSU Core(TM) i3-3227U
Fan4
G1x1PCIe
G2xPCIe G1x1PCIe CTRL/Status

PCIe MI FPGA CPLD


CPLD
CPLD
CPLD

Monticello CR
48 x XFI
XFI

24 x XFI to SFI
Port
LEDs
48 x SFI
Management
Console 48 x SPF+ 10G

USB

#CLUS BRKDCN-3734 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 71
N3K-C3548P-XL ASIC Port-map

MonticelloCR OB Ports

1 1 1 1 1 2 2 2 2 2 3 3 3 3 3 4 4 4 4
2 4 4 6 8 0 2 7 9 1 3 9 1 3 5 7 9 5 7
0 5 7 1 3

1 1 1 1 2 2 2 2 2 3 3 3 3 3 4 4 4 4 4
3 5 5 7 9 1 3 8 0 2 4 0 2 4 6 8 0 2 4 6 8
1 6 8

OB 0
OB 1
OB 2

1 1 1 1 1 2 2 2 2 2 3 3 3 3 3 4 4 4 4
1 3 5 7 9 3 5 7 9 1 3 9 1 3 5 7 9 5 7
1 5 7 1 3

1 1 1 1 1 2 2 2 2 2 3 3 3 3 3 4 4 4 4 4
2 4 6 8 4 6 8 0 2 4 6 8 0 2 4 6 8 0 2 4 6 8
0 2

48 SPF+ 10Gig

#CLUS BRKDCN-3734 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 72
Nexus 3000 Series Switch Portfolio

Nexus 3100 Nexus 3200 Nexus 3400


• ToR Leaf • Fixed High Density • Programmable pipeline
• Full-featured DC access • High throughput & performance • Support for P4-INT
• Broad switch portfolio • Flexible connectivity options • Enable custom use cases
• Based on Trident ASIC family • Based on Tomahawk ASIC family • Based on Tofino ASIC

Nexus 3500 Nexus 3600


• Ultra Low Latency • Deep Buffer
• Financial/HFT workloads • High route scale
• Based on Cisco Monticello ASICs • Video & Drop sensitive deployments
• Based on Jericho+ ASIC family

#CLUS BRKDCN-3734 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 73
• Nexus 3600 Switch Family
• Jericho+ ASIC Architecture
• ASIC Packet Forwarding
Nexus 3600 • N3K-C3636C-R Switch
Architecture
• N3K-C3636C-R ASIC Port-
map

BRKDCN-3734 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 74
Nexus 3600 Switch Family

• Based on 2 Jericho +
36180YC-R • 1.8Tbps
• 8 GB of Buffer

• Based on 4 Jericho +
3636C-R • 3.6Tbps
• 16 GB of Buffer

#CLUS BRKDCN-3734 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 75
Jericho+ ASIC Architecture
• BCM88680 from StrataDNX family (BCM88680)

• 900Gbps, 835Mpps 48 x 25G SerDes

• Integrated Forwarding and Fabric 1588 Time stamping &


Clock Recovery Flexible Counters

interface
Packet Processor (PP)
Two packet processing cores (PP)
L2/L3/ MPLS/ VPLS/ L2VPN/ L3VPN/VXLAN

Packet Buffer Memory Traffic Manager (TM) On Chip
• 96K Virtual Output Queues DRAM Controller Deep buffered VOQs, hierarchical
Buffer
(16MB)
Scheduling

48 x 25G +24 x 12.5G SerDes

…………
12 x 100GE,
24 x 50GE
30 x 40GE
48 x 25GE

#CLUS BRKDCN-3734 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 76
Jericho+ High-Level Forwarding Architecture

On-chip resources Fabric Interface


• 16MB Internal Buffer &TCAM
Ingress Transmit Egress Transmit
• Forwarding Tables Packet Packet
Processing Processing
Expansion via off-chip resources
Ingress Egress
• Deep GDDR5 external packet buffers Packet Buffer Traffic
On Chip
Buffer
On Chip
Buffer Traffic
DRAM Manager Manager
Ingress/Egress Traffic Managers
Ingress Receive Egress Receive
• 96k Virtual Output Queues Packet Forwarding Packet
Processing Table Processing
• WRED, Distributed Arbitration

Network Interface

#CLUS BRKDCN-3734 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 77
Jericho+ Buffering
• Nexus N3600-R switches use traditional VoQ architecture
• Big buffer on Ingress side dedicated to VoQ buffer
• 4GB GDDR5 DRAM-based buffering per port-group used for VoQ buffer
• VOQ buffer has dedicated portion per port and shard buffer among ports
in the same port group
• 16MB of On-chip buffer used for egress buffer

#CLUS BRKDCN-3734 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 78
N3K-C3636C-R
RS-232 serial console port
8 QSFP28 + MACsec
RS-FEC for 25GE

Lane-selected LED

10/100/1000-Mbps
management port 1 RU
(copper or fiber)
USB Port 36 QSFP28

Each QSFP28 can breakout to 4 x 10 or 4 x 25GE

#CLUS BRKDCN-3734 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 79
N3K-C3636C-R Switch Architecture
Redcastle CPU Board
Fan 1 DIMM
SSD Flash
Drive 128GB 400W
PSU

Fan 2 CPU

NVRAM
Broadwell-DE 8 Core,
2MB 2.0GHz 400W
Fan 3 PSU

G1x1PCIe
Redcastle Main Board
Switch

XBAR1 XBAR2

24 x 25G

Jericho+ 1 Jericho+ 2 Jericho+ 3 Jericho+ 4


9x100 9x100 9x100 8x100

4 X BearValley

Management
Console 28 x QSPF28 8 x QSPF28 +
USB MACsec
32 x QSPF28

#CLUS BRKDCN-3734 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 80
N3K-C3636C-R ASIC Port-map
Jericho+ Ports

1 3 5 7 9 2 4 6 8 1 3 5 7 9 2 4 6 8
2 4 6 8 1 3 5 7 9 2 4 6 8 1 3 5 7 9

Jericho+ 1
Jericho+ 2
Jericho+ 3 4 X BearValley
Jericho+ 4

1 3 5 7 9 11 13 15 17 19 21 23 25 27 29 31 33 35
2 4 6 8 10 12 14 16 18 20 22 24 26 28 30 32 34 36

C3636C-R Front Panel Ports

#CLUS BRKDCN-3734 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 81
Nexus 3000 Series Switch Portfolio

Nexus 3100 Nexus 3200 Nexus 3400


• ToR Leaf • Fixed High Density • Programmable pipeline
• Full-featured DC access • High throughput & performance • Support for P4-INT
• Broad switch portfolio • Flexible connectivity options • Enable custom use cases
• Based on Trident ASIC family • Based on Tomahawk ASIC family • Based on Tofino ASIC

Nexus 3500 Nexus 3600


• Ultra Low Latency • Deep Buffer
• Financial/HFT workloads • High route scale
• Based on Cisco Monticello ASICs • Video & Drop sensitive deployments
• Based on Jericho+ ASIC family

#CLUS BRKDCN-3734 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 82
Key Takeaways
Key Takeaways

Full-featured DC access Programmable Pipeline


Broad Switch Portfolio Inband Network Telemetry (INT)
High throughput & performance 3100,3200 3400 In-Network DDoS Detection
Flexible connectivity Layer 4 Load Balancer

Cisco Nexus
3000 Series
Switches

Deep Buffer
Ultra-low latency 3500 3600 16 GB of Buffer
Less than 200ns Ingress VoQ
Normal, Warp, Warp SPAN Multiple ASICs

#CLUS BRKDCN-3734 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 84
Complete your online session evaluation

Give us your feedback to be entered


into a Daily Survey Drawing.
Complete your session surveys through
the Cisco Live mobile app or on
www.CiscoLive.com/us.
Don’t forget: Cisco Live sessions will be available for viewing
on demand after the event at www.CiscoLive.com/Online.

#CLUS BRKDCN-3734 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 85
Continue
your Demos in
the Cisco
Walk-in
self-paced
Meet the
engineer
Related
sessions
education campus labs 1:1
meetings

#CLUS BRKDCN-3734 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 86
Thank you

#CLUS
#CLUS