Beruflich Dokumente
Kultur Dokumente
Multiple Datacenter
using Layer 2
Solutions
BRKDCT-2840
Yves LOUIS
BRKDCT-2840
Cisco Public
Housekeeping
We value your feedback- don't forget to complete your
online session evaluations after each session & complete
the Overall Conference Evaluation which will be available
online from Thursday
Visit the World of Solutions
Please remember this is a 'non-smoking' venue!
Please switch off your mobile phones
Please make use of the recycling bins provided
Please remember to wear your badge at all times
including the Party
BRKDCT-2840
Cisco Public
Cisco Public
Session Agenda
Data Center Interconnection Common Scenarios and Terms
Options for Layer 2 Interconnectivity
L2 Extension Overview
Extented L2 deployment
Scenarios/Application requiring L2 extension
Requirement and problematic
Extented L2 Techniques overview
Dedicated Fiber, L2/L3 trunk, L2 over WAN
L2 Extension Technlogies
MPLS usage to interconnect Data Centers
EoMPLS
VPLS
AToM over GRE
Virtual Switching Systems and virtual PortChannel for DC Interconnect
Future technologies and solutions
BRKDCT-2840
Cisco Public
Data Center
L2 Interconnect
BRKDCT-2840
Cisco Public
BRKDCT-2840
Cisco Public
Virtual Machines
VMWARE and VMotion Requirements
VMotion
Method used by VMWaress ESX Server to migrate active virtual machines
(VMs) within an ESX server farm from one physical ESX host to another.
This is the foundation of several high availability features provided in
VMWares Virtual Insfrastructure product.
Allows the movement of active VMs with minimal downtime.
Server administrators may schedule or initiate the VMotion process manually
through the VMware VirtualCenter management tool.
State
State
BRKDCT-2840
Cisco Public
BRKDCT-2840
Cisco Public
BRKDCT-2840
Cisco Public
HA Cluster/GeoCluster
Requirements
* Microsoft MSCS
Common Functions
Extended L2 VLAN
HP MC/ServiceGuard
HP NonStop
Quorum Disk
HP Open VMS/TruCluster
IBM HACMP
EMS/Legato Automated Availability Mgr
Cisco Public
10
Layer 2 / 3 Clusters
Intra-Cluster node communications
Flow Types: Traditionally Layer2
Communications on Private and/or Public interfaces
IPv4 and/or IPv6 possible depending on clustering package used
Ability to prioritize interfaces
Cisco Public
11
L2 Extension
Requirements and
Problematic
BRKDCT-2840
Cisco Public
12
Layer 2 Risks
Flooding of packets between data centers
Rapid Spanning Tree (RSTP) is not easily scalable and
risk grows as diameter grows
RSTP has no domain isolation issue in single DC can
propagate
First hop resolution and inbound service selection can
cause verbose inter-data center traffic
In general Cisco recommends L3 routing for
geographically diverse locations
This session focuses on making limited L2 connectivity
as stable as possible
BRKDCT-2840
Cisco Public
13
Cisco Public
14
Intranet L3
VIP
Public L2
VIP
HB
Private L2
N1
active
Storage
Ctl 1
Disk Replication
Site 1
BRKDCT-2840
Cisco Public
Requirements:
L2 node to node
(VIP + Bck-up HB)
100% resilient
Extended L2 options:
VIP
1. Dedicated Fiber
Dark Fiber
Gig Ethernet
HB VIP
WDM + STP
N2
WDM + MPLS
(standby)
2. Mix L2/L3 trunk
Storage
STP
Ctl 2
3. Intranet MPLS DIY
L2/L3 VPN
VPLS
4. Intranet SP
L2VPN
Site 2
CsC
L2TPv3
15
L2 Extension
STP concerns
L2
L2
L2
RPVST or MST
RPVST or MST
VIP
VIP
HB
HB
Node B
Site A
BRKDCT-2840
Site B
Cisco Public
16
L2 Extension
Etherchannel
STP Isolation
L2
L2
L2
RPVST or MST
RPVST or MST
VIP
VIP
HB
HB
Node B
Site A
BRKDCT-2840
Site B
Cisco Public
17
Extended L2 VLAN
Cat3750 Cross-stack Etherchannel
STP Isolation
L2
L2
L2
RPVST or MST
RPVST or MST
StackWize
VIP
VIP
HB
HB
Node B
Site A
BRKDCT-2840
Site B
Cisco Public
18
Extended L2 VLAN
Virtual Switching with MEC (VSL local)
L2
STP Isolation
L2
L2
RPVST/MST
RPVST/MST
VSS
VIP
VIP
HB
HB
Node B
Site A
BRKDCT-2840
Site B
Cisco Public
19
Extended L2 VLAN
Virtual PortChannel (MCEC)
L2
STP Isolation
L2
L2
RPVST/MST
RPVST/MST
VSS
VIP
VIP
HB
HB
Node B
Site A
BRKDCT-2840
Site B
Cisco Public
20
Extended L2 VLAN
Virtual PortChannel (VSL local)
vPC or
VSS
vPC or
VSS
L2
STP Isolation
L2
L2
L2
VSS
vPC
VBS
StackWize
VBS, vPC
or VSS
VIP
HB
VIP
HB
Node B
Site A
BRKDCT-2840
Site B
Cisco Public
21
Extended L2 VLAN
Segmentation between distinct Applications
L2
Enterprise Core
WAN/MAN
Enterprise Core
L3
DC Core
No service
required
Extended L2
HA Cluster system
BRKDCT-2840
Cisco Public
22
Extended L2 VLAN
Segmentation between distinct Applications
L2
Enterprise Core
WAN/MAN
Enterprise Core
L3
DC Core
No service
required
Extended L2
HA Cluster system
BRKDCT-2840
Cisco Public
23
Options for
Layer 2
Interconnectivity
BRKDCT-2840
Cisco Public
24
EoMPLS is a pseudo-wire
CE
PE
PE
CE
MPLS
BRKDCT-2840
Cisco Public
25
PE
PE
CE
MPLS
CE
BRKDCT-2840
Cisco Public
26
Loop Prevention
Create full-mesh of Pseudo Wire VCs (EoMPLS)
Unidirectional LSP carries VCs between pair of N-PE Per
VPLS Uses split horizon concepts to prevent loops
BRKDCT-2840
Cisco Public
27
1.1.1.1
2.2.2.2
PE2
PE1
CE1
pos4/1
gi3/0
VLAN100
Self-Managed
Self-Managed
MPLS
MPLSCore
Core
pos3/0
CE2
pos4/3
gi4/4
pos3/1
VLAN100
PE3
gi4/2
3.3.3.3
CE2
VLAN100
Cisco Public
28
CE1
CE2
Subnet
192.168.20.0/24
VLAN100
VLAN100
CE2
VLAN100
Cisco Public
29
1.1.1.1
2.2.2.2
PE1
CE1
pos4/1
MPLS
MPLSCore
Core
PE2
pos4/3
gi3/0
VLAN100
CE2
gi4/4
pos3/0
pos3/1
VLAN100
PE3
gi4/2
3.3.3.3
CE2
VLAN100
Cisco Public
30
2.2.2.2
PE1
CE1
pos4/1
MPLS
MPLSCore
Core
gi3/0
pos3/0
3.3.3.3
VLAN100
CE2
pos4/3
gi4/4
VLAN100
BRKDCT-2840
PE2
Cisco Public
Interface
GigabitEthernet3/0VLAN100
pos3/1
switchport
switchport mode trunk
switchport
trunk encapsulation dot1q
PE3
switchport trunk allowed vlan 100
gi4/2
!
CE2
Interface
vlan 100
no ip address
xconnect vfi VPLS-A
!
vlan 100
state active
31
Edge
Transport
MPLS
Stack
MPLS
Header
Total
1500
14
1522
1500
18
1526
BRKDCT-2840
Cisco Public
32
WCore2
ECore1
WMC1
EMC1
Ten3/0/0
DC Core
DC Core
Ten3/0/0
Po1
WAgg1
Po4
Po3
Agg
Po1
Po2
Po3
Po4
ECore2
Po2
Ten4/0/0
Ten4/0/0
VPLS / EoMPLS
Domain
Ten4/0/0
Ten4/0/0
Po4
Po2
Po2
WAgg2
Po4
EAgg1
Po1
Ten3/0/0
WMC2
Ten3/0/0
Po3
EAgg2
Po3
Agg
Po1
EMC2
Access
Access
L2 Links (GE or 10GE)
L3 Links (GE or 10GE)
Loss of Link/Node
Server Farm
BRKDCT-2840
Server Farm
Cisco Public
33
Agg
Access
BRKDCT-2840
Cisco Public
34
DC Core
Agg
Cisco Public
35
VPLS / EoMPLS
Domain
Metro Core
Metro Core
Cisco Public
36
Ten3/0/0
Ten3/0/0
Ten4/0/0
Ten4/0/0
Ten4/0/0
Ten4/0/0
Ten3/0/0
Ten3/0/0
Metro Core
Metro Core
L3 Links (10GE)
BRKDCT-2840
Cisco Public
37
METRO CORE
DC Core
DC Core
PW Pseudo Wires
Agg
Agg
EoMPLS
Metro Core
Metro Core
Access
Access
L2 Links (GE or 10GE)
L3 Links (GE or 10GE)
Server Farm
BRKDCT-2840
Server Farm
Cisco Public
38
METRO CORE
DC Core
DC Core
VFI
Agg
Agg
PW Pseudo Wires
Metro Core
Metro Core
Access
Access
L2 Links (GE or 10GE)
L3 Links (GE or 10GE)
Server Farm
BRKDCT-2840
Server Farm
Cisco Public
39
Spanning Tree
Spanning-Tree BPDU(s) will NOT traverse
between the Data Centers It isnt needed
(and blocked) with VPLS
We still need to control data plane layer 2
events (i.e. limit the traffic)
Since enterprises want dual N-PE devices, and
VPLS blocks BPDUs, we require method to
block within a local DC
BRKDCT-2840
Cisco Public
40
BRKDCT-2840
Cisco Public
41
BRKDCT-2840
Cisco Public
42
BRKDCT-2840
Cisco Public
43
Layer 3 Core
Intranet
DC Core
DC Core
Single L2
MST Bridge
Single L2
MST Bridge
VPLS / EoMPLS
Domain
MST
MST
Agg
Agg
RSTP
RSTP
Metro Core
Metro Core
Access
Access
L2 Links (GE or 10GE)
L3 Links (GE or 10GE)
Server Farm
BRKDCT-2840
Server Farm
Cisco Public
44
Spanning-Tree
MST (802.1s) chosen to present Metro Cores as single
bridge
Red Layer 2 link is access port channel with a VLAN that
represents the MST0 instance to make the MST group
MST bridge priority set to 0 (Metro Core will be root of
Inter-DC VLANs)
Spanning tree root-guard enabled on Metro Cores toward
aggregation switches (protects in case the red MST link
fails)
Only inter-DC VLANs allowed on trunks to/from
aggregation switches
Set spanning-tree vlan cost to set the priorities on the agg
switches links to metro core will allow us to put some
VLANs on upper Metro Core, some on Lower by default
BRKDCT-2840
Cisco Public
Single L2
MST Bridge
45
BRKDCT-2840
Cisco Public
46
BRKDCT-2840
Cisco Public
47
Storm Control
Traffic storms when packets flood the LAN
Traffic storm control feature prevents LAN ports from being
disrupted by broadcast or multicast flooding
Rate limiting for unknown unicast (UU) must be handled at Data
Center aggregation; unknown unicast flood rate-limiting (UUFRL):
mls rate-limit layer2 unknown rate-in-pps [burst-size]
BRKDCT-2840
Cisco Public
48
BRKDCT-2840
Cisco Public
49
Server Farm
BRKDCT-2840
Server Farm
Cisco Public
50
Server Farm
BRKDCT-2840
Server Farm
Cisco Public
51
BRKDCT-2840
Cisco Public
52
BRKDCT-2840
Cisco Public
53
BRKDCT-2840
Cisco Public
54
BRKDCT-2840
Cisco Public
55
BRKDCT-2840
Cisco Public
56
BRKDCT-2840
Cisco Public
57
BRKDCT-2840
Cisco Public
58
BRKDCT-2840
Cisco Public
59
BRKDCT-2840
Cisco Public
60
BRKDCT-2840
Cisco Public
61
BRKDCT-2840
Cisco Public
62
BRKDCT-2840
Cisco Public
63
BRKDCT-2840
Cisco Public
64
BRKDCT-2840
Cisco Public
65
BRKDCT-2840
Cisco Public
66
BRKDCT-2840
Cisco Public
67
BRKDCT-2840
Cisco Public
68
BRKDCT-2840
Cisco Public
69
BRKDCT-2840
Cisco Public
70
BRKDCT-2840
Cisco Public
71
Virtualised extended
Data Center
Remote User Access
BRKDCT-2840
Cisco Public
72
BRKDCT-2840
Cisco Public
73
ISP B
Public network
private network
Ha_cluster_node1 Active
Ha_cluster_node2 Standby
DC primary
BRKDCT-2840
DC secondary
Cisco Public
74
BRKDCT-2840
Cisco Public
75
BRKDCT-2840
Cisco Public
76
BRKDCT-2840
Cisco Public
77
BRKDCT-2840
Cisco Public
78
BRKDCT-2840
Cisco Public
79
BRKDCT-2840
Cisco Public
80
BRKDCT-2840
Cisco Public
81
BRKDCT-2840
Cisco Public
82
BRKDCT-2840
Cisco Public
83
STP Isolation
using EEM
Cisco Public
84
Enterprise Core
Private Network
Local servers
Local servers
HA Cluster System
BRKDCT-2840
Cisco Public
85
Enterprise Core
VPLS
IP
+
MPLS
Core
N-PE Backup
N-PE Primary
N-PE Backup
Public Network
Dot1Q
EoMPLS
Private Network
Private Network
Local servers
Local servers
HA Cluster System
BRKDCT-2840
Cisco Public
86
EEM is U-PE
Active / Back-up semaphore signaling with Preemption
U-PEs
P signals Primary is up
wait a start-up delay
Backup PW is forced down
B is set down
B signals Backup is down
Primary PW is set up
__________________________
Primary
PW
y
r
a
Prim
Back
-u
p PW
Backup
Cisco Public
87
EEM in N-PE
Nominal mode
Root
N-PE
Primary
N-PE
Primary
MPLS
Core
Local
STP
Local
STP
N-PE
Back-up
N-PE
Back-up
N-PE
Primary
N-PE
Back-up
Local
STP
Root
LDP is ensuring PW
backup
for core link or node
failure
BRKDCT-2840
Root
Cisco Public
88
EEM in N-PE
P Node failure
Root
N-PE
Primary
N-PE
Primary
MPLS
Core
Local
STP
Root
Local
STP
N-PE
Back-up
N-PE
Back-up
N-PE
Primary
N-PE
Back-up
P semaphore is failing
forcing backup mode
Root
Local
STP
BRKDCT-2840
Cisco Public
89
EEM in N-PE
P Node coming back
Root
N-PE
Primary
N-PE
Primary
MPLS
Core
Local
STP
Root
Local
STP
N-PE
Back-up
B signals Backup is active
N-PE
Primary PW is maintained down
Primary
N-PE
Back-up
N-PE
Back-up
P signals Primary is up
wait a 60s start-up delay
Root
P semaphore is up,
but B is still up for a
delay, forcing backup
mode
Local
STP
BRKDCT-2840
Cisco Public
90
EEM in N-PE
P Node is operational (after probing delay)
Root
N-PE
Primary
N-PE
Primary
MPLS
Core
Local
STP
Root
Local
STP
N-PE
Back-up
N-PE
Back-up
N-PE
Primary
N-PE
Back-up
Root
P semaphore is up,
but B is still up for a
delay, forcing backup
mode
P signals Primary is up
wait a 60s start-up delay
Backup PW is forced down
B is set down
B signals Backup is down
Primary PW is set up
Local
STP
BRKDCT-2840
Cisco Public
91
VPLS Split-horizon is
protecting against loop
P
MPLS
Core
Local
STP
Root
Local
STP
Cisco Public
92
Any
STP
Root
VPLS
Non STP
domain
Dot1Q
+
BPDU filter
BRKDCT-2840
QinQ
+
VPLS
xconnect
Active / Back-up
VFI PW
EEM semaphores
Cisco Public
93
BRKDCT-2840
Cisco Public
94
WCore2
ECore1
ECore2
DC Core
DC Core
Po1
WAgg1
Agg
EAgg2
EoMPLSoGRE / VPLSoGRE
GRE Tunnels
IP
WAgg2
EAgg1
Agg
Po1
Access
Access
L2 Links (GE or 10GE)
L3 Links (GE or 10GE)
Loss of Link/Node
Server Farm
BRKDCT-2840
Server Farm
Cisco Public
95
SIP-400
IP
IPCore
Core
GR E
Tunnel
s
Site A
Si
EoMPLSLSoGRE
Site B
Si
SIP-400
EoMPLSoGRE
BRKDCT-2840
Cisco Public
96
BRKDCT-2840
Cisco Public
97
BRKDCT-2840
Cisco Public
98
L3
Core port
any Ethernet port
VPN-SPA
GRE
Crypto
VRF
edge
VPLS PW SIP-400
GRE
Wrap ports
ingres is SIP-400
egress is any port
Cisco Public
99
Layer 2 Extensions
using MultiChassis
Etherchannel
BRKDCT-2840
Cisco Public
100
BRKDCT-2840
Cisco Public
101
BRKDCT-2840
Cisco Public
102
BRKDCT-2840
Cisco Public
103
BRKDCT-2840
Cisco Public
104
BRKDCT-2840
Cisco Public
105
BRKDCT-2840
Cisco Public
106
BRKDCT-2840
Cisco Public
107
BRKDCT-2840
Cisco Public
108
BRKDCT-2840
Cisco Public
109
BRKDCT-2840
Cisco Public
110
BRKDCT-2840
Cisco Public
111
BRKDCT-2840
Cisco Public
112
BRKDCT-2840
Cisco Public
113
BRKDCT-2840
Cisco Public
114
BRKDCT-2840
Cisco Public
115
BRKDCT-2840
Cisco Public
116
BRKDCT-2840
Cisco Public
117
BRKDCT-2840
Cisco Public
118
BRKDCT-2840
Cisco Public
119
BRKDCT-2840
Cisco Public
120
BRKDCT-2840
Cisco Public
121
BRKDCT-2840
Cisco Public
122
BRKDCT-2840
Cisco Public
123
BRKDCT-2840
Cisco Public
124
BRKDCT-2840
Cisco Public
125
BRKDCT-2840
Cisco Public
126
BRKDCT-2840
Cisco Public
127
BRKDCT-2840
Cisco Public
128
BRKDCT-2840
Cisco Public
129
BRKDCT-2840
Cisco Public
130
BRKDCT-2840
Cisco Public
131
BRKDCT-2840
Cisco Public
132
BRKDCT-2840
Cisco Public
133
BRKDCT-2840
Cisco Public
134
BRKDCT-2840
Cisco Public
135
BRKDCT-2840
Cisco Public
136
BRKDCT-2840
Cisco Public
137
BRKDCT-2840
Cisco Public
138
BRKDCT-2840
Cisco Public
139
BRKDCT-2840
Cisco Public
140
BRKDCT-2840
Cisco Public
141
BRKDCT-2840
Cisco Public
142
BRKDCT-2840
Cisco Public
143
BRKDCT-2840
Cisco Public
144
BRKDCT-2840
Cisco Public
145
BRKDCT-2840
Cisco Public
146
BRKDCT-2840
Cisco Public
147
BRKDCT-2840
Cisco Public
148
BRKDCT-2840
Cisco Public
149
BRKDCT-2840
Cisco Public
150
BRKDCT-2840
Cisco Public
151
BRKDCT-2840
Cisco Public
152
BRKDCT-2840
Cisco Public
153
BRKDCT-2840
Cisco Public
154
BRKDCT-2840
Cisco Public
155
BRKDCT-2840
Cisco Public
156
BRKDCT-2840
Cisco Public
157
BRKDCT-2840
Cisco Public
158
BRKDCT-2840
Cisco Public
159
BRKDCT-2840
Cisco Public
160
BRKDCT-2840
Cisco Public
161
BRKDCT-2840
Cisco Public
162
BRKDCT-2840
Cisco Public
163
BRKDCT-2840
Cisco Public
164
BRKDCT-2840
Cisco Public
165
BRKDCT-2840
Cisco Public
166
Next Gen of
L2 extension
BRKDCT-2840
Cisco Public
167
Data
Center I
L3
L2
L3
L2
Server migrations
Disaster recovery and resiliency
High rate encryption may require
an L2 transport between sites
vMotion
BRKDCT-2840
Cisco Public
168
BRKDCT-2840
Cisco Public
169
BRKDCT-2840
Cisco Public
170
BRKDCT-2840
Cisco Public
171
BRKDCT-2840
Cisco Public
172
BRKDCT-2840
Cisco Public
173
BRKDCT-2840
Cisco Public
174
BRKDCT-2840
Cisco Public
175
BRKDCT-2840
Cisco Public
176
BRKDCT-2840
Cisco Public
177
Key Takeaways
The Key Takeaways of this presentation are:
Understand the risk to extend STP while required to
provide redundant path
Where & when to apply different L2VPN options
How Multichassis Etherchannel can help
The future technologies to extend Layer 2
BRKDCT-2840
Cisco Public
178
Q and A
BRKDCT-2840
Cisco Public
179
BRKDCT-2840
Cisco Public
180
Recommendations
Recommended Reading:
MPLS and VPN Architectures, Volume II by
Jim Guichard
Network Virtualization by Victor Moreno
Cisco Public
181
BRKDCT-2840
Cisco Public
182