Sie sind auf Seite 1von 182

Interconnecting

Multiple Datacenter
using Layer 2
Solutions

BRKDCT-2840
Yves LOUIS

BRKDCT-2840

2009 Cisco Systems, Inc. All rights reserved.

Cisco Public

Housekeeping
 We value your feedback- don't forget to complete your
online session evaluations after each session & complete
the Overall Conference Evaluation which will be available
online from Thursday
 Visit the World of Solutions
 Please remember this is a 'non-smoking' venue!
 Please switch off your mobile phones
 Please make use of the recycling bins provided
 Please remember to wear your badge at all times
including the Party

BRKDCT-2840

2009 Cisco Systems, Inc. All rights reserved.

Cisco Public

Goals of this Session


 This intermediate session details a solution for providing a means of Layer 2 extension to
support operating system HA clustering, file system clustering, symmetric traffic flows,
motion for virtual machines and more in a highly resilient multisite data center
infrastructure.
 Starting from the building blocks of spanning-tree implementations and considerations,
the session continues with details on how to control the Layer 2 control and data planes
to limit negative effects present today in geographically diverse Layer 2 domains.
 Considerations are given for tying users to either site in an active/standby, active/active
per application, and active/active within an application.
 Technologies and functions discussed include today transport mechanisms such
Ethernet over MPLS, Virtual Private LAN Service (VPLS), MPLSoGRE, Event Manager,
Virtual Switching Systems and vPort Channel as well as future technology overview.
 Additional discussions will address the remote user access to the virtual DCs extended
using L2 with Route Health Injection and Local Area Mobility, 802.1s and w, load sharing
multi-site traffic on intra-data center VLANs, global site load balancing, and others.
 This session compares alternatives with direct Layer 2 links on dedicated services or
DWDM lambdas, point-to-point and multipoint scenarios, configurations using existing
RPVST or MST deployments within a data center site, sharing Layer 2 and Layer 3
services, and operations and administration considerations.
BRKDCT-2840

2009 Cisco Systems, Inc. All rights reserved.

Cisco Public

Session Agenda
 Data Center Interconnection Common Scenarios and Terms
 Options for Layer 2 Interconnectivity
 L2 Extension Overview
 Extented L2 deployment
Scenarios/Application requiring L2 extension
Requirement and problematic
 Extented L2 Techniques overview
Dedicated Fiber, L2/L3 trunk, L2 over WAN
 L2 Extension Technlogies
MPLS usage to interconnect Data Centers
EoMPLS
VPLS
 AToM over GRE
 Virtual Switching Systems and virtual PortChannel for DC Interconnect
 Future technologies and solutions

BRKDCT-2840

2009 Cisco Systems, Inc. All rights reserved.

Cisco Public

Data Center
L2 Interconnect

BRKDCT-2840

2009 Cisco Systems, Inc. All rights reserved.

Cisco Public

Some Layer 2 Use Cases


 Extending Operating System / File System clusters
 Extending Database clusters
 Virtual machine mobility
 Physical machine mobility
 Legacy devices/apps with embedded IP addressing
 Time to deployment and operational reasons
 Extend DC to solve power/heat/space limitations

BRKDCT-2840

2009 Cisco Systems, Inc. All rights reserved.

Cisco Public

Virtual Machines
VMWARE and VMotion Requirements
VMotion
 Method used by VMWaress ESX Server to migrate active virtual machines
(VMs) within an ESX server farm from one physical ESX host to another.
 This is the foundation of several high availability features provided in
VMWares Virtual Insfrastructure product.
 Allows the movement of active VMs with minimal downtime.
 Server administrators may schedule or initiate the VMotion process manually
through the VMware VirtualCenter management tool.

State

State

BRKDCT-2840

2009 Cisco Systems, Inc. All rights reserved.

Cisco Public

BRKDCT-2840

2009 Cisco Systems, Inc. All rights reserved.

Cisco Public

BRKDCT-2840

2009 Cisco Systems, Inc. All rights reserved.

Cisco Public

HA Cluster/GeoCluster
Requirements
 * Microsoft MSCS

Common Functions

 * Veritas Cluster Server (Local)


 * Solaris Sun Cluster Enterprise

VIP address on both nodes

 VMware Cluster (Local)

Extended L2 VLAN

 Oracle RAC (Real Appl.Cluster)

Dedicated L2 used for heartbeat


& performance control

 HP MC/ServiceGuard
 HP NonStop

Quorum Disk

 HP Open VMS/TruCluster
 IBM HACMP
 EMS/Legato Automated Availability Mgr

Software is unaware of extended


members of cluster

* Veritas offers an extended Cluster solution using L3 for inter-site connectivity


Microsoft Windows Server 2008 supports L3 site to site (def=IPv6).
Sun Geographic Framework Edition
BRKDCT-2840

2009 Cisco Systems, Inc. All rights reserved.

Cisco Public

10

Layer 2 / 3 Clusters
 Intra-Cluster node communications
Flow Types: Traditionally Layer2
Communications on Private and/or Public interfaces
IPv4 and/or IPv6 possible depending on clustering package used
Ability to prioritize interfaces

 Client Access to Cluster


DNS/Active Directory resolution by clients
Shared Virtual IP for service discovery
Caching issues can inhibit Layer3 clustering
Client application can have logic to re-establish connections

 Quorum considerations to avoid split-brain


Additional cluster nodes at alternate sites to achieve a majority node set (MNS)
Possible extensions such as ping-groups (Linux-HA) to have a quorum mechanism without
a member node
Shoot The Other Node In The Head topologies to resolve conflicts (STONITH)

 Mechanisms to facilitate service restoration in another location


VMware Site Recovery Manager (SRM) is one example
Microsoft Server 2008 Layer 3 Clustering is another
Remapping of service to new IP/DNS entry
BRKDCT-2840

2009 Cisco Systems, Inc. All rights reserved.

Cisco Public

11

L2 Extension
Requirements and
Problematic

BRKDCT-2840

2009 Cisco Systems, Inc. All rights reserved.

Cisco Public

12

Layer 2 Risks
 Flooding of packets between data centers
 Rapid Spanning Tree (RSTP) is not easily scalable and
risk grows as diameter grows
 RSTP has no domain isolation issue in single DC can
propagate
 First hop resolution and inbound service selection can
cause verbose inter-data center traffic
 In general Cisco recommends L3 routing for
geographically diverse locations
 This session focuses on making limited L2 connectivity
as stable as possible
BRKDCT-2840

2009 Cisco Systems, Inc. All rights reserved.

Cisco Public

13

Layer 2 Solution Types


 Customer owned fiber to build a separate L2 network
Natively no STP isolation between sites
Multi-Chassis Etherchannel to provide redundant paths without STP

 Purchase multiple wavelengths from SP


Cost rises, still nothing to offer STP isolation

 Redesign data center RSTP domain using Multiple Spanning Tree


(MST) regions
STP domain concept
Fundamental change requiring large time investment
Operational differences and MST database management

 Implement a L2 solution to virtualize transport over L3


Virtual Switching System
EoMPLS for point to point (possible STP isolation issues)
Multipoint bridging using Virtual Private LAN Services (VPLS)
BRKDCT-2840

2009 Cisco Systems, Inc. All rights reserved.

Cisco Public

14

Geographically Dispersed Cluster


Problematic

Intranet L3
VIP

Public L2
VIP

HB

Private L2

N1
active

Storage
Ctl 1
Disk Replication

Site 1
BRKDCT-2840

2009 Cisco Systems, Inc. All rights reserved.

Cisco Public

Requirements:
L2 node to node
(VIP + Bck-up HB)
100% resilient
Extended L2 options:
VIP
1. Dedicated Fiber
Dark Fiber
Gig Ethernet
HB VIP
WDM + STP
N2
WDM + MPLS
(standby)
2. Mix L2/L3 trunk
Storage
STP
Ctl 2
3. Intranet MPLS DIY
L2/L3 VPN
VPLS
4. Intranet SP
L2VPN
Site 2
CsC
L2TPv3
15

L2 Extension
STP concerns
L2
L2

L2

RPVST or MST

RPVST or MST

VIP

VIP

HB

HB

Same extented L2 VLAN 10&11


CLUSTER 1
Node A

Node B

Site A
BRKDCT-2840

2009 Cisco Systems, Inc. All rights reserved.

Site B
Cisco Public

16

L2 Extension
Etherchannel
STP Isolation

L2
L2

L2

RPVST or MST

RPVST or MST

VIP

VIP

HB

HB

Same extented L2 VLAN 10&11


CLUSTER 1
Node A

Node B

Site A
BRKDCT-2840

2009 Cisco Systems, Inc. All rights reserved.

Site B
Cisco Public

17

Extended L2 VLAN
Cat3750 Cross-stack Etherchannel
STP Isolation

L2
L2

L2

RPVST or MST

RPVST or MST

StackWize

VIP

VIP

HB

HB

Same extented L2 VLAN 10&11


CLUSTER 1
Node A

Node B

Site A
BRKDCT-2840

2009 Cisco Systems, Inc. All rights reserved.

Site B
Cisco Public

18

Extended L2 VLAN
Virtual Switching with MEC (VSL local)
L2

STP Isolation

L2

L2

RPVST/MST

RPVST/MST

VSS

VIP

VIP

HB

HB

Same extented L2 VLAN 10&11


CLUSTER 1
Node A

Node B

Site A
BRKDCT-2840

2009 Cisco Systems, Inc. All rights reserved.

Site B
Cisco Public

19

Extended L2 VLAN
Virtual PortChannel (MCEC)
L2

STP Isolation

L2

L2

RPVST/MST

RPVST/MST

VSS

VIP

VIP

HB

HB

Same extented L2 VLAN 10&11


CLUSTER 1
Node A

Node B

Site A
BRKDCT-2840

2009 Cisco Systems, Inc. All rights reserved.

Site B
Cisco Public

20

Extended L2 VLAN
Virtual PortChannel (VSL local)
vPC or
VSS
vPC or
VSS

L2

STP Isolation

L2
L2

L2
VSS
vPC
VBS
StackWize

VBS, vPC
or VSS
VIP

HB

VIP

HB

Same extented L2 VLAN 10&11


CLUSTER 1
Node A

Node B

Site A
BRKDCT-2840

2009 Cisco Systems, Inc. All rights reserved.

Site B
Cisco Public

21

Extended L2 VLAN
Segmentation between distinct Applications
L2

Enterprise Core

WAN/MAN

Enterprise Core

L3

DC Core

Network & Security


Services required

No service
required

Extended L2

HA Cluster system

BRKDCT-2840

2009 Cisco Systems, Inc. All rights reserved.

Cisco Public

22

Extended L2 VLAN
Segmentation between distinct Applications
L2

Enterprise Core

WAN/MAN

Enterprise Core

L3

DC Core

Network & Security


Services required

No service
required

Extended L2

HA Cluster system

BRKDCT-2840

2009 Cisco Systems, Inc. All rights reserved.

Cisco Public

23

Options for
Layer 2
Interconnectivity

BRKDCT-2840

2009 Cisco Systems, Inc. All rights reserved.

Cisco Public

24

EoMPLS (Ethernet over MPLS)


 Encapsulates Ethernet frames inside MPLS packets to pass layer
3 network
 EoMPLS has routing separation from metro core devices providing
connectivity CE flapping routes wont propogate inside MPLS
 Point to point links between locations
 Data plane rate limiting in L2 still needs protection

EoMPLS is a pseudo-wire
CE

PE

PE

CE

MPLS

BRKDCT-2840

2009 Cisco Systems, Inc. All rights reserved.

Cisco Public

25

Virtual Private LAN Service (VPLS)


 VPLS defines an architecture allows MPLS networks offer Layer 2
multipoint Ethernet Services
 Metro Core emulates an IEEE Ethernet bridge (virtual)
 Virtual Bridges linked with EoMPLS Pseudo Wires
 Data plane rate limiting in L2 still needs protection
VPLS is an Architecture
CE

PE

PE

CE

MPLS

CE
BRKDCT-2840

2009 Cisco Systems, Inc. All rights reserved.

Cisco Public

26

Virtual Forwarding Instance (VFI)


 IOS Representation of Virtual Switch Interface
 Flooding / Forwarding
MAC table instances per customer (port/vlan) for each PE
VFI will participate in learning and forwarding process
Associate ports to MAC, flood unknowns to all other ports

 Address Learning / Aging


LDP enhanced with additional MAC List TLV (label withdrawal)
MAC timers refreshed with incoming frames

 Loop Prevention
Create full-mesh of Pseudo Wire VCs (EoMPLS)
Unidirectional LSP carries VCs between pair of N-PE Per
VPLS Uses split horizon concepts to prevent loops
BRKDCT-2840

2009 Cisco Systems, Inc. All rights reserved.

Cisco Public

27

Direct Attachment Configuration

1.1.1.1

2.2.2.2

PE2

PE1

CE1

pos4/1
gi3/0

VLAN100

Self-Managed
Self-Managed
MPLS
MPLSCore
Core
pos3/0

CE2

pos4/3
gi4/4

pos3/1

VLAN100

PE3
gi4/2
3.3.3.3

CE2

VLAN100

 CEs are all part of same VPLS instance (VCID = 56)


CE router connects using VLAN 100 over sub-interface
BRKDCT-2840

2009 Cisco Systems, Inc. All rights reserved.

Cisco Public

28

Direct Attachment CE Router


Configuration
interface GigabitEthernet 2/1.100
encapsulation dot1q 100
ip address 192.168.20.1

interface GigabitEthernet 1/3.100


encapsulation dot1q 100
ip address 192.168.20.2

CE1

CE2
Subnet
192.168.20.0/24

VLAN100

VLAN100

interface GigabitEthernet 2/0.100


encapsulation dot1q 100
ip address 192.168.20.3

CE2
VLAN100

 CE routers sub-interface on same VLAN


Can also be just port based (NO VLAN)
BRKDCT-2840

2009 Cisco Systems, Inc. All rights reserved.

Cisco Public

29

Direct Attachment VFI Configuration


l2 vfi VPLS-A manual
vpn id 56
neighbor 2.2.2.2 encapsulation mpls
neighbor 3.3.3.3 encapsulation mpls

l2 vfi VPLS-A manual


vpn id 56
neighbor 1.1.1.1 encapsulation mpls
neighbor 3.3.3.3 encapsulation mpls

1.1.1.1

2.2.2.2

PE1

CE1

pos4/1

MPLS
MPLSCore
Core

PE2
pos4/3

gi3/0
VLAN100

CE2
gi4/4

pos3/0

pos3/1

VLAN100

PE3
gi4/2
3.3.3.3

CE2

VLAN100

 Create the Pseudo


Wires between N-PE
routers
BRKDCT-2840

2009 Cisco Systems, Inc. All rights reserved.

Cisco Public

l2 vfi VPLS-A manual


vpn id 56
neighbor 2.2.2.2 encapsulation mpls
neighbor 1.1.1.1 encapsulation mpls

30

Direct Attachment CE Router (VLAN


Based)
 Same set of commands on each PE
 Configured on the CE facing interface
1.1.1.1

2.2.2.2

PE1

CE1

pos4/1

MPLS
MPLSCore
Core

gi3/0
pos3/0

3.3.3.3
VLAN100

This command associates the


VLAN with the VPLS instance
VLAN100 = VCID 56

2009 Cisco Systems, Inc. All rights reserved.

CE2

pos4/3
gi4/4

VLAN100

BRKDCT-2840

PE2

Cisco Public

Interface
GigabitEthernet3/0VLAN100
pos3/1
switchport
switchport mode trunk
switchport
trunk encapsulation dot1q
PE3
switchport trunk allowed vlan 100
gi4/2
!
CE2
Interface
vlan 100
no ip address
xconnect vfi VPLS-A
!
vlan 100
state active

31

Calculating Core MTU Requirements


 Core MTU Edge MTU + Transport Header + (MPLS Label Stack
* MPLS Header Size)
 Edge MTU is the MTU configured in the CE-facing PE interface
 Examples (all in Bytes):

Edge

Transport

MPLS
Stack

MPLS
Header

Total

EoMPLS Port Mode

1500

14

1522

EoMPLS VLAN Mode

1500

18

1526

BRKDCT-2840

2009 Cisco Systems, Inc. All rights reserved.

Cisco Public

32

End to End VPLS and EoMPLS Design


Layer 3 Core
Intranet
WCore1

WCore2

ECore1

WMC1

EMC1
Ten3/0/0

DC Core

DC Core

Ten3/0/0

Po1
WAgg1

Po4
Po3

Agg

Po1
Po2

Po3

Po4

ECore2

Po2

Ten4/0/0

Ten4/0/0

VPLS / EoMPLS
Domain
Ten4/0/0

Ten4/0/0

Po4

Po2
Po2

WAgg2

Po4
EAgg1

Po1

Ten3/0/0
WMC2

Ten3/0/0

Po3

EAgg2

Po3
Agg

Po1
EMC2

Access

Access
L2 Links (GE or 10GE)
L3 Links (GE or 10GE)
Loss of Link/Node

Server Farm

BRKDCT-2840

2009 Cisco Systems, Inc. All rights reserved.

Server Farm

Cisco Public

33

Access to Aggregation Connections


 Rapid PVST is existing protocol, and no
desire to force a change
 Aggregation switches are root for all
intra-DC VLANs
 The peer aggregation switch is
secondary root
 DO NOT customize the bridge priorities
to lower values

Agg

Access

 HSRP used for first hop redundancy


from server
Server Farm

BRKDCT-2840

2009 Cisco Systems, Inc. All rights reserved.

Cisco Public

34

Layer 3 Aggregation and Core


Connections
IF dual supervisor modules, need non-stop forwarding (NSF)
under routing process
Layer 3
Enterprise Core

 Layer 3 connections from DC


Core to Enterprise Core
 Aggregation switch L3
connected to DC Core
 Hanging L3 links in diagram,
are to Metro Core switches
which are Ethernet over
MPLS links

DC Core

 Hanging L3 links are for


peering the DC Cores in
each location in a point-topoint scenario

Agg

Bidirectional forwarding detection (bfd)


interval 100 min_rx 100 multiplier 3
BRKDCT-2840

2009 Cisco Systems, Inc. All rights reserved.

Cisco Public

35

EoMPLS / VPLS Infrastructure


 Loopbacks chosen as peering points for
EoMPLS and VPLS xconnects
 Horizontal links represent 10GE on
DWDM service between data centers
(alternate paths)
 Vertical links represent intra-DC 10GE
connections

VPLS / EoMPLS
Domain

 MPLS LDP enabled globally (not a full P /


PE MPLS implmentation)
 Links to/from aggregation switches for
Layer 2, are storm-control limited for
broadcasts and multicasts to 1% (protect
data plane)

Metro Core

Metro Core

 MTU increased to 1522 bytes on the L3


MPLS links for the MPLS tagging
BRKDCT-2840

2009 Cisco Systems, Inc. All rights reserved.

Cisco Public

36

Metro Switch Interconnectivity

Ten3/0/0

Ten3/0/0

Ten4/0/0

IGP Routing Process


connecting
MPLS PEs

Ten4/0/0
Ten4/0/0

Ten4/0/0

Ten3/0/0

Ten3/0/0

Metro Core

Metro Core

L3 Links (10GE)

BRKDCT-2840

2009 Cisco Systems, Inc. All rights reserved.

Cisco Public

37

EoMPLS for Layer3


Layer 3 Core
Intranet

METRO CORE

DC Core

DC Core

PW Pseudo Wires
Agg

Agg

EoMPLS

Metro Core

Metro Core

Access

Access
L2 Links (GE or 10GE)
L3 Links (GE or 10GE)
Server Farm

BRKDCT-2840

2009 Cisco Systems, Inc. All rights reserved.

Server Farm

Cisco Public

38

VPLS for Layer2


Layer 3 Core
Intranet

METRO CORE

DC Core

DC Core

VFI
Agg

Agg

PW Pseudo Wires
Metro Core

Metro Core

Access

Access
L2 Links (GE or 10GE)
L3 Links (GE or 10GE)
Server Farm

BRKDCT-2840

2009 Cisco Systems, Inc. All rights reserved.

Server Farm

Cisco Public

39

Spanning Tree
 Spanning-Tree BPDU(s) will NOT traverse
between the Data Centers It isnt needed
(and blocked) with VPLS
 We still need to control data plane layer 2
events (i.e. limit the traffic)
 Since enterprises want dual N-PE devices, and
VPLS blocks BPDUs, we require method to
block within a local DC

BRKDCT-2840

2009 Cisco Systems, Inc. All rights reserved.

Cisco Public

40

BRKDCT-2840

2009 Cisco Systems, Inc. All rights reserved.

Cisco Public

41

BRKDCT-2840

2009 Cisco Systems, Inc. All rights reserved.

Cisco Public

42

BRKDCT-2840

2009 Cisco Systems, Inc. All rights reserved.

Cisco Public

43

Spanning Tree - MST


Root Bridge in West DC
for all VLANs that Go
Between Data Centers

Root Bridge in East DC


for all VLANs that Go
Between Data Centers

Layer 3 Core
Intranet

DC Core

DC Core
Single L2
MST Bridge

Single L2
MST Bridge

VPLS / EoMPLS
Domain

MST

MST
Agg

Agg

RSTP

RSTP
Metro Core

Metro Core

Access

Access
L2 Links (GE or 10GE)
L3 Links (GE or 10GE)
Server Farm

BRKDCT-2840

2009 Cisco Systems, Inc. All rights reserved.

Server Farm

Cisco Public

44

Spanning-Tree
 MST (802.1s) chosen to present Metro Cores as single
bridge
 Red Layer 2 link is access port channel with a VLAN that
represents the MST0 instance to make the MST group
 MST bridge priority set to 0 (Metro Core will be root of
Inter-DC VLANs)
 Spanning tree root-guard enabled on Metro Cores toward
aggregation switches (protects in case the red MST link
fails)
 Only inter-DC VLANs allowed on trunks to/from
aggregation switches
 Set spanning-tree vlan cost to set the priorities on the agg
switches links to metro core will allow us to put some
VLANs on upper Metro Core, some on Lower by default

BRKDCT-2840

2009 Cisco Systems, Inc. All rights reserved.

Cisco Public

Single L2
MST Bridge

45

BRKDCT-2840

2009 Cisco Systems, Inc. All rights reserved.

Cisco Public

46

BRKDCT-2840

2009 Cisco Systems, Inc. All rights reserved.

Cisco Public

47

Storm Control
 Traffic storms when packets flood the LAN
 Traffic storm control feature prevents LAN ports from being
disrupted by broadcast or multicast flooding
 Rate limiting for unknown unicast (UU) must be handled at Data
Center aggregation; unknown unicast flood rate-limiting (UUFRL):
mls rate-limit layer2 unknown rate-in-pps [burst-size]

 Storm Control is configured as a percentage of the link that storm


traffic is allowed to use.
storm-control broadcast level 1.00 (% of b/w may vary need to
baseline)
storm-control multicast level 1.00 (% of b/w may vary need to
baseline)

BRKDCT-2840

2009 Cisco Systems, Inc. All rights reserved.

Cisco Public

48

3 or More Data Center Locations


 EoMPLS will allow multiple point to point links between
any 2 sites
 Can build a full mesh of links to interconnect layer 3
devices
 VPLS scales by adding peer xconnects under the VFI
in the IOS configuration
 Split horizon with MST local to data center will make for
simple growth
 Limits dependant on amounts of L2 traffic especially
multicast, as these are replicated on each PW

BRKDCT-2840

2009 Cisco Systems, Inc. All rights reserved.

Cisco Public

49

3 Site Drawing with EoMPLS PWs for L3


Server Farm

Server Farm
BRKDCT-2840

2009 Cisco Systems, Inc. All rights reserved.

Server Farm
Cisco Public

50

3 Site Drawing with VPLS PWs for L2


Server Farm

Server Farm
BRKDCT-2840

2009 Cisco Systems, Inc. All rights reserved.

Server Farm
Cisco Public

51

BRKDCT-2840

2009 Cisco Systems, Inc. All rights reserved.

Cisco Public

52

BRKDCT-2840

2009 Cisco Systems, Inc. All rights reserved.

Cisco Public

53

BRKDCT-2840

2009 Cisco Systems, Inc. All rights reserved.

Cisco Public

54

BRKDCT-2840

2009 Cisco Systems, Inc. All rights reserved.

Cisco Public

55

BRKDCT-2840

2009 Cisco Systems, Inc. All rights reserved.

Cisco Public

56

BRKDCT-2840

2009 Cisco Systems, Inc. All rights reserved.

Cisco Public

57

BRKDCT-2840

2009 Cisco Systems, Inc. All rights reserved.

Cisco Public

58

BRKDCT-2840

2009 Cisco Systems, Inc. All rights reserved.

Cisco Public

59

BRKDCT-2840

2009 Cisco Systems, Inc. All rights reserved.

Cisco Public

60

BRKDCT-2840

2009 Cisco Systems, Inc. All rights reserved.

Cisco Public

61

BRKDCT-2840

2009 Cisco Systems, Inc. All rights reserved.

Cisco Public

62

BRKDCT-2840

2009 Cisco Systems, Inc. All rights reserved.

Cisco Public

63

BRKDCT-2840

2009 Cisco Systems, Inc. All rights reserved.

Cisco Public

64

BRKDCT-2840

2009 Cisco Systems, Inc. All rights reserved.

Cisco Public

65

BRKDCT-2840

2009 Cisco Systems, Inc. All rights reserved.

Cisco Public

66

BRKDCT-2840

2009 Cisco Systems, Inc. All rights reserved.

Cisco Public

67

BRKDCT-2840

2009 Cisco Systems, Inc. All rights reserved.

Cisco Public

68

BRKDCT-2840

2009 Cisco Systems, Inc. All rights reserved.

Cisco Public

69

BRKDCT-2840

2009 Cisco Systems, Inc. All rights reserved.

Cisco Public

70

BRKDCT-2840

2009 Cisco Systems, Inc. All rights reserved.

Cisco Public

71

Virtualised extended
Data Center
Remote User Access

BRKDCT-2840

2009 Cisco Systems, Inc. All rights reserved.

Cisco Public

72

BRKDCT-2840

2009 Cisco Systems, Inc. All rights reserved.

Cisco Public

73

Ha_cluster access: inbound traffic via same


primary DC
ISP A

ISP B

Public network
private network
Ha_cluster_node1 Active

Ha_cluster_node2 Standby

DC primary
BRKDCT-2840

2009 Cisco Systems, Inc. All rights reserved.

DC secondary
Cisco Public

74

BRKDCT-2840

2009 Cisco Systems, Inc. All rights reserved.

Cisco Public

75

BRKDCT-2840

2009 Cisco Systems, Inc. All rights reserved.

Cisco Public

76

BRKDCT-2840

2009 Cisco Systems, Inc. All rights reserved.

Cisco Public

77

BRKDCT-2840

2009 Cisco Systems, Inc. All rights reserved.

Cisco Public

78

BRKDCT-2840

2009 Cisco Systems, Inc. All rights reserved.

Cisco Public

79

BRKDCT-2840

2009 Cisco Systems, Inc. All rights reserved.

Cisco Public

80

BRKDCT-2840

2009 Cisco Systems, Inc. All rights reserved.

Cisco Public

81

BRKDCT-2840

2009 Cisco Systems, Inc. All rights reserved.

Cisco Public

82

BRKDCT-2840

2009 Cisco Systems, Inc. All rights reserved.

Cisco Public

83

STP Isolation
using EEM

Credits go to Patrice Bellagamba


BRKDCT-2840

2009 Cisco Systems, Inc. All rights reserved.

Cisco Public

84

Common Geocluster interconnection


Enterprise Core

Enterprise Core

Public Network with STP

Private Network
Local servers

Local servers

HA Cluster System
BRKDCT-2840

2009 Cisco Systems, Inc. All rights reserved.

Cisco Public

85

Common core with MPLS multiplexing


Enterprise Core

Enterprise Core

Extended Public Network


N-PE Primary

VPLS
IP
+
MPLS
Core

N-PE Backup

N-PE Primary

N-PE Backup

Extended Private Network


Public Network

Public Network

Dot1Q

EoMPLS
Private Network

Private Network

Local servers

Local servers

HA Cluster System
BRKDCT-2840

2009 Cisco Systems, Inc. All rights reserved.

Cisco Public

86

EEM is U-PE
Active / Back-up semaphore signaling with Preemption
U-PEs

P signals Primary is up
wait a start-up delay
Backup PW is forced down
B is set down
B signals Backup is down
Primary PW is set up
__________________________

Primary

P signals Primary is down


Backup PW is set up
B is set up immediatly
B signals Backup is active
Primary PW is forced down

PW
y
r
a
Prim

Back
-u

p PW

Backup

Semaphores role is to synch Primary & Backup


Ensure fast backup and return in function
Prohibit Active/Active state (to prevent loops)
BRKDCT-2840

2009 Cisco Systems, Inc. All rights reserved.

Cisco Public

87

EEM in N-PE
Nominal mode
Root

N-PE
Primary

N-PE
Primary

MPLS
Core

Local
STP

Local
STP

N-PE
Back-up

N-PE
Back-up

N-PE
Primary

N-PE
Back-up

B signals Backup is down


Primary PW is set up
P signals Primary is up
Backup PW is forced down
B is set down

Local
STP

2009 Cisco Systems, Inc. All rights reserved.

Root

LDP is ensuring PW
backup
for core link or node
failure
BRKDCT-2840

Root

Cisco Public

88

EEM in N-PE
P Node failure
Root

N-PE
Primary

N-PE
Primary

MPLS
Core

Local
STP

Root

Local
STP

N-PE
Back-up

N-PE
Back-up

N-PE
Primary

N-PE
Back-up

P semaphore is failing
forcing backup mode

P signals Primary is down


Backup PW is set up
B is set up immediatly

Root

Local
STP

BRKDCT-2840

2009 Cisco Systems, Inc. All rights reserved.

Cisco Public

89

EEM in N-PE
P Node coming back
Root

N-PE
Primary

N-PE
Primary

MPLS
Core

Local
STP

Root

Local
STP

N-PE
Back-up
B signals Backup is active
N-PE
Primary PW is maintained down
Primary

N-PE
Back-up
N-PE
Back-up

P signals Primary is up
wait a 60s start-up delay
Root

P semaphore is up,
but B is still up for a
delay, forcing backup
mode
Local
STP

BRKDCT-2840

2009 Cisco Systems, Inc. All rights reserved.

Cisco Public

90

EEM in N-PE
P Node is operational (after probing delay)
Root

N-PE
Primary

N-PE
Primary

MPLS
Core

Local
STP

Root

Local
STP

N-PE
Back-up

N-PE
Back-up

N-PE
Primary

N-PE
Back-up

Root

P semaphore is up,
but B is still up for a
delay, forcing backup
mode

P signals Primary is up
wait a 60s start-up delay
Backup PW is forced down
B is set down
B signals Backup is down
Primary PW is set up

Local
STP

BRKDCT-2840

2009 Cisco Systems, Inc. All rights reserved.

Cisco Public

91

No loop even on unexpected


Active/Active state
Root

VPLS Split-horizon is
protecting against loop

P
MPLS
Core

Local
STP

Root

Local
STP

Unexpected Split-brain !!!


with local semaphores
+ Tie-break failures

Remote back-up node is


inactive

No loop can be created by a one site split-brain


BRKDCT-2840

2009 Cisco Systems, Inc. All rights reserved.

Cisco Public

92

EEM for PW-redundancy


Spanning-tree isolation
EEM
Active / Back-up

Any
STP
Root

VPLS
Non STP
domain

Dot1Q
+
BPDU filter

BRKDCT-2840

QinQ
+
VPLS
xconnect

2009 Cisco Systems, Inc. All rights reserved.

Active / Back-up
VFI PW
EEM semaphores

Cisco Public

93

AToM over GRE


And Encryption

BRKDCT-2840

2009 Cisco Systems, Inc. All rights reserved.

Cisco Public

94

AToM over GRE


VPLS and EoMPLS over GRE Design
Layer 3 Core
Intranet
WCore1

WCore2

ECore1

ECore2

DC Core

DC Core
Po1
WAgg1

Agg

EAgg2

EoMPLSoGRE / VPLSoGRE
GRE Tunnels
IP

WAgg2

EAgg1

Agg

Po1

Access

Access
L2 Links (GE or 10GE)
L3 Links (GE or 10GE)
Loss of Link/Node

Server Farm

BRKDCT-2840

2009 Cisco Systems, Inc. All rights reserved.

Server Farm

Cisco Public

95

DC Interconnect using EoMPLSoGRE


12.2(33)SXI feature
Edge only functionality using SIP400

SIP-400

IP
IPCore
Core

GR E
Tunnel
s

Site A
Si

EoMPLSLSoGRE

Site B
Si

SIP-400
EoMPLSoGRE

BRKDCT-2840

2009 Cisco Systems, Inc. All rights reserved.

Cisco Public

96

BRKDCT-2840

2009 Cisco Systems, Inc. All rights reserved.

Cisco Public

97

BRKDCT-2840

2009 Cisco Systems, Inc. All rights reserved.

Cisco Public

98

VPLS o GRE o IPSEC in one box


with wrap-cable
VRF
core

L3

Core port
any Ethernet port

VPN-SPA

GRE

Crypto
VRF
edge

VPLS PW SIP-400

GRE

Wrap ports
ingres is SIP-400
egress is any port

To integrate both function in one box:


1. Use VRFs to isolate routing
One VRF for edge link
One VRF for Core links

2. Wrap cable to connect SIP-400 toward VRF


BRKDCT-2840

2009 Cisco Systems, Inc. All rights reserved.

Cisco Public

99

Layer 2 Extensions
using MultiChassis
Etherchannel

BRKDCT-2840

2009 Cisco Systems, Inc. All rights reserved.

Cisco Public

100

BRKDCT-2840

2009 Cisco Systems, Inc. All rights reserved.

Cisco Public

101

BRKDCT-2840

2009 Cisco Systems, Inc. All rights reserved.

Cisco Public

102

BRKDCT-2840

2009 Cisco Systems, Inc. All rights reserved.

Cisco Public

103

BRKDCT-2840

2009 Cisco Systems, Inc. All rights reserved.

Cisco Public

104

BRKDCT-2840

2009 Cisco Systems, Inc. All rights reserved.

Cisco Public

105

BRKDCT-2840

2009 Cisco Systems, Inc. All rights reserved.

Cisco Public

106

BRKDCT-2840

2009 Cisco Systems, Inc. All rights reserved.

Cisco Public

107

BRKDCT-2840

2009 Cisco Systems, Inc. All rights reserved.

Cisco Public

108

BRKDCT-2840

2009 Cisco Systems, Inc. All rights reserved.

Cisco Public

109

BRKDCT-2840

2009 Cisco Systems, Inc. All rights reserved.

Cisco Public

110

BRKDCT-2840

2009 Cisco Systems, Inc. All rights reserved.

Cisco Public

111

MST & Flexlink


using EEM
For STP Isolation

BRKDCT-2840

2009 Cisco Systems, Inc. All rights reserved.

Cisco Public

112

BRKDCT-2840

2009 Cisco Systems, Inc. All rights reserved.

Cisco Public

113

BRKDCT-2840

2009 Cisco Systems, Inc. All rights reserved.

Cisco Public

114

BRKDCT-2840

2009 Cisco Systems, Inc. All rights reserved.

Cisco Public

115

BRKDCT-2840

2009 Cisco Systems, Inc. All rights reserved.

Cisco Public

116

BRKDCT-2840

2009 Cisco Systems, Inc. All rights reserved.

Cisco Public

117

BRKDCT-2840

2009 Cisco Systems, Inc. All rights reserved.

Cisco Public

118

BRKDCT-2840

2009 Cisco Systems, Inc. All rights reserved.

Cisco Public

119

BRKDCT-2840

2009 Cisco Systems, Inc. All rights reserved.

Cisco Public

120

BRKDCT-2840

2009 Cisco Systems, Inc. All rights reserved.

Cisco Public

121

BRKDCT-2840

2009 Cisco Systems, Inc. All rights reserved.

Cisco Public

122

BRKDCT-2840

2009 Cisco Systems, Inc. All rights reserved.

Cisco Public

123

BRKDCT-2840

2009 Cisco Systems, Inc. All rights reserved.

Cisco Public

124

BRKDCT-2840

2009 Cisco Systems, Inc. All rights reserved.

Cisco Public

125

BRKDCT-2840

2009 Cisco Systems, Inc. All rights reserved.

Cisco Public

126

BRKDCT-2840

2009 Cisco Systems, Inc. All rights reserved.

Cisco Public

127

BRKDCT-2840

2009 Cisco Systems, Inc. All rights reserved.

Cisco Public

128

BRKDCT-2840

2009 Cisco Systems, Inc. All rights reserved.

Cisco Public

129

BRKDCT-2840

2009 Cisco Systems, Inc. All rights reserved.

Cisco Public

130

BRKDCT-2840

2009 Cisco Systems, Inc. All rights reserved.

Cisco Public

131

BRKDCT-2840

2009 Cisco Systems, Inc. All rights reserved.

Cisco Public

132

BRKDCT-2840

2009 Cisco Systems, Inc. All rights reserved.

Cisco Public

133

BRKDCT-2840

2009 Cisco Systems, Inc. All rights reserved.

Cisco Public

134

BRKDCT-2840

2009 Cisco Systems, Inc. All rights reserved.

Cisco Public

135

BRKDCT-2840

2009 Cisco Systems, Inc. All rights reserved.

Cisco Public

136

BRKDCT-2840

2009 Cisco Systems, Inc. All rights reserved.

Cisco Public

137

BRKDCT-2840

2009 Cisco Systems, Inc. All rights reserved.

Cisco Public

138

BRKDCT-2840

2009 Cisco Systems, Inc. All rights reserved.

Cisco Public

139

BRKDCT-2840

2009 Cisco Systems, Inc. All rights reserved.

Cisco Public

140

BRKDCT-2840

2009 Cisco Systems, Inc. All rights reserved.

Cisco Public

141

BRKDCT-2840

2009 Cisco Systems, Inc. All rights reserved.

Cisco Public

142

BRKDCT-2840

2009 Cisco Systems, Inc. All rights reserved.

Cisco Public

143

BRKDCT-2840

2009 Cisco Systems, Inc. All rights reserved.

Cisco Public

144

BRKDCT-2840

2009 Cisco Systems, Inc. All rights reserved.

Cisco Public

145

BRKDCT-2840

2009 Cisco Systems, Inc. All rights reserved.

Cisco Public

146

BRKDCT-2840

2009 Cisco Systems, Inc. All rights reserved.

Cisco Public

147

BRKDCT-2840

2009 Cisco Systems, Inc. All rights reserved.

Cisco Public

148

BRKDCT-2840

2009 Cisco Systems, Inc. All rights reserved.

Cisco Public

149

BRKDCT-2840

2009 Cisco Systems, Inc. All rights reserved.

Cisco Public

150

BRKDCT-2840

2009 Cisco Systems, Inc. All rights reserved.

Cisco Public

151

BRKDCT-2840

2009 Cisco Systems, Inc. All rights reserved.

Cisco Public

152

BRKDCT-2840

2009 Cisco Systems, Inc. All rights reserved.

Cisco Public

153

BRKDCT-2840

2009 Cisco Systems, Inc. All rights reserved.

Cisco Public

154

BRKDCT-2840

2009 Cisco Systems, Inc. All rights reserved.

Cisco Public

155

BRKDCT-2840

2009 Cisco Systems, Inc. All rights reserved.

Cisco Public

156

BRKDCT-2840

2009 Cisco Systems, Inc. All rights reserved.

Cisco Public

157

BRKDCT-2840

2009 Cisco Systems, Inc. All rights reserved.

Cisco Public

158

BRKDCT-2840

2009 Cisco Systems, Inc. All rights reserved.

Cisco Public

159

BRKDCT-2840

2009 Cisco Systems, Inc. All rights reserved.

Cisco Public

160

BRKDCT-2840

2009 Cisco Systems, Inc. All rights reserved.

Cisco Public

161

BRKDCT-2840

2009 Cisco Systems, Inc. All rights reserved.

Cisco Public

162

BRKDCT-2840

2009 Cisco Systems, Inc. All rights reserved.

Cisco Public

163

BRKDCT-2840

2009 Cisco Systems, Inc. All rights reserved.

Cisco Public

164

BRKDCT-2840

2009 Cisco Systems, Inc. All rights reserved.

Cisco Public

165

BRKDCT-2840

2009 Cisco Systems, Inc. All rights reserved.

Cisco Public

166

Next Gen of
L2 extension

BRKDCT-2840

2009 Cisco Systems, Inc. All rights reserved.

Cisco Public

167

Concept LAN extensions: Inter/intra Data Center


Data
Center II

Data
Center I

L3
L2

L3
L2

 Certain Applications require L2


connectivity among peers
Clusters (Veritas, MSFT)

Home-brewed apps, non-IP sync


2009 Cisco Systems, Inc. All rights reserved.

 Server migrations
 Disaster recovery and resiliency
 High rate encryption may require
an L2 transport between sites

vMotion
BRKDCT-2840

Cisco Public

 Distributed Active-active DCs

168

BRKDCT-2840

2009 Cisco Systems, Inc. All rights reserved.

Cisco Public

169

BRKDCT-2840

2009 Cisco Systems, Inc. All rights reserved.

Cisco Public

170

BRKDCT-2840

2009 Cisco Systems, Inc. All rights reserved.

Cisco Public

171

BRKDCT-2840

2009 Cisco Systems, Inc. All rights reserved.

Cisco Public

172

BRKDCT-2840

2009 Cisco Systems, Inc. All rights reserved.

Cisco Public

173

BRKDCT-2840

2009 Cisco Systems, Inc. All rights reserved.

Cisco Public

174

BRKDCT-2840

2009 Cisco Systems, Inc. All rights reserved.

Cisco Public

175

BRKDCT-2840

2009 Cisco Systems, Inc. All rights reserved.

Cisco Public

176

BRKDCT-2840

2009 Cisco Systems, Inc. All rights reserved.

Cisco Public

177

Key Takeaways
The Key Takeaways of this presentation are:
 Understand the risk to extend STP while required to
provide redundant path
 Where & when to apply different L2VPN options
 How Multichassis Etherchannel can help
 The future technologies to extend Layer 2

BRKDCT-2840

2009 Cisco Systems, Inc. All rights reserved.

Cisco Public

178

Q and A

BRKDCT-2840

2009 Cisco Systems, Inc. All rights reserved.

Cisco Public

179

Meet The Expert


To make the most of your time at Cisco Networkers 2009,
schedule a Face-to-Face Meeting with a top Cisco Expert.
Designed to provide a "big picture" perspective as well as
"in-depth" technology discussions, these face-to-face
meetings will provide fascinating dialogue and a wealth of
valuable insights and ideas.
Visit the Meeting Centre reception desk located in the
Meeting Centre in World of Solutions

BRKDCT-2840

2009 Cisco Systems, Inc. All rights reserved.

Cisco Public

180

Recommendations
 Recommended Reading:
MPLS and VPN Architectures, Volume II by
Jim Guichard
Network Virtualization by Victor Moreno

 Check the Recommended Reading flyer for


suggested books
 Related technology breakouts:

BRKIPM-3071 "Avanced MPLS


designs" with Patrice Bellagamba
 NSITE is compiling test results for both the
MST N-PE and EEM Solution

Available Onsite at the Cisco Company Store


BRKDCT-2840

2009 Cisco Systems, Inc. All rights reserved.

Cisco Public

181

BRKDCT-2840

2009 Cisco Systems, Inc. All rights reserved.

Cisco Public

182

Das könnte Ihnen auch gefallen