Sie sind auf Seite 1von 155

Cisco SD-Access -

Connecting the Fabric to


External Networks

Satish Kondalam, Technical Marketing Engineer


BRKCRS-2811
Cisco Spark
Questions?
Use Cisco Spark to communicate
with the speaker after the session

How
1. Find this session in the Cisco Live Mobile App
2. Click “Join the Discussion”
3. Install Spark or go directly to the space
4. Enter messages/questions in the space

© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public
Cisco’s Intent-based Networking
Learning

DNA Centre
The Network. Intuitive.
Policy Automation Analytics

Intent Context

Network Infrastructure
Powered by Intent.
Informed by Context.
Switching Routers Wireless

Security
BRKCRS-2811 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 4
Software Defined Access
Networking at the speed of Software!

DNA Centre

Identity-Based
Policy & Segmentation
Policy Automatio Analytic
n s Decoupled security policy from
VLAN and IP Address

B B
C Outside Automated
Network Fabric
Single Fabric for Wired & Wireless
with workflow Automation

Insights
SDA
Extension
& Telemetry
User Mobility

Policy stays
Analytics and Insights into
with user User and Application behaviour
© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 4
IoT Network Employee Network BRKCRS-2811
BRKCRS-2811 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 5
What is SD-Access?
Campus Fabric + DNA Centre (Automation & Assurance)
 SD-Access – Available Aug 2017
DNAC
APIC-EM
1.X
GUI approach provides automation &
assurance of all Fabric configuration,
ISE NDP management and group-based policy.
DNA Centre Leverages DNA Centre to integrate
external Service Apps, to orchestrate
your entire LAN, Wireless LAN and
WAN access network.

B B  Campus Fabric – Shipping Now


CLI or API form of the new overlay
C Fabric solution for your enterprise
Campus access networks.

Campus CLI approach provides backwards


compatibility and customisation,
Fabric Box-by-Box. API approach provides
automation via NETCONF / YANG.

APIC-EM, ISE, NDP are all separate.


BRKCRS-2811 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 7
Cisco Digital Network Architecture
DNA Overview
Network-enabled Applications

Cloud Service Management


Policy | Orchestration

DNA Centre
Open APIs | Developers Environment

Automation
Insights &
Experiences

APIC-EM + ISE +Analytics


NDP
Principles Abstraction & Policy Control Network Data, Automation
from Core to Edge Contextual Insights & Assurance

Open & Programmable | Standards-Based


Security &
Virtualisation Compliance
SDA, IWAN & ENFV
Physical & Virtual Infrastructure | App Hosting

Cloud-enabled | Software-delivered

BRKCRS-2811 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 8
Roles & Terminology
What is Software Defined Access?
What is SD-Access?
Fabric Roles & Terminology
DNA  DNA Controller – Enterprise SDN Controller
APIC-EM
DNAC (e.g. DNA Centre) provides GUI management
Identity Controller
and abstraction via Apps that share context
Services
ISE  Identity Services – External ID System(s)
Analytics (e.g. ISE) are leveraged for dynamic Endpoint
to Group mapping and Policy definition
Engine
 Analytics Engine – External Data Collector(s)
(e.g. NDP) are leveraged to analyse Endpoint
Fabric Border Fabric Wireless to App flows and monitor fabric status
Nodes Controller
B B  Control-Plane Nodes – Map System that
manages Endpoint to Device relationships
Control-Plane
Intermediate  Fabric Border Nodes – A Fabric device (e.g.
C Nodes
Nodes (Underlay) Core) that connects External L3 network(s)
to the SDA Fabric
Campus  Fabric Edge Nodes – A Fabric device (e.g.
Fabric Edge Access or Distribution) that connects Wired
Nodes
Fabric Endpoints to the SDA Fabric
 Fabric Wireless Controller – A Fabric device
(WLC) that connects Wireless Endpoints to
the SDA Fabric

BRKCRS-2811 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 10
Agenda

1
SDA Fabric Border Functionality
• Different use cases for the SDA Border
• Border Automation models

2
SDA Fabric Border Deep Dive
• Border ( Internal)
• Default Border ( External )
• Border + Default Border ( Internal + External)
• Border Layer 3 Hand off with VRF-Lite

3 SDA Fabric Border Design


• Collocated Border + C-Plane
• Distributed Border + C-Plane
• One Box vs. Two Box

4 SDA Fabric Border External Connectivity


• Services, WAN, Firewall, DC
• Internet , Cloud

© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public
Current State Topology of the Campus Network
VXLAN Fabric
ACI Fabric Role Platform
Access Node • Cat3K/9300
• Cat4K/9400
Internet Edge

Guest Distribution • Cat3K/9300


WLCs Node • Cat4K/9500
• Cat6K/9500
Internet
Core Node • Cat6K/9500
• NK7K
Centralised WAN
WLC Agg
IWAN HR
• ASR1K-HX
OTT
IWAN HR
IWAN Centralised • 8540
Shared Services WLC • 5520
WAN
IWAN MC
• x800 APs
Edge
IWAN HR/MC • ASR1K
CarrierE • ISR4K
WAN
Campus Internet Edge • ASR9K
Core
• ASR1K
• ISR4K
WAN
Site Data Centre • N9K – NX-OS
Small Small • N7K - NX-OS
Hybrid Internet
Distribution IWAN Site IWAN Site • N9K - ACI
Nodes
Large
Hybrid Security • ISE 2.1
IWAN Site • ASA 55xx
Access • Windows AD
Nodes

BRKCRS-2811 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 12
End state Topology of the SD-Access Fabric
VXLAN Fabric
ACI Fabric
Internet Internet Edge
Edge/
Border Guest
WLCs

Internet

IWAN HR

VXLAN eBGP-
IPV4/ IWAN HR
Centralised EVPN MPLS
MPLS
WLCs

WAN IWAN MC

edge
Shared Services VRF-Lite /Border

DC and WAN
Services Edge

Border
Intermediate
Nodes CarrierE

IWAN Sites

Intermediate
Nodes
WAN
Sites

Edge
Nodes
FEW
WLC

SDA Fabric Domain BRKCRS-2811 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 13
SDA Fabric Border Functionality
What do Customers Need to Know
About the Fabric Border?
SD-Access Border
Border Nodes – A Closer Look
Border Node is an entry & exit point for all data traffic going in & out of the Fabric

There are 2 Types of Border Node! C


Known Unknown
Networks Networks

B B
• Fabric Border
• Used for “Known” Routes in your company

• Default Border
• Used for “Unknown” Routes outside your company
Fabric Edge Nodes

BRKCRS-2811 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 15
SD-Access Border
Border Nodes – Border and Default Border

Known Unknown
Networks Networks

B B
Border Default Border
• Connects the Campus Fabric to • Connects the Campus Fabric to
Known networks. (Use case 2.1 Un-Known networks (Use case 1)
and 2.2) • not part of the company network
• part of your company network
• Un-known networks are generally the
• Known networks are generally WAN, Internet and/or Public Cloud.
DC, Shared Services, etc.
• Responsible for advertising prefixes only
• Responsible for advertising prefixes to from (export) the local fabric to external
(import) and from (export) the local domain.
fabric and external domain.

BRKCRS-2811 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 16
Why Border Vs Default Border
SD-Access Fabric
Why Border vs Default Border?

Edge Node
IP Network B

Default Border External Network

Wan Edge WAN/Branch

DC Edge Datacentre
BRKCRS-2811 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 18
SD-Access Fabric
Why Border vs Default Border?

Edge Node
IP Network B

Default Border External Network

Border WAN/Branch

Border Datacentre
BRKCRS-2811 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 19
SD-Access Border Deployment Options
Use Case 1 : SDA fabric Connecting to Unknown Networks

B B

Un-Known Networks

Fabric Edge Nodes

BRKCRS-2811 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 20
SD-Access Border Deployment Options
Use Case 1 : SDA fabric Connecting to Unknown Networks – A Closer Look

Public Cloud
C

B B

Internet
Fabric Edge Nodes

BRKCRS-2811 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 21
SD-Access Border
Use Case 1 : SDA fabric Connecting to Unknown Networks
• Default Border is a “Gateway of Last Resort” for
unknown destinations
• Connects to any “unknown” IP prefixes (e.g. Internet,
Public Cloud, 3rd Party, etc.) C
Known Unknown
Networks Networks
• Exports all internal IP Pools outside (as aggregate) into B B
traditional IP routing protocol(s).
• Default Border is a “default” domain exit point, if no
other (specific) entry present in Map System.
• Outside hand-off requires mapping the prefix context
(VRF & SGT) from one domain to another.

Fabric Edge Nodes

BRKCRS-2811 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 22
SD-Access Border Deployment Options
Use Case 1 : SDA fabric Connecting to Unknown Networks – Automation

BRKCRS-2811 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 23
SD-Access Border Deployment Options
Use Case 1 : SDA fabric Connecting to Unknown Networks – Automation

BRKCRS-2811 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 24
SD-Access Border Deployment Options
Use Case 2.1 : SDA fabric Connecting to known Networks

B B

Known Networks

Fabric Edge Nodes

BRKCRS-2811 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 25
SD-Access Deployment Options
Use Case 2.1 : SDA fabric Connecting to known Networks – A Closer Look

DC
C

B B

Branch

Fabric Edge Nodes

BRKCRS-2811 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 26
SD-Access Border Deployment Options
Use Case 2.2 : SDA fabric as a Transit Network

B B

External Domain 1 External Domain 2

Fabric Edge Nodes

BRKCRS-2811 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 27
SD-Access Border
Use Case 2 : SDA fabric Connecting to known Networks

• Border advertises Endpoints to outside, and known


Subnets to inside

• Connects to any “known” IP subnets attached to the C


outside network (e.g. DC, WLC, FW, etc.) Known
Networks
Unknown
Networks

B B
• Exports all internal IP Pools to outside (as aggregate),
using a traditional IP routing protocol(s).

• Imports and registers (known) IP subnets from outside,


into the Fabric Control Plane System

• Outside hand-off requires mapping the prefix context


(VRF & SGT) from one domain to another.
Fabric Edge Nodes

BRKCRS-2811 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 28
SD-Access Border Deployment Options
Use Case 2 : SDA fabric Connecting to known Networks – Automation

BRKCRS-2811 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 29
SD-Access Border Deployment Options
Use Case 2 : SDA fabric Connecting to known Networks – Automation

BRKCRS-2811 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 30
SD-Access Border Deployment Options
Use Case 3 : SDA fabric Connecting to known and Un-known Networks

Data Centre
C
WAN
B B

Internet
Fabric Edge Nodes

BRKCRS-2811 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 31
SD-Access Border Deployment Options
Use Case 3 : SDA fabric Connecting to Everything– Automation

BRKCRS-2811 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 32
SDA Fabric Border Deep Dive
A look Under the Hood !!
Fabric Border (Internal)
SD-Access Border Automation
SD-Access simplifies Border provisioning with 2 steps

BRKCRS-2811 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 35
SD-Access Border
Border - Forwarding from Fabric Domain to External Domain
3 EID-prefix: 192.1.1.0/24
Path Preference
Mapping Locator-set: Controlled
Entry 2.1.1.1, priority: 1, weight: 100 (D1) by Destination Site
192.1.1.0/24
Branch

Border 5.1.1.1

Control Plane
5 2.1.1.1
nodes

10.1.1.1  192.1.1.1 5.2.2.2

SDA Fabric
4
1.1.1.1  2.1.1.1
10.1.1.1  192.1.1.1
1.1.1.1 Edge 1.1.2.1 1.1.3.1 Edge 1.1.4.1

2
10.1.1.1  192.1.1.1

1 S
DNS Entry: Campus
Campus
10.1.1.0/24 10.3.0.0/24 Bldg 2
D.abc.com A 192.1.1.1 Bldg 1

BRKCRS-2811 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 36
SD-Access Border
Border - Forwarding from External Domain to Fabric Domain
1
Routing Entry: 3 EID-prefix: 10.1.1.1/32
Send traffic to exit point of Path Preference
Mapping Locator-set: Controlled
domain(Internal Border)
Entry 1.1.1.1, priority: 1, weight: 100 (D1) by Destination Site
192.1.1.0/24
Branch

Border 5.1.1.1

Control Plane
2 2.1.1.1
nodes

192.1.1.1  10.1.1.1 5.2.2.2

4 SDA Fabric
2.1.1.1  1.1.1.1
192.1.1.1  10.1.1.1
1.1.1.1 Edge 1.1.2.1 1.1.3.1 Edge 1.1.4.1

5
192.1.1.1  10.1.1.1
D
Campus Campus
10.1.1.0/24 10.3.0.0/24 Bldg 2
Bldg 1

BRKCRS-2811 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 37
fabric outside
SD-Access Border Config
Advertise Fabric Prefixes to the external network
10.1.1.1/24 1.1.1.1/32 2.1.1.1/32 192.1.1.1/24

IP Network
10.1.1.0/24 BGP 192.1.1.0/24

Host Pool 10 Edge Node 1 External Domain(


Border Node
DC,WAN)

• The Border node advertises the EID router lisp


locator-table default
prefix into external protocol of locator-set border
choice(eBGP). IPv4-interface Loopback0 priority 10 weight 10
!
• The advertisement is summarised so router bgp 65004
!
that /32 host routes are not exposed to address-family ipv4 vrf USER
the external domain. redistribue LISP metric 10
aggregate-address 10.1.1.0 255.255.255.0 summary-only
• Repeat for other IP Subnets and exit-address-family
VRF’s in Fabric

BRKCRS-2811 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 38
fabric outside
SD-Access Border Config
Register External (known) prefixes in the Fabric
10.1.1.1/24 1.1.1.1/32 2.1.1.1/32 192.1.1.1/24

IP Network
10.1.1.0/24 BGP 192.1.1.0/24

Host Pool 10 Edge Node 1 External Domain(


Border Node
DC,WAN)

• The Border also imports the external router lisp


locator-table default
prefixes into the Campus Fabric LISP locator-set border
domain. IPv4-interface Loopback0 priority 10 weight 10
!
• Repeat for other IP Subnets and VRF’s eid-table vrf USER instance-id 10
ipv4 route-import database bgp 65004 locator-set border
in Fabric exit
!

BRKCRS-2811 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 39
punt

fabric outside
SD-Access Border Config
Activate LISP forwarding for Internal prefixes
10.1.1.1/24 1.1.1.1/32 2.1.1.1/32 192.1.1.1/24

IP Network
10.1.1.0/24 BGP 192.1.1.0/24

Host Pool 10 Edge Node 1 External Domain(


Border Node
DC,WAN)

router lisp
• Add a Map Cache + Map-Request for locator-table default
Dynamic EIDs locator-set border
IPv4-interface Loopback0 priority 10 weight 10
• trigger a lookup for traffic coming from outside !
eid-table vrf USER instance-id 10
• Repeat for other IP Subnets and ipv4 map-cache site-registration
VRF’s in Fabric exit

BRKCRS-2811 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 40
Default Border (External)
SD-Access Default Border Automation
SD-Access simplifies Default Border provisioning with 2 steps

BRKCRS-2811 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 42
SD-Access Border
Default Border - Forwarding to External Domain
2 EID-Prefix: Not found , map-cache miss
Mapping Locator-Set: ( use-petr)
Entry 3.1.1.1, priority: 1, weight: 100 (D1)
INTERNET

193.3.0.0/24 D
4 Default
Border
10.2.0.1  193.3.0.1
3.1.1.1
5.1.1.1

Control Plane
nodes
3 5.2.2.2
SDA Fabric
1.1.2.1  3.1.1.1
10.2.0.1  193.3.0.1

1.1.1.1 Edge 1.1.2.1 1.1.3.1 Edge 1.1.4.1

1
10.2.0.1  193.3.0.1

Campus S Campus
Bldg 1 10.2.0.0/24 10.3.0.0/24 Bldg 2

BRKCRS-2811 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 43
fabric outside
SD-Access Default Border Config
Advertise Fabric Prefixes to the external network
10.1.1.1/24 1.1.1.1/32 3.1.1.1/32 192.1.1.1/24
172.1.1.1/24

IP Network
10.1.1.0/24 150.1.1.0/24
192.1.1.0/24

Host Pool 10 Edge Node 1 Default Internet


Border Node

• The EID prefixes are exported from router lisp


locator-table default
Control plane node to the Default Border locator-set border
node with AD of “250” IPv4-interface Loopback0 priority 10 weight 10
!
• The Border node only advertises the EID router bgp 65004
!
prefix into external protocol of address-family ipv4 vrf USER
redistribue LISP metric 10
choice(BGP) aggregate-address 10.1.1.0 255.255.255.0 summary-only
exit-address-family
• Repeat for other IP Subnets and
VRF’s in Fabric

BRKCRS-2811 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 44
punt

fabric outside
SD-Access Default Border Config
Activate LISP forwarding for Internal prefixes
10.1.1.1/24 1.1.1.1/32 3.1.1.1/32 192.1.1.1/24
172.1.1.1/24

IP Network
10.1.1.0/24 150.1.1.0/24
192.1.1.0/24

Host Pool 10 Edge Node 1 Default Internet


Border Node
• Add a Map Cache + Map-Request for router lisp
locator-table default
Dynamic EIDs locator-set border
• trigger a lookup for traffic coming from outside IPv4-interface Loopback0 priority 10 weight 10
!
• Repeat for for other IP Subnets and eid-table vrf USER instance-id 10
ipv4 map-cache site-registration
VRF’s in Fabric exit
!

BRKCRS-2811 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 45
SD-Access Default Border Config
Configure Edge devices to use the default border
10.1.1.1/24 1.1.1.1/32 3.1.1.1/32 172.1.1.1/24

IP Network
10.1.1.0/24 150.1.1.0/24
192.1.1.0/24

Host Pool 10 Edge Node 1 Default Internet


Border Node

router lisp
• Fabric edge node has a default route to locator-table default
the External Border. locator-set edge
IPv4-interface Loopback0 priority 10 weight 10
!
ipv4 use-petr 3.1.1.1

BRKCRS-2811 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 46
Border + Default Border (Internal +
External)
SD-Access Border + Default Border Automation
SD-Access simplifies Default Border provisioning with 2 steps

BRKCRS-2811 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 48
fabric outside

SD-Access Border + Default Border Config


Register external (known) prefixes in the fabric / Filter out defaults/unknowns
10.1.1.1/24 1.1.1.1/32 2.1.1.1/32 192.1.1.1/24

IP Network
10.1.1.0/24 BGP 192.1.1.0/24

Host Pool 10 Edge Node 1 Border + External Domain (Internet +


Default DC,WAN)
Border Node
• The Border imports the external router lisp
prefixes into the Campus Fabric except locator-table default
locator-set border
the default route LISP domain. IPv4-interface Loopback0 priority 10 weight 10
!
• Repeat for other IP Subnets and VRF’s eid-table vrf USER instance-id 10
in Fabric ipv4 route-import database bgp 65004 route-map deny_0.0.0.0/0
locator-set border
exit
!
route-map deny_0.0.0.0/0 deny 10
match ip address prefix-list deny_0.0.0.0/0
!
route-map deny_0.0.0.0/0 permit 20
!
ip prefix-list deny_0.0.0.0/0 permit 0.0.0.0/0

BRKCRS-2811 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 49
SD-Access Border + Default Border Config
Configure Edge devices to use the default border
10.1.1.1/24 1.1.1.1/32 3.1.1.1/32 172.1.1.1/24

IP Network
10.1.1.0/24 150.1.1.0/24
192.1.1.0/24

Host Pool 10 Edge Node 1 Border +


External Domain (Internet +
Default
DC,WAN)
Border Node
router lisp
• Fabric edge node has a default route to locator-table default
the External Border. locator-set edge
IPv4-interface Loopback0 priority 10 weight 10
!
ipv4 use-petr 3.1.1.1

BRKCRS-2811 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 50
SDA Fabric Border Design
Considerations
Fabric Border Platform Support and
Recommendations
SD-Access – Border Node
Platform Support
Catalyst 3K Catalyst 9K Catalyst 6K ASR1K & ISR4K Nexus 7K

• Catalyst 3850 • Catalyst 9300 • Catalyst 6800 • ASR 1000-X/HX • Nexus 7700
• 1/10G SFP+ • Catalyst 9400 • Catalyst 6500 • ISR 4451/4431 • Sup2E
• 10/40G NM Cards • Catalyst 9500 • Sup2T/6T • 1/10G/40G • M3 Cards
• IOS-XE 16.6.1+ • 40G QSFP • 6880-X or 6840-X • IOS-XE 16.6.1+ • NXOS 7.3.2+
• 10/40G NM Cards • IOS 15.5.1SY+
• IOS-XE 16.6.1+

BRKCRS-2811 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 53
SD-Access – Border Node Scale
Platform Scale
Catalyst
Catalyst3850
3K Catalyst
Catalyst 9500
9500 Catalyst
Catalyst 6K
6K ASR1K
ASR1K & ISR4K
ISR4K Nexus
Nexus 7K
7K

 Virtual Networks: 64  Virtual Networks: 256  Virtual Networks: 512  Virtual Networks: 4K  Virtual Networks: 500
 SGT’s in Fabric: 4K  SGT’s in Fabric: 32K  SGT’s in Fabric: 30K  SGT’s in Fabric: 64K  SGT’s in Fabric: 64K
 SGT ACL’s: 1350  SGT ACL’s: 32K  SGT ACL’s: 30K  SGT ACL’s: 64K  SGT ACL’s: 64K
 Security ACL’s: 3K  Security ACL’s: 18K  Security ACL’s: 32K  Security ACL’s: 4K  Security ACL’s: 128K
 IPv4 TCAM: 16K/8K  IPv4 TCAM: 96K/48K  IPv4 TCAM: 256K  IPv4 TCAM: 1M  IPv4 TCAM: 1M

• Numbers listed are HW scale limits , SW numbers might be different


BRKCRS-2811 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 54
SD-Access – Control-Plane
Platform Support
Catalyst 3K Catalyst 9K ASR1K/ISR4K and
Catalyst 6K
CSR1Kv

• Catalyst 3850 • Catalyst 9300 • Catalyst 6800/6500 • ASR 1000-X/HX


• 1/10G SFP+ • Catalyst 9500 • Sup2T/6T • ISR 4430/4450
• 10/40G NM Cards • 40G QSFP • 6880-X or 6840-X • 1/10G/40G
• IOS-XE 16.6.1+ • 1/10G NM Cards • IOS 15.5.1SY+ • IOS-XE 16.6.1+
• IOS-XE 16.6.1+

BRKCRS-2811 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 55
SD-Access – Control-Plane Node Scale
Platform Scale
ASR1K/ISR4K and
Catalyst 3850 Catalyst 9500 Catalyst 6K
CSR1Kv

• 4K Host entries • 96K Host entries • 25K Host entries • 200K Host entries

BRKCRS-2811 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 56
Fabric Border Design Options
SD-Access Fabric
Border Nodes – Collocated vs. Distributed

B C B C

Collocated Design Distributed Design


• Border and Control Plane node Border and Control Plane node
is on the same device are on different devices
• Simple Design, without any extra Additional configurations required to
configurations between Border and share EID mapping from Border to
Control Plane node Control Plane node
• Best when only a few (e.g. 2) Multiple Border nodes can all
Collocated Border + Control Plane connect to the same (single or set
nodes are used of) Control Plane nodes

BRKCRS-2811 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 58
Border Design Options
Use case 1: Border with Collocated Control Plane Node
10.1.1.1/24 1.1.1.1/32 2.1.1.1/32 192.1.1.1/24
B C
IP Network
10.1.1.0/24 BGP 192.1.1.0/24

Host Pool 10 Edge Node 1 Border +


External Domain
Control Plane
(DC,WAN)
Node

• The Border and Control Plane node is on the same device


• Border node must perform export (and/or import) of routes between domains
• Control Plane node maintains the database of every prefix/subnet in the Fabric Domain
• Simplified Design (no additional configuration)
• No additional routing protocols needed to synch Border & Control Plane
• Best when only a few Border nodes are used (e.g. 2 to 4 per Domain)

NOTE: Control Plane node scale is different on different platforms (select accordingly)

BRKCRS-2811 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 59
Border Design Options
Use case 2: Border with Distributed Control Plane
C Node
5.1.1.1/32
Control-Plane Node

10.1.1.1/24 1.1.1.1/32 2.1.1.1/32 192.1.1.1/24


B
BGP
IP Network
10.1.1.0/24 OSPF 192.1.1.0/24

Host Pool 10 Edge Node 1 Border Node External Domain

• The Border node and Control plane node are different devices
• Device 1 - Border node must perform export (and/or import) of routes between domains
• Device 2 - Control Plane node maintains the database of every prefix/subnet in the Fabric Domain
• Additional configurations are required
• Need additional protocol (iBGP) to share EID mapping information from Border to Control Plane node.
• Multiple Border nodes can connect to the same Control Plane nodes (single or set of)

NOTE: Control Plane node scale is different on different platforms (select accordingly)

BRKCRS-2811 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 60
Fabric Border with
Collocated Control-Plane Node
SD-Access Border Automation

SD-Access simplifies Co-located Border and Control Plane provisioning with 1 steps

BRKCRS-2811 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 62
B C
CP RIB
SD-Access Fabric Config
Export EID mappings from the LISP MS to the Border RIB
10.1.1.1/24 1.1.1.1/32 2.1.1.1/32 192.1.1.1/24
B C
IP Network
10.1.1.0/24 BGP 192.1.1.0/24

Host Pool 10 Edge Node 1 Border +


External Domain
Control Plane
(DC,WAN)
Node
router lisp
• Control Plane operates as an LISP Map- locator-table default
server & Map-resolver locator-set border
IPv4-interface Loopback0 priority 10 weight 10
• Fabric EID prefixes are exported !
from Control plane node (internally) to eid-table vrf USER instance-id 10
ipv4 route-export site-registrations
the Border node. ipv4 distance site-registrations 250
!
• Use Admin Distance of “250” to site Campus
prefer the existing RIB/FIB route. authentication-key cisco
eid-prefix instance-id 10 0.0.0.0/0 accept-more-specifics
eid-prefix instance-id 10 10.1.1.0/24 accept-more-specifics
!
ipv4 map-server
ipv4 map-resolver

= Automated via DNA Centre


BRKCRS-2811 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 63
punt
B C
fabric outside
SD-Access Fabric Config
Activate LISP forwarding for Internal prefixes
10.1.1.1/24 1.1.1.1/32 2.1.1.1/32 192.1.1.1/24
B C
IP Network
10.1.1.0/24 BGP 192.1.1.0/24

Host Pool 10 Edge Node 1 Border +


External Domain
Control Plane
(DC,WAN)
Node

• Add a Map Cache + Map-Request for router lisp


all registered Dynamic EIDs locator-table default
locator-set border
• used for traffic coming from outside IPv4-interface Loopback0 priority 10 weight 10
!
• triggers a map-server lookup to locate eid-table vrf USER instance-id 10
destinations in the fabric ipv4 map-cache site-registration
exit

BRKCRS-2811 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 64
B C
fabric outside
SD-Access Fabric Config
Advertise Fabric Prefixes to the external network
10.1.1.1/24 1.1.1.1/32 2.1.1.1/32 192.1.1.1/24
B C
IP Network
10.1.1.0/24 BGP 192.1.1.0/24

Host Pool 10 Edge Node 1 Border +


External Domain
Control Plane
(DC,WAN)
Node
• The Border node advertises the EID router lisp
locator-table default
prefix into external protocol of choice locator-set border
(e.g. eBGP). IPv4-interface Loopback0 priority 10 weight 10
!
• The advertisement is summarised so router bgp 65535
!
that /32 host routes are not exposed to address-family ipv4 vrf USER
the external domain. redistribue LISP metric 10
aggregate-address 10.1.1.0 255.255.255.0 summary-only
exit-address-family

BRKCRS-2811 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 65
B C
fabric outside
SD-Access Fabric Config
Register external known prefixes into the Fabric
10.1.1.1/24 1.1.1.1/32 2.1.1.1/32 192.1.1.1/24
B C
IP Network
10.1.1.0/24 BGP 192.1.1.0/24

Host Pool 10 Edge Node 1 Border +


External Domain
Control Plane
(DC,WAN)
Node
• Border also imports the external router lisp
locator-table default
prefixes into the Campus Fabric LISP locator-set border
domain. ( This is not done for a default IPv4-interface Loopback0 priority 10 weight 10
Border node) !
eid-table vrf USER instance-id 10
ipv4 route-import database bgp 65004 locator-set border
exit
!

BRKCRS-2811 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 66
Fabric Border with
Distributed Control-Plane Node
SD-Access Border Automation
SD-Access simplifies Distributed Border and Control Plane provisioning with 2 steps

BRKCRS-2811 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 68
SD-Access Border Automation
SD-Access simplifies Distributed Border and Control Plane provisioning with 2 steps

BRKCRS-2811 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 69
B C
CP RIB
Export EID mappings
Control Plane Node – EID to RIB C
5.1.1.1/32
Control-Plane Node

10.1.1.1/24 1.1.1.1/32 2.1.1.1/32 192.1.1.1/24


B
IP Network
10.1.1.0/24 BGP OSPF 192.1.1.0/24

Host Pool 10 Edge Node 1 Border Node External Domain

router lisp
• Control Plane operates as an IPv4 locator-table default
LISP Map-server & Map-resolver locator-set control_node
IPv4-interface Loopback0 priority 10 weight 10
• Fabric EID prefixes are exported !
eid-table vrf USER instance-id 10
from Control plane node to its own RIB ipv4 route-export site-registrations
(routing information base) with AD of ipv4 distance site-registrations 250
“250” !
site Campus
• Add the IP prefixes to be mapped authentication-key cisco
eid-prefix instance-id 10 0.0.0.0/0 accept-more-specifics
• accept more-specific updates (e.g. /32) eid-prefix instance-id 10 10.1.1.0/24 accept-more-specifics
!
ipv4 map-server
ipv4 map-resolver

= Automated via DNA Centre 70


BRKCRS-2811 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public
B C
CP border
Export EID mappings
Control Plane Node – iBGP to border C
5.1.1.1/32
Control-Plane Node

10.1.1.1/24 1.1.1.1/32 2.1.1.1/32 192.1.1.1/24


B
IP Network
10.1.1.0/24 BGP OSPF 192.1.1.0/24

Host Pool 10 Edge Node 1 Border Node External Domain

router bgp 65555


• The Control plane uses an iBGP !
connection to the Border node to neighbor 2.1.1.1 remote-as 65555
advertise the EID prefix into BGP !
address-family vpvnv4
• The advertisement is summarised so neighbor 2.1.1.1 activate
neighbor 2.1.1.1 send-community both
that /32 host routes are not exposed to !
the external domain. address-family ipv4 vrf USER
redistribue LISP metric 10
• Border node learns the EID prefixes aggregate-address 10.1.1.0 255.255.255.0 summary-only
exit-address-family
in the Local Fabric domain from the
Control Plane node.

BRKCRS-2811 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 71
fabric outside B C

Register external prefixes


Border Node – Ingest EIDs over iBGP
5.1.1.1/32 C Control-Plane Node
10.1.1.1/24 1.1.1.1/32 2.1.1.1/32 192.1.1.1/24
B
IP Network
10.1.1.0/24 BGP OSPF 192.1.1.0/24

Host Pool 10 Edge Node 1 Border Node External Domain

router lisp
• The Border receives the EID prefix locator-table default
information from the Control Plane node locator-set border
through the iBGP connection. IPv4-interface Loopback0 priority 10 weight 10
!
• Border also imports the external prefixes eid-table vrf USER instance-id 10
ipv4 route-import database ospf 123 locator-set border
into the LISP domain. !
router bgp 65555
• Does not apply to Default Border !
neighbor 5.1.1.1 remote-as 65555
!
address-family vpvnv4
neighbor 5.1.1.1 activate
neighbor 5.1.1.1 send-community both
!

BRKCRS-2811 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 72
fabric outside B C

Advertise EIDs externally (e.g. OSPF)


Border with Distributed Control Plane Node C
5.1.1.1/32
Control-Plane Node

10.1.1.1/24 1.1.1.1/32 2.1.1.1/32 192.1.1.1/24


B
IP Network
10.1.1.0/24 BGP OSPF 192.1.1.0/24

Host Pool 10 Edge Node 1 Border Node External Domain

• The Border advertises the EID prefix to router lisp


locator-table default
the external domain through OSPF by locator-set border
redistributing routes. IPv4-interface Loopback0 priority 10 weight 10
!
• Learnt earlier from the Control Plane eid-table vrf USER instance-id 10
ipv4 route-import database ospf 123 locator-set border
node via iBGP exit
!
router ospf 123 VRF USER
!
redistribute bgp metric 10 subnets

BRKCRS-2811 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 73
punt
B C
fabric outside
SD-Access Fabric Border Config
Activate LISP forwarding for Internal prefixes C
5.1.1.1/32
Control-Plane Node

10.1.1.1/24 1.1.1.1/32 2.1.1.1/32 192.1.1.1/24


B
IP Network
10.1.1.0/24 BGP OSPF 192.1.1.0/24

Host Pool 10 Edge Node 1 Border Node External Domain

• Add a Map Cache + Map-Request for router lisp


all registered Dynamic EIDs locator-table default
locator-set border
• used for traffic coming from outside IPv4-interface Loopback0 priority 10 weight 10
!
• triggers a map-server lookup to locate eid-table vrf USER instance-id 10
destinations in the fabric ipv4 map-cache site-registration
exit

BRKCRS-2811 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 74
Border Resiliency Options
Multiple Borders - Loop Prevention
10.1.1.1/24
B
10.1.1.0/24
Border Node eBGP
Host Pool 10 Edge Node 1
SDA FABRIC

192.1.1.0/24
10.1.1.1/24
B eBGP
Shared Services

10.1.1.0/24

Host Pool 10 Edge Node 2 Border Node

• eBGP is preferred to break any loops caused by the bidirectional advertisement (redistribution) of routes
from the fabric to external domain (and vice-versa), when using multiple Internal Borders for redundancy.
• eBGP uses AS-Path loop prevention.
• If you are using any other protocol than eBGP, some appropriate loop prevention mechanism needs to be used
(distribute-list, prefix-list, or route tags with route-map, etc).

BRKCRS-2811 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 75
Fabric Border
One Box -vs- Two Box
SD-Access Fabric
Border Nodes – One Box vs. Two Box
OUT OUT
B
B
One Box Design IN
Two Box Design
IN

• Internal and External domain routing is on


the same device Internal and External domain routing are on
different devices
• Simple design, without any extra
configurations between the Border and Requires two Devices with BGP in between
outside routers to exchange connectivity and reachability
information
• The Border device will advertise routes to
and from the Local Fabric domain to the This model is chosen if the Border does not
External Domain support the functionality (This can due to
hardware or software support on the device)
to run the external domain on the same
device (e.g. DMVPN, EVPN, etc.)

BRKCRS-2811 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 77
Border Design Options
One Box Border - Control Plane
CONTROL-PLANE

1
LISP External Domain(BGP/IGP)

C
B
B
External
Domain
B

BRKCRS-2811 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 78
Border Design Options
One Box Border - Data Plane

C
B
B
External
Domain
B

DATA-PLANE

2 External Domain(IP/MPLS/VXLAN)
VXLAN

BRKCRS-2811 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 79
Border Design Options
One Box Border - Policy semantics in the Data Plane

C
B
B
External
Domain
B

Policy Metadata

3
SGT in VXLAN External Domain(IP ACL/SGT)

BRKCRS-2811 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 80
Border Design Options
Two Box Border - Control Plane
CONTROL-PLANE

11
LISP BGP External Domain(BGP/IGP)

C
B
B External
Domain
B

BRKCRS-2811 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 81
Border Design Options
Two Box Border - Data Plane

C
B
B External
Domain
B

DATA-PLANE

12
VXLAN VRF-LITE External Domain(IP/MPLS/VXLAN)

BRKCRS-2811 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 82
Border Design Options
Two Box Border - Policy semantics in the Data Plane

C
B
B External
Domain
B

POLICY-METADATA

13
SGT in VXLAN SGT Tagging External Domain ( IP ACL/SGT)

BRKCRS-2811 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 83
Fabric Border Automation

Layer 3 Hand off


Fabric Border Automation
Layer 3 VRF-LITE Hand-Off
LISP BGP
CONTROL-PLANE

C
B
B
External
Domain
B
SDA Fabric

VXLAN VRF-LITE
DATA-PLANE

BRKCRS-2811 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 84
Fabric Border Automation
Layer 3 VRF-LITE Hand-Off

16.6.2
3 Select the Layer
3 hand off
CORE

SJC22

San_Jose 1 Select the


Border Node role
4 Select the Type of
Hand Off
7 Select Remote
AS

5 Select Subnet for


Hand off

2 Select the 8 Select VRF


Connection type advertisement *

6 Select the External


Interface(s)

BRKCRS-2811 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 85
Border Resiliency (HA)
Resiliency at the Border
Track or propagate events across domains

External
Border
Router
B IP Network

Border External
SDA Fabric Router External Domain

BRKCRS-2811 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 88
Resiliency at the Border
Use Case 1 : Track failures in the External Domain
CONTROL-PLANE LISP IGP/MP-BGP/BGP-EVPN

External
Border
Router
B IP Network

Border External
SDA Fabric Router External Domain

VXLAN/+SGT IP/MPLS/VXLAN

DATA-PLANE

BRKCRS-2811 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 89
Failures & Changes in the External Domain
External advertisements to reflect state of the External Domain

Border
B IP Network

Border
SDA Fabric External Domain

Border Routing Tables updated Host reachability from


to remove the faulty route(s) router is lost or degraded
Host advertisements from
this router are withdrawn

BRKCRS-2811 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 90
Resiliency at the Border
Use Case 1 : Track failures in the External Domain

 No additional configuration is needed on the fabric border to achieve resiliency.

 Traffic is re-routed away from the failure point based on routing


protocols configured on the fabric border.

 Convergence depends on the routing protocols convergence times.

BRKCRS-2811 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 91
Resiliency at the Border
Use Case 2.1 : Track failures in the Fabric Domain @ Border and CP Co-located
CONTROL-PLANE LISP IGP/MP-BGP/BGP-EVPN

External
Border
Router
B IP Network

Border External
SDA Fabric Router External Domain

VXLAN/+SGT IP/MPLS/VXLAN

DATA-PLANE

BRKCRS-2811 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 92
Failures & Changes in the SD-Access Fabric
Internal redistribution of Fabric state into External Domain @ Border and CP Co-located
I. Border and Control plane Node Co-located

Border
B
IP Network

Border
SDA Fabric External Domain

Registration State Border connectivity to Campus Prefix Routing Tables


Changes Communicated Fabric network is degraded – advertisements from updated to route
to Border this border withdrawn around failure
• How can this be tracked ?

BRKCRS-2811 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 93
Failures & Changes in the SD-Access Fabric
Internal redistribution of Fabric state into External Domain @ Border and CP Co-located

 Since Border and Control Plane node are Co-located, when a Failure happens
the state of the network needs to be tracked and informed to the control plane
node so that the fabric border can withdraw its route advertisements.

 To Track the state of the Network we can use either an EEM script or Object
tracking.

 Since above requires configuration's on the border nodes an workaround to


alleviate this issue is explained in next slide.

BRKCRS-2811 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 94
Resiliency at the Border
Use Case 2.1 : Track failures in the Fabric Domain @ Border and CP Co-located

As a workaround the border node’s can be Connected via


a Layer 3 link.

B B This Layer 3 link/’s will have lesser cost to reach the


fabric edge nodes than the underlay , meaning when
underlay is available this direct connect link is not used.

If one of the border’s connectivity to the underlay is


degraded then the traffic from external domain will come
to that border and using the Layer 3 link will flow to the
other border node and then on to the fabric edge nodes.

Convergence times depends on routing protocol between


the border nodes

BRKCRS-2811 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 95
Resiliency at the Border
Use Case 2.2 : Track failures in the Fabric Domain @ Border and CP Distributed
CONTROL-PLANE LISP IGP/MP-BGP/BGP-EVPN

C Border External
Router
B IP Network
BFD
Adjacency
Border External
SDA Fabric Router External Domain

VXLAN/+SGT IP/MPLS/VXLAN

DATA-PLANE

BRKCRS-2811 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 96
Failures & Changes in the SD-Access Fabric
Internal redistribution of Fabric state into External Domain @ Border and CP Distributed
B
C
Border
BFD B
Adjacency IP Network

Border
SDA Fabric External Domain

• SDA fabric domain prefixes are advertised via BGP from Control Plane node to Border node
• BGP adjacencies between Control Plane and Border node are monitored with BFD
• Upon BFD adjacency fail, prefixes associated with the Border are withdrawn immediately
• Fast Convergence (150-200ms)

BRKCRS-2811 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 97
SDA Fabric Border External
Connectivity
How do Things Connect to the Fabric
Border?
Shared Services
with Border
Border Deployment Options
Shared Services (DHCP, AAA, etc) with Border

• Hosts in the fabric domain (in their respective Virtual Networks)


will need to have access to common “Shared Services”:
 Identity Services (e.g. AAA/RADIUS)
 Domain Name Services (DNS)
 Dynamic Host Configuration (DHCP)
 IP Address Management (IPAM)
 Monitoring tools (e.g. SNMP)
 Data Collectors (e.g. Netflow, Syslog)
 Other infrastructure elements
• These shared services will generally reside outside of the fabric domain.

BRKCRS-2811 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 100
Border Deployment Options
Shared Services (DHCP, AAA, etc) with Border

• RLOC Underlay connectivity in Global Routing Table


• Access Points and Extended Nodes will be in their Fabric Scope
own VN – INFRA_ VN which is in the Global Routing USER #2
Table
USER #1 Border
• Other VNs can be used for segmentation for users,
devices, roles, and others USER2
INFRA_VN USER1
• Scalable Group Tags (SGTs) can be used for further Default
access control within a VN RLOC Underlay GRT

• The “USER” VN is being shown in this slide deck as


an example.
• Similar steps can be followed for other VNs shown

BRKCRS-2811 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 101
Border Deployment Options
Shared Services (DHCP, AAA, etc) with Border in Global Routing Table

B B APIC
EM

APIC-EM DHCP/ Identity Service


DNS
GRT
Shared Services

BRKCRS-2811 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 102
SD-Access Fabric Config
Shared Services (DHCP, AAA, etc) with Border in Global Routing Table
5.1.1.1/32 C
Control-Plane Node

10.1.1.1/24 1.1.1.1/32 2.1.1.1/32 192.1.1.1/24


B
IP Network
10.1.1.0/24 EIGRP 172.10.10.0/24

Host Pool 10 Edge Node 1 Border Node Shared Services

router eigrp 65535


• The Shared Services are in the !
Global routing table address-family ipv4 vrf USER
redistribute lisp metric 10000 1 255 1 9100
• Will form a routing adjacency using network 192.1.1.1 0.0.0.0
the Global routing table. distribute-list in USER
autonomous-system 65535
• On Campus Fabric side we will form exit-address-family
address-family ipv4 vrf default
a routing adjacency using the VRF network 172.1.1.1 0.0.0.0
table of the EID space. distribute-list in default
autonomous-system 65535
• NOTE: If the outside protocol is not exit-address-family
BGP (e.g. EIGRP), then you need to !
use a distribute-list to filter routes.

BRKCRS-2811 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 103
SD-Access Fabric Config
Shared Services (DHCP, AAA, etc) with Border in Global Routing Table
5.1.1.1/32 C Control-Plane Node

10.1.1.1/24 1.1.1.1/32 2.1.1.1/32 192.1.1.1/24

B
IP Network
10.1.1.0/24 EIGRP 172.10.10.0/24

Host Pool 10 Edge Node 1 Border Node Shared Services

router eigrp 65535


• The Shared Services are in the !
Global routing table network 192.2.1.1 0.0.0.0
exit-address-family
• Will form a routing adjacency using the !
Global routing table.
• On Campus Fabric side we will form a
routing adjacency using the VRF table of
the EID space.
• Outside routing protocol is configured
and operates normally.

BRKCRS-2811 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 104
Border Resiliency Options
* Recap
Multiple Borders - Loop Prevention
10.1.1.1/24 1.1.1.1/32 2.1.1.1/32 192.1.1.1/24

B
10.1.1.0/24
eBGP
Host Pool 10 Edge Node 1 Border Node

IP Network
192.1.1.5/24 192.1.1.0/24
10.1.1.1/24 1.1.2.1/32 3.1.1.1/32
Shared Services
B eBGP

10.1.1.0/24

Host Pool 10 Edge Node 2 Border Node

• eBGP is preferred to break any loops caused by the bidirectional advertisement (redistribution) of routes
from the fabric to external domain (and vice-versa), when using multiple Internal Borders for redundancy.
• eBGP uses AS-Path loop prevention.
• If you are using any other protocol than eBGP, some appropriate loop prevention mechanism needs to be used
(distribute-list, prefix-list, or route tags with route-map, etc).

BRKCRS-2811 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 105
Border Deployment Options
Shared Services (DHCP, AAA, etc) with Border in dedicated VRF

B B APIC
EM

APIC-EM DHCP/ Identity Service


VRF DNS
Fusion
Router Shared Services

BRKCRS-2811 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 106
Border Deployment Options
Shared Services (DHCP, AAA, etc) with Border in dedicated VRF
5.1.1.1/32 C
Control-Plane Node

10.1.1.1/24 1.1.1.1/32 2.1.1.1/32 192.1.1.1/24


B
IP Network
10.1.1.0/24 BGP BGP 172.10.10.0/24

Host Pool 10 Edge Node 1 Border Node Fusion Router Shared Services

• The Shared Services are in a unique ip vrf User


rd 1:1
dedicated VRF of their own. route-target export 1:1
route-target import 1:1
• Will form a routing adjacency in route-target import 3:3
!
each Address Family. ip vrf User
rd 2:2
• Use route-target import / export route-target export 2:2
route-target import 2:2
(leaking) to ”share” routes route-target import 3:3

• An external Fusion router is used to ip vrf Services


exchange routes from the VRF’s in rd 3:3
route-target export 3:3
Campus fabric to the Services VRF. route-target import 3:3
route-target export 1:1
route-target export 2:2

BRKCRS-2811 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 107
WAN Connectivity
with Border
Border Deployment Options
WAN Connectivity with Border- IWAN2.x and MPLS

B B
IWAN 2.x/MPLS

BRKCRS-2811 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 119
Border Design Options
IWAN2.x Connectivity with Border - Control Plane
CONTROL-PLANE

11
LISP BGP DMVPN

C
B
B iWAN 2.x

12

BRKCRS-2811 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 120
* Recap
Border Design Options
IWAN2.x Connectivity with Border - Data Plane
DATA-PLANE

11
VXLAN VRF-LITE DMVPN

C
B
B iWAN 2.x

12

BRKCRS-2811 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 121
Border Design Options
IWAN2.x Connectivity with Border - Policy Metadata
POLICY-METADATA

11
SGT in VXLAN SGT Tagging SGT in DMVPN

C
B
B iWAN 2.x

12

BRKCRS-2811 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 122
Border Deployment Options
MPLS WAN Connectivity with Border CONTROL-PLANE

LISP MP-BGP IGP/BGP

B B
MPLS Domain

BRANCH

SXP with ISE

VXLAN+SGT ISE with SXP bindings for SGT exchange IP/MPLS + SGT

DATA+POLICY PLANE
BRKCRS-2811 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 123
Border Deployment Options
Viptela SD-WAN hand off
CONTROL-PLANE

LISP VRF-LITE
Viptela Control Plane VRF-LITE
LISP

WAN App

C Border vEdge
vEdge Border C
B B B
B
SD-WAN

DNA Centre

SXP with ISE


SDA Fabric Site 1 SDA Fabric Site 2
DOT 1Q DOT 1Q
VXLAN+SGT Viptela Data Plane VXLAN + SGT

DATA+POLICY PLANE
BRKCRS-2811 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 123
Multiple Fabric Domains Connectivity
with Border
Border Deployment Options
Multiple Fabric Domains

B B B B

VRF- LITE

BRKCRS-2811 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 126
Border Deployment Options
Multiple Fabric Domains
CONTROL-PLANE

11
LISP BGP/IGP LISP

B B B B

BRKCRS-2811 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 127
Border Deployment Options
Multiple Fabric Domains
DATA-PLANE

12
VXLAN VRF-LITE/IP/MPLS VXLAN

B B B B

BRKCRS-2811 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 128
Border Deployment Options
Multiple Fabric Domains
POLICY-METADATA

12
SGT in VXLAN SGT Tagging/SXP SGT in VXLAN

B B B B

SXP Connection between the Border’s


for SGT information exchange

* Check Platform support if using the SXP Model

BRKCRS-2811 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 129
Border Deployment Options
Multiple Fabric Domains
CONTROL-PLANE

1
LISP DMVPN/GRE LISP

c c

B B B B
IP Network

DATA PLANE w/POLICY


12
VXLAN+SGT IP+SGT inline tagging VXLAN+SGT

BRKCRS-2811 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 130
Border Deployment Options
Multiple Fabric Domains
CONTROL-PLANE

11 LISP LISP

c c
B N7K

B B

1 Fabric Domain 1 Fabric Domain 2


12 VXLAN+SGT VXLAN+SGT

DATA PLANE w/POLICY

BRKCRS-2811 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 131
C9K
Border Deployment Options IOS-XE 16.8

SD-Access Multi-Site

East Site

West Site
Transit Site

Control Plane

South Site Border Router

Edge

BRKCRS-2811 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 131
Control Plane

Border Deployment Options


Border Router

Edge
SD-Access Multi-Site

West site Prefixes Only East + West East site Prefixes Only

Register west Register east


prefixes prefixes

West Site
Transit Site East Site
BR-W BR-E

BRKCRS-2811 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 132
Service Chaining
with Border
Border Deployment Options
Service Chaining with Border

Non-Cisco Firewall: Cisco Firewall :


• Firewall is connected externally to the • Firewall is connected externally to the Campus Fabric.
Campus Fabric.
• The prefixes from the local Campus Fabric domain will
• The prefixes from the local Campus be advertised to the firewall with a routing protocol of
Fabric domain will be advertised to the choice.
firewall with a routing protocol of choice.
• SXP connection between ISE and Firewall used for
• Firewall policy is based Interface or derivation of SGTs on the Firewall.
Subnet IP/mask and IP ACL’s.
• Firewall policy is based on SGT’s and SG ACL’s
(Group based Policy).

• Firewall also has Interface or Subnet IP based policy,


for brownfield integration
BRKCRS-2811 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 135
Border Deployment Options
Service Chaining with Border - Firewall

B
B
B
Firewall

BRKCRS-2811 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 136
Border Deployment Options
Service Chaining with Border - Firewall
CONTROL-PLANE

1
LISP BGP/IGP

B
B
B
Firewall

BRKCRS-2811 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 137
Border Deployment Options
Service Chaining with Border – Data-Plane (Routed firewall)
DATA-PLANE
2
VXLAN VRF-LITE

B
B
B
Firewall

BRKCRS-2811 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 138
Border Deployment Options
Service Chaining with Border – Policy Metadata
POLICY-METADATA
3
SGT in VXLAN SGT in-line Tagging

B
B
B
Firewall

BRKCRS-2811 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 139
Border Deployment Options
Service Chaining with Internal Border – Cisco Firewall, Checkpoint
ISE
POLICY-METADATA
3
SGT in VXLAN SGT in-line Tagging
Group Tags

SXP/PXGRID
B
B
B
Firewall

Firewall gets Group


Based Tags from ISE

BRKCRS-2811 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 140
Data Centre Connectivity
with Border
Border Deployment Options
Data Centre Connectivity With Border – ACI Fabric
CONTROL-PLANE

1 LISP BGP/IGP

B
ACI Fabric
Border
Map Server B

Border
SDA Fabric Border Leaf’s

DATA-PLANE

2
VXLAN+SGT VRF-LITE

BRKCRS-2811 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 142
Recap - ACI Fabric Integrated VXLAN Overlay
Decoupled Identity, Location and Policy
ACI Spine Nodes

ACI Fabric
VTEP VXLAN IP Payload

ACI Leaf Nodes

 Forwarding within the Fabric is between VTEPs (ACI VXLAN tunnel endpoints) and leverages an
extended VXLAN header format referred to as the ACI VXLAN policy header

 Any workload any where, Consistent Latency, Mapping of tenant MAC or Ip address to location is
performed by VTEP using distributed mapping database

BRKCRS-2811 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 143
Recap: What is an L3Out?

 L3Out is a logical construct defined to


allow L3 connectivity between the ACI
Fabric and the external network
 One or more L3Outs can be defined for
L3Outs Container
each given tenant
 L3 interfaces are used on specific ACI
Specific L3Out devices (named Border Leaf nodes) to
interconnect to the external routed network
L3 Interface on  The external routed domain is modeled
Border Leaf Node with one (or more) External EPGs
Border Leaf (‘Networks’)
Node
A security policy (contract) is required to allow
External EPG communication between External and Internal
EPGs

BRKCRS-2811 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 144
SD-Access SGTs Provisioned in ACI
SD-Access Domain ACI
ISE
ISE dynamically provisions
SGTs and IP mappings
(SXP service) into APIC-
DC

EXT- EXT-
EPG1 EPG3

Security Groups External (Outside Fabric) EPGs

BRKCRS-2811 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 145
ACI EPGs Automatically Propagated into SD-Access
ACI

ISE

ISE dynamically learns


EPGs and VM Bindings
from ACI fabric – shared to
SXP

VM1

SD-Access Domain VM25

Security Group from APIC-DC Internal (Inside Fabric) EPGs

BRKCRS-2811 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 146
Enabling Group-based Policy in each Domain

DB DB

SG-FW
SG-ACL
Contract

Campus / Branch Data Centre


SD-Access Policy Domain APIC
APIC Policy Domain

Voice Employee Supplier BYOD


Web App DB
Voice Data
VLAN VLAN SD-Access ACI Fabric

BRKCRS-2811 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 147
Hardware and Software recommendations
Shipping
NOW!
ACI Fabric
ACI Software ISE APIC
Hardware

Nexus 9K* 12.1 2.1 2.1

* – Please check release notes for latest information


* – (9396PX/TX, 9372PX/TX, 93120TX, 93128TX, 9736PQ LC, 9336PQ, 93108-EX, 93180-EX

BRKCRS-2811 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 148
SD-Access SGT Info Used in ACI Policies

ISE
SD-Access Policy Domain ACI Policy Domain
ISE Retrieves:
Controller Layer

ISE Exchanges:

Controller Layer
EPG Name:
SGT PCI EPG
Name: Auditor
EPG Binding = 10.1.100.52
SGT Binding = 10.1.10.220

PCI EPG
EPG Name = Auditor 10.1.100.52
Groups= 10.1.10.220

Network Layer
Network Layer

17000 ACI Spine (N9K)


5 Plain
SRC:10.1.10.220 SRC:10.1.10.220
DST: 10.1.100.52Ethernet DST: 10.1.100.52
SRC:10.1.10.220 SD-Access
DST: 10.1.100.52
SGT: 5 (no CMD) EPG
ACI Leaf
ACI Border PCI
Auditor Leaf (N9K) (N9K) 10.1.100.52
10.1.10.220
SGT Groups available in ACI Policies

BRKCRS-2811 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 149
Internet Connectivity
with Border
Border Deployment Options
Internet Connectivity With Border
CONTROL-PLANE

1 LISP BGP

Border
Map Server B

Border
SDA Fabric Internet

DATA-PLANE

2 VXLAN+SGT IP

BRKCRS-2811 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 151
Cloud Connectivity
with Border
Border Deployment Options
Cloud Connectivity With Border Cloud Edge gets
CONTROL-PLANE ISE
3 Group Based Policy
from ISE
1 LISP LISP
Group Policy

SXP
B
IP/MPLS
Border
Network CLOUD
Map Server B
Cloud Edge

CSR1Kv
Border
SDA Fabric ✔

DATA-PLANE

2 * Roadmap
VXLAN+SGT VXLAN+SGT

BRKCRS-2811 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 153
Take Away
When To Get Started?
SD-Access Support
Fabric ready platforms for your digital ready network

Switching Routing Wireless Extended


NEW Catalyst 9400
NEW
ASR-1000-X AIR-CT5520
NEW Catalyst 9300

NEW
AIR-CT8540 CDB
ASR-1000-HX NEW

Catalyst 9500
AIR-CT3504
ISR 4430 NEW

3560-CX
BRKCRS-
2811 NEW
Wave 2 APs (1800,2800,3800)
Catalyst 4500E Catalyst 6800 Nexus 7700 ISR 4450

IE (2K/3K/4K/5K)
Catalyst 3650 and 3850 ISRv/CSRv Wave 1 APs* (1700,2700,3700)
* with Caveats
BRKCRS-2811 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 155
What to Do Next?

SD-Access DNA Cisco


Capable Centre Services

Refresh your Deploy the Engage with


Hardware & Software DNA Centre Cisco Services

Get SD-Access Capable Devices Get DNA Centre Appliances Cisco Services can help you
with DNA Advantage OS License with DNA Centre Software to Test - Migrate - Deploy

BRKCRS-2811 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 156
The First Step…
#NewEra
#CiscoDNA
#NetworkIntuitive

BRKCRS-2810 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 157
SD-Access - Cisco on Cisco
Live SD-Access Deployment @ Cisco Systems

750
Wired & Wireless

SJC23 users
2 7 24
Fabric Border Fabric Fabric
Control-Plane Edge Access
Nodes Nodes Points

3 Virtual
Networks
16 Scalable
Groups
2 Wireless
SSIDs
8 Address
Pools

Built and managed by the Cisco Engineering team, in conjunction with Cisco IT Services

BRKCRS-2811 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 158
What to Do Next?

Technical Advisor y

Managed Im plem entation

Optim ization Tr aining

SD-Access DNA Cisco


Capable Centre Services

Refresh your Deploy the Engage


Hardware and Software DNA Centre Cisco Services

Get SD-Access Capable Devices Get DNA Centre Appliances Cisco Services can help you
with DNA Advantage OS License with DNA Centre Software to Test / Migrate / Deploy

BRKCRS-2811 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 160
Cisco Spark
Questions?
Use Cisco Spark to communicate
with the speaker after the session

How
1. Find this session in the Cisco Live Mobile App
2. Click “Join the Discussion”
3. Install Spark or go directly to the space
4. Enter messages/questions in the space

cs.co/ciscolivebot#BRKCRS-2811

© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public
Continue Your Education
• World of Solutions
• Meet The Engineer
• Related Sessions
• Hands-On Labs
• Lunch & Learn
DNA Centre
Simple Workflows

DESIGN PROVISION POLICY ASSURANCE

BRKCRS-2811 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 163
Q&A
Complete Your Online
Session Evaluation
• Give us your feedback and
receive a Cisco Live 2018 Cap
by completing the overall event
evaluation and 5 session
evaluations.
• All evaluations can be completed
via the Cisco Live Mobile App.
Don’t forget: Cisco Live sessions will be
available for viewing on demand after the
event at www.CiscoLive.com/Global.

© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public
Thank you