Beruflich Dokumente
Kultur Dokumente
Techniques
19th
Philip Smith
<philip@apnic.net>
APRICOT 2013
Singapore
February 1st March 2013
Presentation Slides
p Will
be available on
http://thyme.apnic.net/ftp/seminars/
APRICOT2013-Multihoming.pdf
n And on the APRICOT2013 website
n
p Feel
Preliminaries
p Tutorial
n
p Aimed
n
at Service Providers
Multihome?
p Definition & Options
p How to Multihome
p Principles & Addressing
p Basic Multihoming
p Service Provider Multihoming
p Using Communities
Why Multihome?
Its all about redundancy,
diversity & reliability
Why Multihome?
p Redundancy
n
Why Multihome?
p Reliability
Why Multihome?
p Supplier
Diversity
Why Multihome?
p Not
Why Multihome?
p Summary:
In real life?
p For the network?
p For the Internet?
p
Multihome?
p Definition & Options
p How to Multihome
p Principles & Addressing
p Basic Multihoming
p Service Provider Multihoming
p Using Communities
Multihoming: Definitions
& Options
What does it mean, what do we
need, and how do we do it?
Multihoming Definition
p More
p Usually
n
Two ranges
n
n
Usage:
n
n
n
n
n
n
n
0-65535
65536-4294967295
0 and 65535
1-64495
64496-64511
64512-65534
23456
65536-65551
65552-4294967295
(reserved)
(public Internet)
(documentation RFC5398)
(private use only)
(represent 32-bit range in 16-bit world)
(documentation RFC5398)
(public Internet)
p
p
See www.iana.org/assignments/as-numbers
Private-AS Application
p
Applications
n
65001
193.0.32.0/24
C
1880
193.0.34.0/24
65002
193.0.33.0/24
65003
193.0.35.0/24
193.0.32.0/22 1880
16
Private-AS Removal
p Private
p As
p Cisco
IOS
Transit/Peering/Default
p Transit
p Peering
p Default
n
Configuring Policy
p Three
p Route-maps
Policy Tools
p Local
n
preference
p Metric
n
(MED)
p AS-PATH
n
prepend
p Communities
n
Originating Prefixes
p
The RIRs publish their minimum allocation sizes per /8 address block
n
n
n
n
n
n
AfriNIC:
www.afrinic.net/docs/policies/afpol-v4200407-000.htm
APNIC:
www.apnic.net/db/min-alloc.html
ARIN:
www.arin.net/reference/ip_blocks.html
LACNIC:
lacnic.net/en/registro/index.html
RIPE NCC:
www.ripe.net/ripe/docs/smallest-alloc-sizes.html
Note that AfriNIC only publishes its current minimum allocation size, not
the allocation size for its address blocks
www.iana.org/assignments/ipv4-address-space
Multihome?
p Definition & Options
p How to Multihome
p Principles & Addressing
p Basic Multihoming
p Service Provider Multihoming
p Using Communities
How to Multihome
Choosing between transit and
peer
Transits
p
Only one
p
Too many
p
p
p
no redundancy
more difficult to load balance
no economy of scale (costs more per Mbps)
hard to provide service quality
Common Mistakes
p
No diversity
n
n
Peers
p
Public peer
n
Common Mistakes
p Mistaking
p Ignoring/avoiding
competitors because
they are competition
n
Multihoming Scenarios
p Stub
network
p Multi-homed stub network
p Multi-homed network
p Multiple sessions to another AS
Stub Network
AS101
AS100
p
p
p
p
p
p
p
p
Multi-homed Network
Global Internet
AS200
AS300
AS100
Multiple Sessions to an AS
ebgp multihop
p
Use ebgp-multihop
n
n
Cisco IOS
router bgp 100
neighbor 1.1.1.1
neighbor 1.1.1.1
!
ip route 1.1.1.1
ip route 1.1.1.1
ip route 1.1.1.1
AS 200
1.1.1.1
remote-as 200
ebgp-multihop 2
AS 100
Multiple Sessions to an AS
ebgp multihop
p
R1
R3
AS 100
AS 200
R2
Desired Path
Used Path
p Many
Multiple Sessions to an AS
bgp multi path
p
p
p
AS 200
AS 100
Multiple Sessions to an AS
bgp attributes & filters
p
p
p
No magic solution
AS 200
C
AS 201
Multihome?
p Definition & Options
p How to Multihome
p Principles & Addressing
p Basic Multihoming
p Service Provider Multihoming
p Using Communities
Basic Principles of
Multihoming
Lets learn to walk before we try
running
40
p Announcing
41
Results in:
n
n
n
42
43
IP Addressing &
Multihoming
How Good IP Address Plans
assist with Multihoming
44
101.10.0.1
101.10.5.255
101.10.6.255 /24
Infrastructure Loopbacks
45
Unplanned IP addressing
p
12345
Customer Addresses
p
ISP
n
n
Dividing the range into two pieces will result in one /22
with all the customers, and one /22 with just the ISP
infrastructure the addresses
No loadbalancing as all traffic will come in the first /22
Means further subdivision of the first /22 = harder work
47
Planned IP addressing
p
13579
2 4 6 810
Customer Addresses
Customer Addresses
ISP
48
Planned IP Addressing
p This
p Dont
Multihome?
p Definition & Options
p How to Multihome
p Principles & Addressing
p Basic Multihoming
p Service Provider Multihoming
p Using Communities
Basic Multihoming
Lets try some simple worked
examples
Basic Multihoming
p No
frills multihoming
p Will look at two cases:
Multihoming with the same ISP
n Multihoming to different ISPs
n
p Will
Basic Multihoming
p This
p Can
Basic Multihoming
Multihoming to the Same ISP
Basic Multihoming:
Multihoming to the same ISP
p Use
p Upstream
n
AS 100
E
p AS100
AS 65534
D
B
backup
primary link:
Outbound announce /19 unaltered
p Inbound receive default route
p
backup link:
Outbound announce /19 with increased metric
p Inbound received default, and reduce local
preference
p
p When
Router A Configuration
router bgp 65534
network 121.10.0.0 mask 255.255.224.0
neighbor 122.102.10.2 remote-as 100
neighbor 122.102.10.2 description RouterC
neighbor 122.102.10.2 prefix-list aggregate out
neighbor 122.102.10.2 prefix-list default in
!
ip prefix-list aggregate permit 121.10.0.0/19
ip prefix-list default permit 0.0.0.0/0
!
ip route 121.10.0.0 255.255.224.0 null0
60
Router B Configuration
router bgp 65534
network 121.10.0.0 mask 255.255.224.0
neighbor 122.102.10.6 remote-as 100
neighbor 122.102.10.6 description RouterD
neighbor 122.102.10.6 prefix-list aggregate out
neighbor 122.102.10.6 route-map routerD-out out
neighbor 122.102.10.6 prefix-list default in
neighbor 122.102.10.6 route-map routerD-in in
!
..next slide
61
62
63
64
Router E Configuration
router bgp 100
neighbor 122.102.10.17
neighbor 122.102.10.17
neighbor 122.102.10.17
!
ip prefix-list Customer
p
p
remote-as 110
remove-private-AS
prefix-list Customer out
permit 121.10.0.0/19
65
common case
p End sites tend not to buy circuits and
leave them idle, only used for backup as
in previous example
p This example assumes equal capacity
circuits
n
AS 100
E
AS 65534
D
B
Link two
68
p
p
Router A Configuration
router bgp 65534
network 121.10.0.0 mask 255.255.224.0
network 121.10.0.0 mask 255.255.240.0
neighbor 122.102.10.2 remote-as 100
neighbor 122.102.10.2 prefix-list routerC out
neighbor 122.102.10.2 prefix-list default in
!
ip prefix-list default permit 0.0.0.0/0
ip prefix-list routerC permit 121.10.0.0/20
ip prefix-list routerC permit 121.10.0.0/19
!
ip route 121.10.0.0 255.255.240.0 null0
ip route 121.10.0.0 255.255.224.0 null0
70
Router C Configuration
router bgp 100
neighbor 122.102.10.1 remote-as 65534
neighbor 122.102.10.1 default-originate
neighbor 122.102.10.1 prefix-list Customer in
neighbor 122.102.10.1 prefix-list default out
!
ip prefix-list Customer permit 121.10.0.0/19 le 20
ip prefix-list default permit 0.0.0.0/0
p
p
Router E Configuration
router bgp 100
neighbor 122.102.10.17
neighbor 122.102.10.17
neighbor 122.102.10.17
!
ip prefix-list Customer
remote-as 110
remove-private-AS
prefix-list Customer out
permit 121.10.0.0/19
72
73
configuration is only on
customer router
p Upstream ISP has to
remove customer subprefixes from external
announcements
n remove private AS from external
announcements
n
p Could
p Look
AS 100
E
A1
AS 65534
B1
D
A2
AS 65534
B2
A3
p
AS 65534
B3
77
example
p Use the same private AS for each
customer
documented in RFC2270
n address space is not overlapping
n each customer hears default only
n
p Router
Router A1 Configuration
router bgp 65534
network 121.10.0.0 mask 255.255.224.0
network 121.10.0.0 mask 255.255.240.0
neighbor 122.102.10.2 remote-as 100
neighbor 122.102.10.2 prefix-list routerC out
neighbor 122.102.10.2 prefix-list default in
!
ip prefix-list default permit 0.0.0.0/0
ip prefix-list routerC permit 121.10.0.0/20
ip prefix-list routerC permit 121.10.0.0/19
!
ip route 121.10.0.0 255.255.240.0 null0
ip route 121.10.0.0 255.255.224.0 null0
79
Router C Configuration
router bgp 100
neighbor bgp-customers peer-group
neighbor bgp-customers remote-as 65534
neighbor bgp-customers default-originate
neighbor bgp-customers prefix-list default out
neighbor 122.102.10.1 peer-group bgp-customers
neighbor 122.102.10.1 description Customer One
neighbor 122.102.10.1 prefix-list Customer1 in
neighbor 122.102.10.9 peer-group bgp-customers
neighbor 122.102.10.9 description Customer Two
neighbor 122.102.10.9 prefix-list Customer2 in
80
prefix-list
prefix-list
prefix-list
prefix-list
Router E Configuration
assumes customer address space is not part of
upstreams address block
router bgp 100
neighbor 122.102.10.17 remote-as 110
neighbor 122.102.10.17 remove-private-AS
neighbor 122.102.10.17 prefix-list Customers out
!
ip prefix-list Customers permit 121.10.0.0/19
ip prefix-list Customers permit 121.16.64.0/19
ip prefix-list Customers permit 121.14.192.0/19
n
82
Router E configuration:
router bgp 100
neighbor 122.102.10.17 remote-as 110
neighbor 122.102.10.17 prefix-list my-aggregate out
!
ip prefix-list my-aggregate permit 121.8.0.0/13
83
Multihoming Summary
p Use
84
Basic Multihoming
Multihoming to different ISPs
a Public AS
p Address
both upstreams or
n Regional Internet Registry
n
p Configuration
Inconsistent-AS?
p
IOS command is
AS 65534
AS 200
AS 210
Internet
87
AS 100
AS 120
C
Announce /19 block
D
A
AS 130
89
p When
Router A Configuration
router bgp 130
network 121.10.0.0 mask 255.255.224.0
neighbor 122.102.10.1 remote-as 100
neighbor 122.102.10.1 prefix-list aggregate out
neighbor 122.102.10.1 prefix-list default in
!
ip prefix-list aggregate permit 121.10.0.0/19
ip prefix-list default permit 0.0.0.0/0
!
ip route 121.10.0.0 255.255.224.0 null0
91
Router B Configuration
router bgp 130
network 121.10.0.0 mask 255.255.224.0
neighbor 120.1.5.1 remote-as 120
neighbor 120.1.5.1 prefix-list aggregate out
neighbor 120.1.5.1 route-map routerD-out out
neighbor 120.1.5.1 prefix-list default in
neighbor 120.1.5.1 route-map routerD-in in
!
ip prefix-list aggregate permit 121.10.0.0/19
ip prefix-list default permit 0.0.0.0/0
!
route-map routerD-out permit 10
set as-path prepend 130 130 130
!
route-map routerD-in permit 10
set local-preference 80
92
p But
AS 100
AS 120
C
Announce first
/20 and /19 block
D
A
Announce second
/20 and /19 block
AS 130
95
p When
Router A Configuration
router bgp 130
network 121.10.0.0 mask 255.255.224.0
network 121.10.0.0 mask 255.255.240.0
neighbor 122.102.10.1 remote-as 100
neighbor 122.102.10.1 prefix-list firstblock out
neighbor 122.102.10.1 prefix-list default in
!
ip prefix-list default permit 0.0.0.0/0
!
ip prefix-list firstblock permit 121.10.0.0/20
ip prefix-list firstblock permit 121.10.0.0/19
97
Router B Configuration
router bgp 130
network 121.10.0.0 mask 255.255.224.0
network 121.10.16.0 mask 255.255.240.0
neighbor 120.1.5.1 remote-as 120
neighbor 120.1.5.1 prefix-list secondblock out
neighbor 120.1.5.1 prefix-list default in
!
ip prefix-list default permit 0.0.0.0/0
!
ip prefix-list secondblock permit 121.10.16.0/20
ip prefix-list secondblock permit 121.10.0.0/19
98
AS 100
AS 120
C
Announce /19 block
D
A
AS 130
101
p Vary
Router A Configuration
router bgp 130
network 121.10.0.0 mask 255.255.224.0
neighbor 122.102.10.1 remote-as 100
neighbor 122.102.10.1 prefix-list default in
neighbor 122.102.10.1 prefix-list aggregate out
!
ip prefix-list aggregate permit 121.10.0.0/19
ip prefix-list default permit 0.0.0.0/0
!
ip route 121.10.0.0 255.255.224.0 null0
103
Router B Configuration
router bgp 130
network 121.10.0.0 mask 255.255.224.0
network 121.10.16.0 mask 255.255.240.0
neighbor 120.1.5.1 remote-as 120
neighbor 120.1.5.1 prefix-list default in
neighbor 120.1.5.1 prefix-list subblocks out
neighbor 120.1.5.1 route-map routerD out
!
route-map routerD permit 10
match ip address prefix-list aggregate
set as-path prepend 130 130
route-map routerD permit 20
!
ip prefix-list subblocks permit 121.10.0.0/19 le 20104
ip prefix-list aggregate permit 121.10.0.0/19
Multihome?
p Definition & Options
p How to Multihome
p Principles & Addressing
p Basic Multihoming
p BGP Traffic Engineering
p Using Communities
Service Provider
Multihoming
BGP Traffic Engineering
p Transit
p The
2.
People who sell router memory would like you to believe this
Only true if you are a transit provider
Full routing table can be a significant hindrance to multihoming
3.
BGP is complex
n
p Prefixes
p Use
n
p Require
Service Provider
Multihoming
One upstream, one local peer
C
Local Peer
AS120
AS 110
p Accept
Router A Configuration
Prefix filters
inbound
117
Router C Configuration
router bgp 110
network 121.10.0.0 mask 255.255.224.0
neighbor 122.102.10.1 remote-as 130
neighbor 122.102.10.1 prefix-list default in
neighbor 122.102.10.1 prefix-list my-block out
!
ip prefix-list my-block permit 121.10.0.0/19
ip prefix-list default permit 0.0.0.0/0
!
ip route 121.10.0.0 255.255.224.0 null0
119
p Local
Aside:
Configuration Recommendations
p Private
Peers
p Be
Service Provider
Multihoming
One upstream, Local Exchange
Point
C
A
AS 110
p Accept
Router A Configuration
interface fastethernet 0/0
description Exchange Point LAN
ip address 120.5.10.1 mask 255.255.255.224
!
router bgp 110
neighbor ixp-peers peer-group
neighbor ixp-peers prefix-list my-block out
neighbor ixp-peers remove-private-AS
neighbor ixp-peers send-community
neighbor ixp-peers route-map set-local-pref in
next slide
126
remote-as 100
peer-group ixp-peers
prefix-list peer100 in
remote-as 101
peer-group ixp-peers
prefix-list peer101 in
remote-as 102
peer-group ixp-peers
prefix-list peer102 in
remote-as 103
peer-group ixp-peers
prefix-list peer103 in
127
128
This ensures that BGP Best Path for local traffic will be
across the IXP
Router C Configuration
router bgp 110
network 121.10.0.0 mask 255.255.224.0
neighbor 122.102.10.1 remote-as 130
neighbor 122.102.10.1 prefix-list default in
neighbor 122.102.10.1 prefix-list my-block out
!
ip prefix-list my-block permit 121.10.0.0/19
ip prefix-list default permit 0.0.0.0/0
!
ip route 121.10.0.0 255.255.224.0 null0
130
Router A configuration
p IXP
Aside:
IXP Configuration Recommendations
p
IXP peers
n
n
Be aware that the IXP border router should carry only the
prefixes you want the IXP peers to receive and the
destinations you want them to be able to reach
n
Service Provider
Multihoming
Two Upstreams, One local peer
p Connect
AS140
C
Local Peer
AS120
AS 110
Upstream ISP
p Accept
p Router
n
Router C Configuration
router bgp 110
network 121.10.0.0 mask 255.255.224.0
neighbor 122.102.10.1 remote-as 130
neighbor 122.102.10.1 prefix-list default in
neighbor 122.102.10.1 prefix-list my-block out
!
ip prefix-list my-block permit 121.10.0.0/19
ip prefix-list default permit 0.0.0.0/0
!
ip route 121.10.0.0 255.255.224.0 null0
137
Router D Configuration
router bgp 110
network 121.10.0.0 mask 255.255.224.0
neighbor 122.102.10.5 remote-as 140
neighbor 122.102.10.5 prefix-list default in
neighbor 122.102.10.5 prefix-list my-block out
!
ip prefix-list my-block permit 121.10.0.0/19
ip prefix-list default permit 0.0.0.0/0
!
ip route 121.10.0.0 255.255.224.0 null0
138
139
configuration options:
140
Internet
Transit
AS 140
AS 130
Cust4
Cust5
AS 110
141
Router C Configuration
Router D Configuration
144
C configuration:
p Router
Full Routes
AS140
430000 @ lp100
AS130
30000 @ lp 120
400000 @ lp 80
Total
860000
Partial Routes
p Ask
Router C Configuration
...next slide
AS filter list filters
prefixes based on
origin ASN
149
150
Router D Configuration
router bgp 110
network 121.10.0.0 mask 255.255.224.0
neighbor 122.102.10.5 remote-as 140
neighbor 122.102.10.5 prefix-list default in
neighbor 122.102.10.5 prefix-list my-block out
!
ip prefix-list my-block permit 121.10.0.0/19
ip prefix-list default permit 0.0.0.0/0
!
ip route 121.10.0.0 255.255.224.0 null0
151
C configuration:
152
Full Routes
Partial Routes
AS140
430000 @ lp100
1 @ lp 100
AS130
30000 @ lp 120
400000 @ lp 80
30000 @ lp 100
1 @ lp 80
Total
860000
30002
Aside:
Configuration Recommendation
p When
Multihome?
p Definition & Options
p How to Multihome
p Principles & Addressing
p Basic Multihoming
p BGP Traffic Engineering
p Using Communities
160
161
RFC1998
p Informational
RFC
p Describes how to implement loadsharing
and backup on multiple inter-AS links
n
p Gives
n
p Simplifies
n
upstreams configuration
162
RFC1998
p
p
ASx :90
n
ASx :80
n
ASx :70
n
RFC1998
p
p
If upstream is AS 100
To declare a particular path as a backup path, their
customer would announce the prefix with community
100:70 to AS100
AS100 would receive the prefix with the community
100:70 tag, and then set local preference to be 70
164
RFC1998
p
RFC1998
p
RFC1998
route-map customer-policy-in
match community 7
set local-preference 70
!
route-map customer-policy-in
match community 8
set local-preference 80
!
route-map customer-policy-in
match community 9
set local-preference 90
!
route-map customer-policy-in
set local-preference 100
!
permit 10
permit 20
permit 30
permit 40
167
RFC1998
p
p
p
168
169
Background
p RFC1998
situations
p ISPs create backbone support for many
other communities to handle more
complex situations
Simplify ISP BGP configuration
n Give customer more policy control
n
170
RFC1998
The five standard communities
p
www.iana.org/assignments/bgp-well-known-communities
totem.info.ucl.ac.be/publications/papers-elec-versions/draftquoitin-bgp-comm-survey-00.pdf
But so far nothing more L
Collection of ISP communities at www.onesc.net/communities
www.nanog.org/meetings/nanog40/presentations/
BGPcommunities.pdf
X:80
n
X:120
n
X:3
n
X:2
n
X:1
n
X:666
n
permit 10
permit 20
permit 30
permit 40
174
ISP Example:
Verizon Business Europe
aut-num: AS702
descr:
Verizon Business EMEA - Commercial IP service provider in Eur
remarks: VzBi uses the following communities with its customers:
702:80
Set Local Pref 80 within AS702
702:120
Set Local Pref 120 within AS702
702:20
Announce only to VzBi AS'es and VzBi customers
702:30
Keep within Europe, don't announce to other VzBi AS
702:1
Prepend AS702 once at edges of VzBi to Peers
702:2
Prepend AS702 twice at edges of VzBi to Peers
702:3
Prepend AS702 thrice at edges of VzBi to Peers
Advanced communities for customers
702:7020 Do not announce to AS702 peers with a scope of
National but advertise to Global Peers, European
Peers and VzBi customers.
702:7001 Prepend AS702 once at edges of VzBi to AS702
peers with a scope of National.
702:7002 Prepend AS702 twice at edges of VzBi to AS702
peers with a scope of National.
(more)
177
ISP Example:
Verizon Business Europe
(more)
mnt-by:
source:
178
ISP Example:
BT Ignite
aut-num:
descr:
remarks:
remarks:
remarks:
remarks:
remarks:
remarks:
remarks:
remarks:
remarks:
remarks:
remarks:
remarks:
remarks:
remarks:
remarks:
remarks:
<snip>
notify:
mnt-by:
source:
AS5400
BT Ignite European Backbone
Community to
Not announce
Community to
AS prepend 5400
To peer:
5400:2000
5400:1500
5400:1501
5400:1502
5400:1503
5400:1504
5400:1506
5400:2500
5400:2501
5400:2502
5400:2503
5400:2504
5400:2506
All Transits
Sprint Transit (AS1239)
SAVVIS Transit (AS3561)
Level 3 Transit (AS3356)
AT&T Transit (AS7018)
GlobalCrossing Trans(AS3549)
5400:2001
5400:2002
5400:2004
And many
many more!
179
ISP Example:
Level 3
aut-num:
descr:
<snip>
remarks:
remarks:
remarks:
remarks:
remarks:
remarks:
remarks:
remarks:
remarks:
remarks:
remarks:
<snip>
remarks:
remarks:
remarks:
remarks:
<snip>
mnt-by:
source:
AS3356
Level 3 Communications
------------------------------------------------------customer traffic engineering communities - Suppression
------------------------------------------------------64960:XXX - announce to AS XXX if 65000:0
65000:0
- announce to customers but not to peers
65000:XXX - do not announce at peerings to AS XXX
------------------------------------------------------customer traffic engineering communities - Prepending
------------------------------------------------------65001:0
- prepend once to all peers
65001:XXX - prepend once at peerings to AS XXX
3356:70
3356:80
3356:90
3356:9999
LEVEL3-MNT
RIPE
set local
set local
set local
blackhole
preference to 70
preference to 80
preference to 90
(discard) traffic
And many
many more!
180
Conclusion: Communities
p Communities
are fun! J
p And they are extremely powerful tools
p Think about community policies, e.g. like
the additions described here
p Supporting extensive community usage
makes customer configuration easy
p Watch out for routing loops!
183
Summary
Summary
p Multihoming
n
p Full
BGP Multihoming
Techniques
End of Tutorial