BGP Monitoring

Protocol
Bertrand Duvivier (bduvivie@cisco.com)
Sr Product Manager
September, 2014

2013 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 1
BMP overview
What is BMP ?

The BGP Monitor Protocol

Provides access to the Adj-RIB-In (pre BGP inbound filtering)

of a peer (BR: Internet Border Router or PE: Provider Edge
Router) on an ongoing basis and a periodic dump of certain
statistics that the monitoring station can use for further
analysis.

http://tools.ietf.org/html/draft-ietf-grow-bmp-07.txt
BMP overview
BMP client/collector

BMP
BMP collector
collector

BMP message
Adj-RIB-in
Adj-RIB-in (pre-inbound-filter)
(pre-inbound-filter)
BGP
BGP Monitor
Monitor Protocol
Protocol update
update

Loc-RIB
Loc-RIB (post-inbound-filter)
(post-inbound-filter) Adj-RIB-in
Adj-RIB-in (pre-inbound-filter)
(pre-inbound-filter)
iBGP update
iBGP update eBGP update
eBGP update
BGP peer (internal)
BMP client
Inbound BGP peers (external)
filtering
policing
BMP overview
Full picture

1.BMP devices (e.g. routers) send BMP

messages to a BMP collector.

2.Open Daylight (ODL) controller SQL plugin

with SQL using Yang interfaces with the BMP
receiver database. ODL in this fashion provides
an abstract view of all BMP collector data.

3.Admins, Network Engineers, automated

programs/scripts, etc. interact via ODL
northbound interfaces to run various BMP
analytics.

4.Admins, Network Engineers, automated

programs/scripts, etc. can also go direct to the
BMP database as needed
BMP overview
Why BMP ?

There are many reasons to use BMP, but to highlight a few common ones:
1. Looking Glasses - IPv4, IPv6, and VPN4

2. Route Analytics - Track convergence times, history of prefixes as they change

over time, monitor and track BGP policy changes, etc...
3. Traffic Engineering Analytics - Adapt dynamically to change and know what is
the best shift
4. BGP pre-policy What-Ifs - Pre-policy routing information provides insight into all
path attributes from various points in the network allowing nonintrusive what-if
topology views for new policy validations
many more
BMP implementation
BMP client
- ASR1K, CSR1000v : IOS-XE 3.11
- ASR9K, CRS, NCS6K, XRv: IOS-XR 5.2.2
- N3K, N7K, N9K: TBD
- Others: Juniper (older version of the draft)

BMP collector
- OpenBMP: Open Source: using MySQL
https://www.openbmp.org/

- BMP receiver: Open Source: Writen from scratch in Python https

://code.google.com/p/bmpreceiver/
BMP client
Design (1)

BMP receiver can be configured with both ipv4 & ipv6 host addresses.
The BGP speaker process is referred to as the BMP Client.
BMP client provides only pre-policy view of the ADJ-RIB-IN of a peer
Post-policy view is not supported

A BGP peer can be monitored by multiple BMP reciever

Any update message from the peer ( irrespective of the address-family ) is sent to the
BMP receiver

Multiple BMP receivers can be configured across all BGP instances

Each BGP instance will send update messages for peers under it to the BMP receivers
monitoring the corresponding peers
BMP Client
Design (2)

Maximum of 8 BMP receivers can be configured.

No limitation on number of BGP peers to be monitored by a receiver

No limitation on the number of clients interacting with a receiver
Each BGP instance maintains a queue per BMP receiver
Once the session with the BMP receiver comes up, the client starts sending
the current view (unless configured not to) and any subsequent updates.
The BGP client is the Active side & the BMP receiver is the passive listener
BMP Client
Design (3)

Once the TCP connection with a BMP receiver comes up, BMP client does the
following
Sends the Initiation message to BMP receiver
This contains the host name and system software info (version)
Sends Peer-Up message for every ESTABLISHED peer
Send Route-Refresh message to all ESTABLISHED peers (unless configured not to)
Peer-Down message for every peer which goes down
Periodic Stats-Report messages
Route-monitoring messages for any update received from the monitored peer
Termination message if session with BMP receiver is brought down gracefully
( unconfigured )
BMP Client
Design (4)

When BMP client receives updates from a peer, it makes a copy of the
message and add it to the queue of all BMP receivers monitoring the peer
A separate thread in BGP speaker process then flushes the updates out on the tcp
session to the BMP receiver
A copy of the message is made for every monitoring BMP receiver

If a TCP session with the BMP receiver cannot be established, the Client
process will retry the connect every 7 secs
BMP
BMP messages
Type Description
0 Route Monitor
1 Statistics Report
2 Peer Down Notification
3 Peer Up Notification
4 Initiation
5 Termination
Implementation brief
Received update messages
No Cisco specific statistics
BGP peer down
BGP peer up
BMP session starts, will carry the session
description and basic informations
BMP session ends
BMP
BMP Stats Reports

Stat type
0 (standard) Number of prefixes rejected by inbound policy.
1 (standard) Number of (known) duplicate prefix advertisements. Currently sending 0.
2 (standard) Number of (known) duplicate withdraws. Currently sending 0.
3 (standard) Number of updates invalidated due to CLUSTER_LIST loop.
4 (standard) Number of updates invalidated due to AS_PATH loop.
5 (standard) Number of updates invalidated due to ORIGINATOR_ID.
6 (standard) Number of updates invalidated due to AS_CONFED loop.
7 (standard) Number of routes in Adj-RIBs-In.
8 (standard) Number of routes in Loc-RIB.
 BMP Client
Client config.

bmp receiver <1-8>

description <string>
dscp <dscp-value>
host <name | ipv4/6 address> port <1-65535>
initial-delay <1-3600> // before attempting to connect to bmp receiver
initial-refresh {delay <1-3600> spread <1-3600> | skip}
precedence <precedence-value>
shutdown
stats-reporting-period <1-3600>
update-source <interface-id>
vrf <vrf-id>
router bgp <asn>
neighbor x.x.x.x
bmp-activate receiver <1-8>
OpenBMP
Overview

1. OpenBMP use MySQL to store BMP record

use MySQL Workbench (client) to read it

2. Live OpenBMP server:

db1.openbmp.org (login/pw = demo/demo)

3. Few command:

Command MySQL Qweries

Show current status of BGP peers select * from v_peers order by RouterName
Show BMP Routers select * from routers
Show BGP peer statistics interval based select * from v_peer_prefix_report_last
report info
OpenBMP
MySQL Worbench initial config
OpenBMP
MySQL Qwery
BMP summary

1. Allows to collect all peers adj-RIB-in BGP updates

2. BMP cisco implementation is based on draft BMPv7
3. Supported on IOS-XE 3.11 and IOS XR 5.2.2
4. Easy to deploy using OpenBMP & BMPreceiver open source receiver
Question: ask-bmp@cisco.com
Thank you.
 BMP Client - Show
RP/0/0/CPU0:R2#sh bgp bmp receiver 1 RP/0/0/CPU0:R2#sh bgp bmp receiver 5
BMP receiver 1 BMP receiver 5
Host 10.0.101.1 Port 16666 Host 10.0.101.1 Port 16666
Connected for 00:30:40 since Dec 31 16:30:40.613 Connected for 00:25:07 since Dec 31 16:25:07.269
Precedence: internet Precedence: internet
BGP neighbors: 1 BGP neighbors: 1
VRF: - (0x60000000) VRF: - (0x60000000)
Update Source: - (-) Update Source: 9.9.9.9 (Lo9)
Update Source Vrf ID: 0x0, Active: N Update Source Vrf ID: 0x60000000, Active: Y

Message Stats: Message Stats:

Total messages sent: 31 Total messages sent: 60
INITIATION: 1 INITIATION: 1
TERMINATION: 0 TERMINATION: 0
STATS-REPORT: 0 STATS-REPORT: 0
PER-PEER messages: 30 PER-PEER messages: 59

Neighbor 20.0.101.11 Neighbor 20.0.101.11

Messages pending: 0 Messages pending: 0
Messages sent: 30 Messages sent: 59
PEER-UP: 1 PEER-UP: 1
PEER-DOWN: 0 PEER-DOWN: 0
ROUTE-MON: 29 ROUTE-MON: 58
BMP Client - Show

RP/0/0/CPU0:R2#sh bgp nei 20.0.101.11 | beg BMP

BGP Monitoring(BMP) activated for receivers:

1, 3, 5
.
 BMP Client Debug debug bgp bmp

Type 0 - Route monitoring message:

RP/0/0/CPU0:Jul 1 14:31:57 : bgp[1039]: [default-iord]: [BMP Messagw Dump : receiver (NULL), nbr 20.0.101.11, type Route Monitoring]:
RP/0/0/CPU0:Jul 1 14:31:57 : bgp[1039]: [default-iord]: 0300 0000 cb00 0000 0000 0000 0000 0000
RP/0/0/CPU0:Jul 1 14:31:57 : bgp[1039]: [default-iord]: 0000 0000 0000 0000 0000 0000 1400 650b
RP/0/0/CPU0:Jul 1 14:31:57 : bgp[1039]: [default-iord]: 0000 0065 1400 650b 53b3 28cd 0000 6e5c
P/0/0/CPU0:RJul 1 14:31:57 : bgp[1039]: [default-iord]: ffff ffff ffff ffff ffff ffff ffff ffff
RP/0/0/CPU0:Jul 1 14:31:57 : bgp[1039]: [default-iord]: 009b 0200 0000 2040 0101 0040 0304 1400
RP/0/0/CPU0:Jul 1 14:31:57 : bgp[1039]: [default-iord]: 650b 4002 0a02 0400 6500 c900 ca00 cbf0
RP/0/0/CPU0:Jul 1 14:31:57 : bgp[1039]: [default-iord]: 0800 0400 6503 e720 4702 0101 2047 0201
RP/0/0/CPU0:Jul 1 14:31:57 : bgp[1039]: [default-iord]: 0220 4702 0103 2047 0201 0420 4702 0105
RP/0/0/CPU0:Jul 1 14:31:57 : bgp[1039]: [default-iord]: 2047 0201 0620 4702 0107 2047 0201 0820
RP/0/0/CPU0:Jul 1 14:31:57 : bgp[1039]: [default-iord]: 4702 0109 2047 0201 0a20 4702 010b 2047
RP/0/0/CPU0:Jul 1 14:31:57 : bgp[1039]: [default-iord]: 0201 0c20 4702 010d 2047 0201 0e20 4702
RP/0/0/CPU0:Jul 1 14:31:57 : bgp[1039]: [default-iord]: 010f 2047 0201 1020 4702 0111 2047 0201
RP/0/0/CPU0:Jul 1 14:31:57 : bgp[1039]: [default-iord]: 1220 4702 0113 2047 0201 14
BMP Client Debug debug bgp bmp

Type 1 - Stat-report message:

RP/0/0/CPU0:Aug 18 10:22:03 : bgp[1039]: [default-ext]: [BMP Message Dump : receiver 10.0.101.1,
nbr 20.0.101.11, type Statistics Report]:
RP/0/0/CPU0:Aug 18 10:22:03 : bgp[1039]: [default-ext]: 0300 0000 6401 0000 0000 0000 0000 0000
RP/0/0/CPU0:Aug 18 10:22:03 : bgp[1039]: [default-ext]: 0000 0000 0000 0000 0000 0000 1400 650b
RP/0/0/CPU0:Aug 18 10:22:03 : bgp[1039]: [default-ext]: 0000 0065 1400 650b 53f2 363b 000b 20f1
RP/0/0/CPU0:Aug 18 10:22:03 : bgp[1039]: [default-ext]: 0000 0005 0001 0004 0000 0090 0004 0004
RP/0/0/CPU0:Aug 18 10:22:03 : bgp[1039]: [default-ext]: 0000 0028 0005 0004 0000 0038 0007 0008
RP/0/0/CPU0:Aug 18 10:22:03 : bgp[1039]: [default-ext]: 0000 0000 0000 0018 0008 0008 0000 0000
RP/0/0/CPU0:Aug 18 10:22:03 : bgp[1039]: [default-ext]: 0000 0018
 BMP Client Debug debug bgp bmp
Type 2 - Peer-down message:

RP/0/0/CPU0:Aug 18 11:01:14 : bgp[1039]: [default-mgmt]: [BMP Message Dump : receiver (NULL), nbr 20.0.101.11, type
Peer Down Notification]:

RP/0/0/CPU0:Aug 18 11:01:14 : bgp[1039]: [default-mgmt]: 0300 0000 4602 0000 0000 0000 0000 0000

RP/0/0/CPU0:Aug 18 11:01:14 : bgp[1039]: [default-mgmt]: 0000 0000 0000 0000 0000 0000 1400 650b

RP/0/0/CPU0:Aug 18 11:01:14 : bgp[1039]: [default-mgmt]: 0000 0065 1400 650b 53f2 3f6a 000c fd3a

RP/0/0/CPU0:Aug 18 11:01:14 : bgp[1039]: [default-mgmt]: 01ff ffff ffff ffff ffff ffff ffff ffff

RP/0/0/CPU0:Aug 18 11:01:14 : bgp[1039]: [default-mgmt]: ff00 1503 0602

 BMP Client Debug debug bgp bmp
Type 3 Peer-up message:
RP/0/0/CPU0:Aug 18 11:03:22 : bgp[1039]: [default-iord]: [BMP Message Dump : receiver (NULL), nbr 20.0.101.11, type Peer Up Notification]:

RP/0/0/CPU0:Aug 18 11:03:22 : bgp[1039]: [default-iord]: 0300 0000 ac03 0000 0000 0000 0000 0000

RP/0/0/CPU0:Aug 18 11:03:22 : bgp[1039]: [default-iord]: 0000 0000 0000 0000 0000 0000 1400 650b

RP/0/0/CPU0:Aug 18 11:03:22 : bgp[1039]: [default-iord]: 0000 0065 0000 0000 53f2 3fea 0005 3282

RP/0/0/CPU0:Aug 18 11:03:22 : bgp[1039]: [default-iord]: 0000 0000 0000 0000 0000 0000 1400 0002

RP/0/0/CPU0:Aug 18 11:03:22 : bgp[1039]: [default-iord]: 00b3 7f51 ffff ffff ffff ffff ffff ffff

RP/0/0/CPU0:Aug 18 11:03:22 : bgp[1039]: [default-iord]: ffff ffff 003b 0104 0064 00b4 0202 0202

RP/0/0/CPU0:Aug 18 11:03:22 : bgp[1039]: [default-iord]: 1e02 0601 0400 0100 0102 0280 0002 0202

RP/0/0/CPU0:Aug 18 11:03:22 : bgp[1039]: [default-iord]: 0002 0641 0400 0000 6402 0440 0200 78ff

RP/0/0/CPU0:Aug 18 11:03:22 : bgp[1039]: [default-iord]: ffff ffff ffff ffff ffff ffff ffff ff00

RP/0/0/CPU0:Aug 18 11:03:22 : bgp[1039]: [default-iord]: 2d01 0400 6500 b414 0065 0b10 0206 0104

RP/0/0/CPU0:Aug 18 11:03:22 : bgp[1039]: [default-iord]: 0001 0001 0202 8000 0202 0200
 BMP Client Debug debug bgp bmp

Type 4 - Initiation message:

RP/0/0/CPU0:Aug 18 10:08:13 : bgp[1039]: [default-ext]: [BMP Message Dump : receiver 10.0.101.1, nbr (None), type Initiation
Message]:

RP/0/0/CPU0:Aug 18 10:08:13 : bgp[1039]: [default-ext]: 0300 0000 6b04 0001 005b 4369 7363 6f20

RP/0/0/CPU0:Aug 18 10:08:13 : bgp[1039]: [default-ext]: 494f 5320 5852 2053 6f66 7477 6172 652c

RP/0/0/CPU0:Aug 18 10:08:13 : bgp[1039]: [default-ext]: 2056 6572 7369 6f6e 2035 2e32 2e32 2e31

RP/0/0/CPU0:Aug 18 10:08:13 : bgp[1039]: [default-ext]: 3849 5b44 6566 6175 6c74 5d0a 436f 7079

RP/0/0/CPU0:Aug 18 10:08:13 : bgp[1039]: [default-ext]: 7269 6768 7420 2863 2920 3230 3134 2062

RP/0/0/CPU0:Aug 18 10:08:13 : bgp[1039]: [default-ext]: 7920 4369 7363 6f20 5379 7374 656d 732c

RP/0/0/CPU0:Aug 18 10:08:13 : bgp[1039]: [default-ext]: 2049 6e63 2e00 0200 0252 32

 BMP Client Debug debug bgp bmp
Type 5 - Termination message:
RP/0/0/CPU0:Aug 18 09:37:33 : bgp[1039]: [default-mgmt]: [10.0.101.1] receiver close called with reason 0x3 [state ESTAB]

RP/0/0/CPU0:Aug 18 09:37:33 : bgp[1039]: [default-mgmt]: [BMP Message Dump : receiver 10.0.101.1, nbr (None), type
Termination Message]:

RP/0/0/CPU0:Aug 18 09:37:33 : bgp[1039]: [default-mgmt]: 0300 0000 1f05 0000 000f 636f 6e66 6967

RP/0/0/CPU0:Aug 18 09:37:33 : bgp[1039]: [default-mgmt]: 2073 6875 7464 6f77 6e00 0100 0200 00

RP/0/2/CPU0:Aug 18 09:37:33 : bgp[1039]: [default-mgmt]: [10.0.101.1] receiver close called with reason 0x3 [state NONE]