You are on page 1of 93

3302

1300_05_2000_c2

2000,
2001, Cisco Systems, Inc.

ISP Essentials
Best Practice
Cisco IOS Techniques
to Scale the Internet
Session XXXX
Version 4
I3302
1300_05_2000_c2

2000, Cisco Systems, Inc.

Who Should Attend?


Engineers from Existing ISPs, ASPs,
Telcos, and other Internet based
service providers.
Consultants/CCIEs working with
Internet based service providers.
Anyone else interested in the gory
IOS details.
3302
1300_05_2000_c2

2001, Cisco Systems, Inc.

Prerequisites

This is not for people brand new to


networking and IOS
Know a bit about IOS.
Know a bit about OSPF and BGP
Know a bit about TCP/IP
3302
1300_05_2000_c2

2001, Cisco Systems, Inc.

Agenda for the Day

General Features
ISP Security
Routing Configuration Guidelines and
Updates
Operations Essentials

3302
1300_05_2000_c2

2001, Cisco Systems, Inc.

Changes from Last Year


Fundamentals and Essentials do not
change much.
The objective of this Power Session is to
get ISPs to consider and turn features on
that will make their life easier.
This Power Session is given through out
the year as a stand alone ISP Seminar. It is
part of the ISP Workshop program:
http://www.cisco.com/public/cons/

3302
1300_05_2000_c2

2001, Cisco Systems, Inc.

Changes from Last Year


Updates/Changes since last year:

3302
1300_05_2000_c2

New updates, features, and clarifications


added.

ISP Architecture Essentials Section pulled


into a separate session RST-211.

BGP Updates pulled into the BGP Power


Session (PS-545) and the BGP sessions
during the week.

Operations Essentials Section added so


people building/running NOCs understand
some of the essentials of what a ISPs
Operations Team should be doing.

2001, Cisco Systems, Inc.

General IOS Features

I3302
1300_05_2000_c2

2000, Cisco Systems, Inc.

Which IOS Version?

I3302
Presentation_ID
1300_05_2000_c2

1999,
2000, Cisco Systems, Inc.

www.cisco.com

Cisco IOS Roadmap

3302
1300_05_2000_c2

http://www.cisco.com/warp/public/620/roadmap.shtml
2001, Cisco Systems, Inc.

10

Cisco IOS Roadmap

3302
1300_05_2000_c2

http://www.cisco.com/warp/public/620/roadmap.shtml
2001, Cisco Systems, Inc.

11

12.0S and its Children


11.2GS
11.3T
11.1CC

12.0 (Mainline)
12.0S (SP)(New features and Platforms)

11.3AA

(Controlled Release)
12.0ST (T for Tag integration branch)
MPLS//VPN/FRR/LDP...

Near Future:

12.0S Hardware
Additions Only

12.0ST New Software


Features

(12.0SL Cisco 10000)


12.0SC (uBR)

Both will run through the


same internal testing.
3302
1300_05_2000_c2

2001, Cisco Systems, Inc.

12

Parent Child Relationship


12.0S and 12.0ST Example
12.0(15)S
4xOC48

12.0S

12.0(15)ST

12.0(16)ST
4xOC48

12.0ST
TESTING

Sync from S to ST software train


4xOC48

Line card (feature) example

All Bug Fixes in 12.0S get synced to 12.0ST


New feature (eg. line cards) in 12.0S syncs to 12.0ST
The feature must be regression and dev tested in 12.0ST
The feature is FCSd in 12.0ST the next release cycle
3302
1300_05_2000_c2

2001, Cisco Systems, Inc.

13

Which IOS version?


Platforms
GSR,

7500 series, 7200 series

Recommended release is 12.0S train


Current

version is 12.0(16)S (as of May 2001)

Available

on CCO

Has all of latest ISP supported


features
3302
1300_05_2000_c2

2001, Cisco Systems, Inc.

14

Which IOS version?


Platforms

OSR 7600

Recommended release is 12.1E train today


and 12.2S in the future.

Current version is 12.1(7)E (as of May 2001)

Available on CCO

Working with customers to have all the


necessary features needed by ISP.
Processes updated to ISP Expectations
(work in progress)
3302
1300_05_2000_c2

2001, Cisco Systems, Inc.

15

Which IOS version?


Platforms

10000

Recommended release is 12.0SL train


today and 12.0ST in the near future

Current version is 12.0(15)SL (as of May 2001)

Available on CCO

Child of 12.0S with some platform specific


features.
3302
1300_05_2000_c2

2001, Cisco Systems, Inc.

16

Which IOS version?


Platforms

4x00, 3600, 2600 and 2500 series

Recommended release is the 12.0


mainline train

3302
1300_05_2000_c2

Current version is 12.0(16)

Has many of the features found in 11.1CC,


11.2P and 11.3T

Available on CCO

2001, Cisco Systems, Inc.

17

IOS Road Map


Future Direction
12.1E (OSR,
CAt6K, 7X00)
12.0S (12XXX,
7500, 7200)
12.0ST (GSR,
7500, 7200, 10K)
3302
1300_05_2000_c2

2001, Cisco Systems, Inc.

12.1E is transitioning to
support similar to 12.0S

12.2S (12XXX,
10XXX, & 7XXX)

It will be approx one


year from the launch of
12.2S before ISPs start
considering a move.
18

Cisco IOS Feature Navigator

http://www.cisco.com/go/fn/
3302
1300_05_2000_c2

2001, Cisco Systems, Inc.

19

IOS Software and


Router Management

I3302
Presentation_ID
1300_05_2000_c2

1999,
2000, Cisco Systems, Inc.

www.cisco.com

20

IOS Software Management


Flash Memory
Good practice is to have at least two
distinct flash memory volumes

allows backup image(s)

back out path in case of upgrade problems

Partition the built-in flash

partition flash 2 8 8

Install a PCMCIA flash card in external


slot(s) - 20Meg flash cards are worth it.
3302
1300_05_2000_c2

2001, Cisco Systems, Inc.

21

IOS Software Management


Flash Memory
Ensure that there is a configured backup
to selected IOS image

backup image is previous good image


boot system flash slot0:rsp-pv-mz.120-10.S
boot system flash slot1:rsp-pv-mz.111-32.CC
boot system flash

3302
1300_05_2000_c2

which means boot quoted image from slot0:.


If it isnt there, boot the quoted image in
slot1:. If that isnt there, try the first image
available in flash

2001, Cisco Systems, Inc.

22

IOS Software Management


System Memory
Good practice is to maximise router
memory

allows for the rapidly growing Internet

128Mbytes needed for full Internet routing


table

will (just) work with 64Mbytes, but BGP


inefficient

Recognised that equipment works best


when left alone
3302
1300_05_2000_c2

2001, Cisco Systems, Inc.

23

IOS Software Management


When to Upgrade
Upgrades needed when:

bug fixes released

new hardware support

new software features required

Otherwise:

If it isnt broken, dont fix it!


3302
1300_05_2000_c2

2001, Cisco Systems, Inc.

24

Digression - Loopback
Interface
Most ISPs make use of the router
loopback interface.
IP address configured is a host address
Configuration example:
interface loopback 0
description Loopback Interface of CORE-GW3
ip address 215.18.3.34 255.255.255.255

3302
1300_05_2000_c2

2001, Cisco Systems, Inc.

25

Digression - Loopback
Interface
Loopback interfaces on ISP backbone
usually numbered:
out

of one contiguous block, or

using

a geographical scheme, or

using

a per PoP scheme

Aim is to increase stability, aid


administration, and improve security
3302
1300_05_2000_c2

2001, Cisco Systems, Inc.

26

Digression - Loopback
Interface
ACLs
Router
w/Loopback
Exporting
Information

TCP
Wrapper
TFTP
TFTP

Backbone
Backbone

SYSLOG
SYSLOG

NOC Services

TACACS+
TACACS+
SNMP
SNMP

3302
1300_05_2000_c2

Topology changes do not effect the


source IP address of the packets
coming from the Router.
2001, Cisco Systems, Inc.

TCP
Wrapper

27

Digression - Loopback
Interface
Loopback interface is not
redundant or superfluous
Multitude of uses to ease security,
access, management, information
and scalability of router and network
Protects the ISPs Management
Systems
Use the loopback!
3302
1300_05_2000_c2

2001, Cisco Systems, Inc.

28

Configuration Management
Backup NVRAM configuration off the
router:

write configuration to TFTP server

TFTP server files kept under revision control

router configuration built from master


database

Allows rapid recovery in case of


emergency
3302
1300_05_2000_c2

2001, Cisco Systems, Inc.

29

Configuration Management
TFTP
Source
Loopback 0

Secure the TFTP


Server

TFTP Loopback 0 on
Router

Firewall/ACL

Wrapper on TFTP
Server which only
allows the routers
loopback address

Firewall
or ACL

ip
ip tftp
tftp source-interface
source-interface Loopback0
Loopback0
3302
1300_05_2000_c2

2001, Cisco Systems, Inc.

TFTP
server

TCP
Wrapper or
other tool

30

FTP Client Support

TFTP has its security limitations.


FTP Client support is added in 12.0. This
allows for FTP upload/downloads.
Remember to use the same
security/redundancy options with
loopback 0:
ip ftp source-interface loopback 0

3302
1300_05_2000_c2

2001, Cisco Systems, Inc.

31

FTP Client Support


7206-AboveNet-SJ2#copy ftp://bgreene:XXX@ftp.cisco.com slot0:
Source filename []? /cisco/ios/12.0/12.0.9S/7200/c7200-k3pmz.120-9.S.bin
Destination filename [c7200-k3p-mz.120-9.S.bin]?
Accessing ftp://bgreene:XXX@ftp.cisco.com
//cisco/ios/12.0/12.0.9S/7200/c7200-k3p-mz.1209.S.bin...Translating "ftp.cisco.com"...domain server
(207.126.96.162) [OK]

Loading /cisco/ios/12.0/12.0.9S/7200/c7200-k3p-mz.120-9.S.bin

3302
1300_05_2000_c2

2001, Cisco Systems, Inc.

32

Larger Configurations
Compress Configuration

Used when configuration required is larger


than configuration memory (NVRAM)
available.

service compress-config

FLASH or remote server

3302
1300_05_2000_c2

Used when NVRAM compression is not


enough

2001, Cisco Systems, Inc.

33

Use Detailed Logging


Off load logging information to a logging
server.
Use the full detailed logging features to keep
exact details of the activities.
service timestamps debug datetime msec localtime show-timezone
service timestamps log datetime msec localtime show-timezone
logging buffered 16384
logging trap debugging
logging facility local7
logging 169.223.32.1
logging source-interface loopback0
no logging console
3302
1300_05_2000_c2

2001, Cisco Systems, Inc.

! Optional - keeps the console port free


34

Use Detailed Logging


Two Topologies used:

Central Syslog Servers in Operations Center

Syslog Servers in Major POPs

unix% tail cisco.log


Feb 17 21:48:26 [10.1.1.101.9.132] 31: *Mar 2 11:51:55 CST:
%SYS-5-CONFIG_I: Configured from console by vty0 (10.1.1.2)
unix% date
Tue Feb 17 21:49:53 CST 1998
unix%
3302
1300_05_2000_c2

2001, Cisco Systems, Inc.

35

Network Time Protocol


If you want to cross compare logs, you
need to synchronize the time on all the
devices.
Use NTP

From external time source


Upstream ISP, Internet, GPS, atomic clock

3302
1300_05_2000_c2

From internal time source

Router can act as stratum 1 time source

2001, Cisco Systems, Inc.

36

Network Time Protocol


Set timezone
clock timezone <name> [+/-hours [mins]]

Router as source
ntp master 1

External time source (master)


ntp server a.b.c.d

External time source (equivalent)


ntp peer e.f.g.h
3302
1300_05_2000_c2

2001, Cisco Systems, Inc.

37

Network Time Protocol

Example Configuration:
clock timezone SST 8
ntp update-calendar
ntp source loopback0
ntp server <other time source>
ntp peer <other time source>
ntp peer <other time source>

3302
1300_05_2000_c2

2001, Cisco Systems, Inc.

38

Network Time Protocol


Network Time Protocol (NTP) used to
synchronize the time on all the devices.
NTP packets leave router with loopback
address as source
Configuration example:
ntp source loopback0
ntp server 169.223.1.1 source loopback 1
3302
1300_05_2000_c2

2001, Cisco Systems, Inc.

39

Network Time Protocol


Motivation - NTP Security:

NTP systems can be protected by filters


which only allow the NTP port to be
accessed from the loopback address
block

Motivation - Easy to understand NTP


peerings:

3302
1300_05_2000_c2

NTP associations have the loopback


address recorded as source address, not
the egress interface.

2001, Cisco Systems, Inc.

40

Network Time Protocol


NTP "Source" - Stratum 1
Atomic or GPS Based

Core Backbone
Routers

Neighboring
POP

Neighboring
POP

NTP "Backbone" - Stratum 2


Core 1

Core 2

NTP
Servers for
the POP

POP
Interconnect
Medium

AAA
Server w/
Radius

Cache
Engine
Cluster

SW 1

SW 2

Access 2

Dedicated
Access

Customer's NTP
Stratum 4
3302
1300_05_2000_c2

2001, Cisco Systems, Inc.

Customer's NTP
Stratum 4

NAS 1

POP Services
&
Applications
NetFlow
Collector
and
Syslog
Server

NTP in the POP - Stratum 3

Access 1

NTP
Servers for
the POP

NAS 2

Dial-up

Dial-up SNTP
Stratum 4

41

Network Time Protocol


Where to get NTP Reference
Sources?

http://www.eecis.udel.edu/~ntp/hardware.html

Attaching a Telecom Solutions GPS


Clock to the Routers AUX port:
Excalabur(config)#line aux 0
Excalabur(config-line)#ntp refclock telecom-solutions pps ?
cts PPS on CTS
none No PPS signal available
ri
3302
1300_05_2000_c2

PPS on RI

2001, Cisco Systems, Inc.

42

SNMPv1
Remove any SNMP commands if SNMP is
not going to be used!
If SNMP is going to be used:

3302
1300_05_2000_c2

access-list 98 permit 169.223.1.1

access-list 98 deny

snmp-server community 5nmc02m RO 98

snmp-server trap-source Loopback0

snmp-server trap-authentication

snmp-server host 169.223.1.1 5nmc02m

2001, Cisco Systems, Inc.

any

43

SNMPv1
Recommend that all ISPs aggressively and
consistently metric their network.
Despite SNMPv2 and SNMPv3, most ISPs are
still using SNMPv1 (personal observation)
SNMPv3 supported since 12.0(6)S.

3302
1300_05_2000_c2

2001, Cisco Systems, Inc.

44

HTTP Server
HTTP Server in IOS from 11.1CC and 12.0S

router configuration via web interface

Disable if not going to be used (disabled


by default):
no ip http server

Configure securely if going to be used:


ip http server
ip http port 8765
ip http authentication aaa
ip http access-class <1-99>
3302
1300_05_2000_c2

2001, Cisco Systems, Inc.

45

Core Dumps
Cisco routers have a core dump feature
that will allow ISPs to transfer a copy of
the core dump to a specific FTP server.
Set up a FTP account on the server the
router will send the core dump to.
The server should NOT be a public server

3302
1300_05_2000_c2

Use filters and secure accounts

Locate in NOC with NOC Staff access only

Enough Disk Space to handle the dumps

2001, Cisco Systems, Inc.

46

Core Dumps

Example configuration:
ip ftp username cisco
ip ftp password 7 045802150C2E
ip ftp source-interface loopback 0
exception protocol ftp
exception dump 169.223.32.1

3302
1300_05_2000_c2

2001, Cisco Systems, Inc.

47

General Features

I3302
ISP/IXP
Workshops
1300_05_2000_c2

1999,
2000, Cisco Systems, Inc.

www.cisco.com

48

Command Line Interface


Features
Some Convenient Editing Keys

3302
1300_05_2000_c2

TAB

arrow keys scroll history buffer

ctrl A

beginning of line

ctrl E

end of line

ctrl K

delete all chars to end of line

ctrl X

delete all chars to beginning of line

ctrl W

esc B

back one word

esc F

forward one word

2001, Cisco Systems, Inc.

command completion

delete word to left of cursor

49

Command Line Interface


Features
CLI now has string searches

show configuration | [begin|include|exclude]


<regexp>

Pager --more-- now has string searches

/<regexp>, -<regexp>, +<regexp>

More command has string searches

3302
1300_05_2000_c2

more <filename> | [begin|include|exclude]


<regexp>

2001, Cisco Systems, Inc.

50

Command Line Interface


Features
Example:
Defiant#show running-config | begin router bgp
router bgp 200
no synchronization
neighbor 4.1.2.1 remote-as 300
neighbor 4.1.2.1 description Link to Excalabur
neighbor 4.1.2.1 send-community
neighbor 4.1.2.1 version 4
neighbor 4.1.2.1 soft-reconfiguration inbound
neighbor 4.1.2.1 route-map Community1 out
maximum-paths 2

--More-3302
1300_05_2000_c2

2001, Cisco Systems, Inc.

51

Interface Configuration
ip unnumbered

no need for an IP address on point-to-point links

keeps IGP small

description

customer name, circuit id, cable number, etc

on-line documentation!

bandwidth

3302
1300_05_2000_c2

used by IGP

documentation!

2001, Cisco Systems, Inc.

52

Interface Configuration Example


ISP router

Customer router

interface loopback 0

interface Ethernet 0

description Loopback interface on GW2 Router

description Galaxy Publications LAN

ip address 215.17.3.1 255.255.255.255

ip address 215.34.10.1 255.255.252.0

interface Serial 5/0

interface Serial 0

description 128K HDLC link to Galaxy


Publications Ltd [galpub1] WT50314E R5-0

description 128K HDLC link to Galaxy


Internet Inc WT50314E C0

bandwidth 128

bandwidth 128

ip unnumbered loopback 0

ip unnumbered ethernet 0

ip route 215.34.10.0 255.255.252.0 Serial 5/0

ip route 0.0.0.0 0.0.0.0 Serial 0

3302
1300_05_2000_c2

2001, Cisco Systems, Inc.

53

Cisco Express Forwarding


(CEF)
Rationalechanging Internet traffic/topology dynamics
required optimized L3 switching paradigm for IP:
Traffic Driven

Topology Driven

Stable traffic patterns


Performance fluctuations
Demand caching

Dynamic environment
Predictable, scaleable, performance
Full topology forwarding

NetFlow Services
Deployed at
Backbone Periphery
for Network Services:
Traffic Accounting
QoS Policy

Cisco Express Forwarding


Deployed at Network Core for:
Performance
Scalability
Quality of Service

Security
3302
1300_05_2000_c2

2001, Cisco Systems, Inc.

54

What Is CEF?
CEF: Cisco Express Forwarding

3302
1300_05_2000_c2

Better known as FIB

Designed to be simple, fastest forwarding path


for IPv4 packets, for use in core internet routers,
that is resilient to network flaps

Necessary move from demand cashe based


forwarding otherwise high bandwidth/PPS
speed would not be achieved.

CEF has had a lot a teething issues along the


way. Yet, other companies are moving down the
same path.

2001, Cisco Systems, Inc.

55

What Is CEF?

This is a simple operational taste of CEF.


Check out the detailed sessions:

3302
1300_05_2000_c2

PS-540 - Router and Switch Internal Architecture


and Operation

PS-201 - Router Internals and IOS Operations

2001, Cisco Systems, Inc.

56

New Terminology
Routing Information Base (RIB)
Generated

by each routing protocol

Forwarding Information Base (FIB)


Network
New

layer routing information

Term used to describe the Forwarding Table

Adjacency Table (Adj)

Next hop link layer information

Distributed FIB
FIB

push out to the Line Cards on a router so that


forwarding can be done locally on each line card.
3302
1300_05_2000_c2

2001, Cisco Systems, Inc.

57

dFIB
on LC
dFIB
on LC
dFIB
on LC
dFIB
on LC
dFIB

Forward Information Base on RP

Routing Tables (RIB) Feeds


the Forwarding Table (FIB)
BGP 4 Routing Table
(RIB)

OSPF - Link State Database


(RIB)
Static
Routes
Connected
Interfaces

on LC
3302
1300_05_2000_c2

2001, Cisco Systems, Inc.

58

FIBs MTRIE Data Structure


M-node

ROOT

Child Link
10.0.0.0
Leaf

54.0.0.0

10.1.0.0

10.10.0.0

54.10.0.0

10.1.1.0
Leaf

10.10.5.0
Leaf

54.10.1.0
Leaf

10.1.1.1
Leaf

192.0.0.0

192.5.0.0
Leaf

192.8.0.0

54.10.4.0

192.8.2.0

Leaf

Leaf
192.8.2.0
Leaf

192.8.2.128
Leaf

m-node is an internal node containing an array of M child


links.
A child link points to another m-node, a leaf (FIB), or Null.
A leaf (cached), bottom of tree, points to a FIB.
3302
1300_05_2000_c2

2001, Cisco Systems, Inc.

59

MTRIE Structures

8-8-8-8 used by generic IOS


16-8-8 used by the GSR Engine 4
10-9-5-8 used by the ESR (Omega)
11-8-5-8 used by ESR (Pulsar)

3302
1300_05_2000_c2

2001, Cisco Systems, Inc.

60

MTRIE Data Structure Effect


PPS Performance
Route Lookup Performance

Percent of Line Rate for 64 byte packets

90.0%

8-1-1-1-1-1-1...
16-8-8

80.0%

70.0%

60.0%

50.0%

40.0%

30.0%

Vendors
Vendors
Marketing
Marketing
Collateral
Collateral

20.0%

The
Internet
The Internet
Today
Today

10.0%

0.0%
13
3302
1300_05_2000_c2

14

2001, Cisco Systems, Inc.

15

16

17

18

19

20

21

22

23

24

Prefix Length (bits)

25

26

27

28

29

30
61

CEF Defaults in 12.0S


On this platform...

The default is...

Cisco 7000 series equipped with


RSP7000
Cisco 7200 series

CEF is not enabled.

Cisco 7500 series

CEF is enabled.

Cisco 12000 series Gigabit Switch


Router

Distributed CEF is enabled.

CEF is not enabled.

7500 with VIP2/4 Cards should have ip cef


distributed turned on!
7200 should have ip cef turned on!
Remember the memory requirements on the
line/VIP cards
3302
1300_05_2000_c2

2001, Cisco Systems, Inc.

62

CEF Based Features


CEF Based Features are defined as
functions that use the FIBs MTRIE as a
core foundation of their function.

Stores information in the the FIBs Leaf.

Why use store it in the FIB?

3302
1300_05_2000_c2

Consistent look-ups (4 steps in a 8-8-8-8


MTRIE)

Per prefix information.

Update of the information via a routing


protocol.

2001, Cisco Systems, Inc.

63

FIB Entries
ROOT

10.0.0.0

10.1.0.0

10.10.0.0

10.1.1.0

10.10.5.0

FIB Entries or (also called items added


to the struct fibtype_ ; in the leaf)
are used to distribute policy through
the network via BGP and the table-map
command. They are used for Security,
Accounting, QOS, and any other
service/feature the customer dreams
up. The values are obtained via CEFs
MTRE look-up.

Current FIB Entries added to the Leaf (not in all hardware):


10.1.1.1

Precedence = Values 0-7 for use in QOS Features


QOS-Group = Values 0-99 for use in QOS Features
WCCP-Tag = Values 0-99 for use with WCCP
struct fibtype_ :
Precedence
QOS_Group
WCCP_TAG

Traffic-Index = Values 0-7 for use in with BGP Policy Accounting

Traffic_Index

3302
1300_05_2000_c2

2001, Cisco Systems, Inc.

64

Fib Entries in the Leaf


struct fibtype_ {
mtrie_leaf
adjacency

mtrie_info;
*fastadj;

/* Mtrie crap */
/* Cache adj when no load sharing */

loadinfotype *loadinfo;

/* Load sharing information */

ulonglong

packets;

/* Packets switched */

ulonglong

bytes;

/* Bytes switched */

void

*fasttag_rew;

/* tag rewrite when no load sharing */

ushort

origin_as;

/* Autonomous System */

uchar

mask_bits;

/* Number of bits on in the net mask */

uchar

precedence;

/* precedence for pkts to this dest */

uchar

flags;

/* see below */

uchar

next_index;

/* index of next path to use */

uchar

count;

/* number of paths */

uchar

qos_group;

/* qos group for pkts to this source or dest */

tag_info
ulong
uchar

*tag_info;
version;

/* Tag information for this prefix */


/* FIB entry version number */

*hwleaf;

uchar

wccp_tag;

/* WCCP service */

uchar

traffic_index;

/* Traffic Group for acct purposes */

uchar

dummy1;

fib_path

path[0];

/* Possible paths */

};
3302
1300_05_2000_c2

2001, Cisco Systems, Inc.

65

dFIB
on LC
dFIB
on LC
dFIB
on LC
dFIB
on LC
dFIB

Forward Information Base on RP

Routing Tables (RIB) Feeds


the Forwarding Table (FIB)
BGP 4 Routing Table
(RIB)

OSPF - Link State Database


(RIB)
Static
Routes
Connected
Interfaces

on LC
3302
1300_05_2000_c2

2001, Cisco Systems, Inc.

66

Routing Tables (RIB) Feeds


the Forwarding Table (FIB)
ROOT

10.0.0.0

10.1.0.0

10.1.1.0

BGP 4 Routing Table


(RIB)

10.10.0.0

10.10.5.0

Table-map
command
updates FIB
Entry

Route
Map set
value

10.1.1.1

struct fibtype_ :
Precedence
QOS_Group
WCCP_TAG
Traffic_Index

3302
1300_05_2000_c2

2001, Cisco Systems, Inc.

67

What Problem are We


Solving?
Peer A
IXP-W

A
Peer B
IXP-E

Upstream A
Upstream A

Upstream B

Upstream B

Target

G
F
3302
1300_05_2000_c2

2001, Cisco Systems, Inc.

NOC

Central
Device uses a
network
protocol to
distribute
policy across
the network.

POP

68

FIB Entry Based Features


Today
CAR with QOS_ID
Marketing

name is Quality Policy


Propagation with BGP

WCCPv2 with WCCP_Tag


BGP Policy Accounting with
Traffic_Index
3302
1300_05_2000_c2

2001, Cisco Systems, Inc.

69

CEF Load
-Sharing
Load-Sharing
Destination

Sources

Per packet and enhanced per destination


Enhanced per destination is based on source and
destination IP addresses
Each destination flow takes a single, separate path
Reduces need for per packet load-sharing
3302
1300_05_2000_c2

2001, Cisco Systems, Inc.

70

CEF Accounting
C
A

AS 100

DMZ
Network

Per prefix
Per adjacency
Per DMZ nexthop
accounting
3302
1300_05_2000_c2

2001, Cisco Systems, Inc.

F
AS 101
D

E
AS 102

71

Netflow
Providers network administrators with
packet flow information
Allows:

Security monitoring

Network management and planning

Customer billing

Traffic flow analysis

Available from 11.1CC for 7x00 and 12.0


for remaining router platforms
3302
1300_05_2000_c2

2001, Cisco Systems, Inc.

72

NetFlow Infrastructure

Network Planning
RMON Probe

Accounting/Billing

NetFlow
FlowCollector:
NetFlow
Accounting:

Data Collection

Data Switching

Data Aggregation

Data Export

Data Storage

Data Aggregation

File System Management

3302
1300_05_2000_c2

2001, Cisco Systems, Inc.

Data Filtering

Network Data Analyzer:


Data Presentation
NFC Control and Configuration

Partner Applications
73

Netflow - Capacity Planning


Public Routers 1 , 2, 3 Month of September Outbound Traffic
1% 1% 1% 1% 1% 1%
1% 1%1%
1%

2%
4%

32%
6%

8%

8%
20%

10%

WEC

WebTV

ABSNET

AOL

Compuserve

SURAnet

IBM

OARNet

NIH

PacBell Internet Service

JHU

C&W

UMD

AT&T

BBN

Erols

Digex

Other

3302
1300_05_2000_c2

2001, Cisco Systems, Inc.

74

NetFlow Data Record (V5)


From/To

Usage

Packet Count
Byte Count

Source IP Address
Destination IP Address

Time
of Day

Start Timestamp
End Timestamp

Source TCP/UDP Port


Destination TCP/UDP Port

Port
Utilization

Input Interface Port


Output Interface Port

QoS

Type of Service
TCP Flags
Protocol

Time
Stamp

Start Timestamp
End Timestamp
Call Duration

Available via Netflow only

3302
1300_05_2000_c2

2001, Cisco Systems, Inc.

Application

Next Hop Address


Source AS Number
Dest. AS Number
Source Prefix Mask
Dest. Prefix Mask

Routing
and
Peering

Next Hop Address


Lost Datagrams
Also available via RMON
75

Netflow format variations


Version 7 - Cat6k only
In

connection with MultiLayer switching (MLS)

Version 8, the aggregated version


For

reduction of data export from the router:

ProtocolPort

, AS
DestinationPrefix, Prefix

,SourcePrefix,
,TOS New

Sampled GSR only


For

speeds higher than OC-3 strongly


recommended
3302
1300_05_2000_c2

2001, Cisco Systems, Inc.

76

The Switching Path


Switching Vector

Flow entry

Engine Feature check

Creation

CEF

IP packet

CEF+FLOW

Pkt
Buffer

CEF+Features

Flow
Lookup

Early

Late

Feature

Feature

Lookup
ACL

FIB

Lookup
Qos

Policy

CAR

CEF+VPN+

WCCP

Crypto

FLOW

etc.

Output

FAST
FAST+FLOW
Packet Reference

3302
1300_05_2000_c2

2001, Cisco Systems, Inc.

77

Typical Netflow Deployment


Network Core
GSR

Edge Aggregation
7500/7200/6509

NFC

Interface
To apps

Billing
Traffic engineering

Access Devices
Head End, MUX,
DSL/Wireless/Cable
3302
1300_05_2000_c2

2001, Cisco Systems, Inc.

78

NetFlow Platform Support


Cisco IOS Software Supported NetFlow
Release Version
Export Version(s)
11.1CA, 11.1CC
v1, v5
11.2, 11.2P
v1
11.2P
v1
11.3, 11.3T
v1
12.0
v1, v5
12.0T
12.0S
12.0(3)T and later
12.0(3)S and later

v1, v5

12.04XE
N/A

v1, v5, v8
v7

12.0(6)S

v8

v1, v5, v8

Supported Cisco Hardware Platforms


7200, 7500, RSP7000
7200, 7500, RSP7000
Route Switch Module (RSM), 11.2(10)P and later
7200, 7500, RSP7000
1720, 2600, 3600, 4500, 4700, AS5800,
7200, uBR7200, 7500, RSP7000, RSM
1720, 2600, 3600, 4500, 4700, AS5800, 7200, uBR7200,
7500, RSP7000, RSM, MGX 8800 RPM, BPX 8600
1400*, 1600*, 1720, 2500*,2600, 3600, 4500, 4700,
AS5800, AS5300**, 7200, uBR7200, 7500, RSP7000,
RSM, MGX8800 RPM, BPX 8650
7100
Catalyst 5K NetFlow Feature Card (NFFC)
Catalyst 6K with MSFC card
12000

*Support for NetFlow Export v1, v5, and v8 on 1600 and


2500 platforms is targeted for Cisco IOS software release
12.0(5)T. NetFlow support for these platforms will not be
available in the Cisco IOS 12.0 mainline release.
3302
1300_05_2000_c2

2001, Cisco Systems, Inc.

**Support for NetFlow Export v1, v5, and v8 on


AS5300 platform is targeted for Cisco IOS
software release 12.0(7)XR.

79

Netflow
Configuration example:
interface serial 5/0
ip route-cache flow

If CEF not configured, Netflow enhances


existing switching path (i.e. optimum
switching)
If CEF configured, Netflow becomes a flow
information gatherer and feature
acceleration tool
3302
1300_05_2000_c2

2001, Cisco Systems, Inc.

80

Netflow
Information export:

router to collector system

ip flow-export version 5 [origin-as|peer-as]


ip flow-export destination x.x.x.x <udp-port>

Flow aggregation (new in 12.0S):

router sends aggregate records to collector


system

ip flow-aggregation cache as|prefix|dest|source|proto


enabled
export destination x.x.x.x <udp-port>
3302
1300_05_2000_c2

2001, Cisco Systems, Inc.

81

Aggregation Schemes
Netflow Main Cache(128k RAM)
Flow Entries

Flow Expired
Cache Full
Timer expired
Export buffer

UDP

Agg Scheme Cache


AS- Matrix
Prefix-Matrix

Router
Based
Aggregation
Enabled
3302
1300_05_2000_c2

2001, Cisco Systems, Inc.

Dst- Prefix
Src-Prefix
Port-Protocol
82

Netflow - Simple Traffic


Engineering
Sample Output on router:
Beta-7200-2>sh ip cache flow
IP packet size distribution (17093 total packets):
1-32
64
96 128 160 192 224 256 288 320 352 384 416 448 480
.000 .735 .088 .054 .000 .000 .008 .046 .054 .000 .009 .000 .000 .000 .000
512 544 576 1024 1536 2048 2560 3072 3584 4096 4608
.000 .000 .000 .000 .000 .000 .000 .000 .000 .000 .000
IP Flow Switching Cache, 1257536 bytes
3 active, 15549 inactive, 12992 added
210043 ager polls, 0 flow alloc failures
last clearing of statistics never
Protocol
Total
Flows
Packets Bytes
-------Flows
/Sec
/Flow /Pkt
TCP-Telnet
35
0.0
80
41
UDP-DNS
20
0.0
1
67
UDP-NTP
1223
0.0
1
76
UDP-other
11709
0.0
1
87
ICMP
2
0.0
1
56
Total:
12989
0.0
1
78
SrcIf
Et1/1
Et1/1
Et1/1

3302
1300_05_2000_c2

SrcIPaddress
144.254.153.10
144.254.153.112
144.254.153.50

2001, Cisco Systems, Inc.

DstIf
Null
Null
Local

Packets Active(Sec) Idle(Sec)


/Sec
/Flow
/Flow
0.0
14.5
12.7
0.0
0.0
15.3
0.0
0.0
15.5
0.0
0.1
15.5
0.0
0.0
15.2
0.0
0.1
15.4

DstIPaddress
144.254.153.127
255.255.255.255
144.254.153.51

Pr
11
11
06

SrcP
008A
0208
701D

DstP
008A
0208
0017

Pkts
1
1
63
83

Netflow Feature Acceleration


NetFlow Accelerates

NetFlow Policy Routing


(NPR)

Network Address Translation


(NAT)

Router-based network data


encryption

Committed Access Rate (CAR)

Web Cache Control Protocol


(WCCP)

MultiNode Load Balancing


(MNLB) (not in 12.0S)

Access Control Lists (ACL)

RSVP

IP Accounting

Availability of such acceleration will be


announced on a feature-by-feature basis
ip flow-cache feature-accelerate
3302
1300_05_2000_c2

2001, Cisco Systems, Inc.

84

IP Switching Path - Hidden


Commands

show interface switching


show interface <interface> switching
show interface stat
show interface <interface> stat

3302
1300_05_2000_c2

2001, Cisco Systems, Inc.

85

Using DNS
Map names to addresses
Descriptive names
ip domain-name
ip name-server

Sample trace through network:


4:Received echo from sj-wall-2.cisco.com [198.92.1.138] in 440 msec.
5:Received echo from barrnet-gw.cisco.com [192.31.7.37] in 335 msec.
6:Received echo from paloalto-cr1.bbnplanet.net [131.119.26.9] in 335 msec.
7:Received echo from paloalto-br2.bbnplanet.net [131.119.0.194] in 327 msec.
8:Received echo from core6-hssi6-0.SanFrancisco.mci.net [206.157.77.21] in 468 msec.
9:Received echo from bordercore1-loopback.Washington.mci.net [166.48.36.1] in 454 msec.
10:Received 48 bytes from www.getit.org [199.233.200.55] in 466 msec
3302
1300_05_2000_c2

2001, Cisco Systems, Inc.

86

Turn on Nagle

Telnet was designed to do one character,


one packet dialog.
John Nagle's algorithm (RFC 896) helps
alleviate the small-packet problem in TCP.
service nagle

Lessens the load on the CPU when using


show XXXX commands
3302
1300_05_2000_c2

2001, Cisco Systems, Inc.

87

IP MAC accounting
Calculate total packet counts and byte
counts for a LAN interface which
receives/sends IP packets from/to each
unique MAC address
Record a timestamp for the last packet
received/sent for each unique MAC
address
Available only on ethernet,
FastEthernet and FDDI
Available from 11.1(19)CC

3302
1300_05_2000_c2

2001, Cisco Systems, Inc.

88

IP MAC Accounting
Use command ip accounting mac
{input | output} to enable
show interface <interface> mac
Example:
Ethernet0/1/3
Input (511 free)
0000.0c04.7ad5(167): 9 packets, 1026 bytes, last: 20512ms ago
Total: 9 packets, 1026 bytes
Output (510 free)
ffff.ffff.ffff(0 ): 16 packets, 960 bytes, last: 58108ms ago
0000.0c04.7ad5(167): 9 packets, 1026 bytes, last: 21060ms ago
Total: 25 packets, 1986 bytes
3302
1300_05_2000_c2

2001, Cisco Systems, Inc.

89

IP MAC accounting - the fine


print

Fast Ether Channel supported


512 mac address per interface per
direction(input or output)
Support fast/optimum/flow/CEF
switching

3302
1300_05_2000_c2

2001, Cisco Systems, Inc.

90

IP Precedence Accounting
Calculate the total packet counts and byte
counts for an interface which
receives/sends IP packets, and sorts out the
results based on different IP precedence
8 precedence levels
Supported on any interface and subinterface
Switching mode supported:
CEF/DCEF/Flow/Optimum
3302
1300_05_2000_c2

2001, Cisco Systems, Inc.

91

IP Precedence Accounting
Use command ip accounting precedence
{input | output} to enable
show interface <interface> precedence
Example:
Ethernet0/1/3
Input
Precedence 0: 9 packets, 1026 bytes
Output
Precedence 0: 9 packets, 1026 bytes
Precedence 6: 16 packets, 960 bytes
3302
1300_05_2000_c2

2001, Cisco Systems, Inc.

92

Command Summary

Global Commands

Interface Commands

ip cef (-distributed)

description

ip cef accounting [perprefix] [non-recursive]

bandwidth

ip flow-cache featureaccelerate

ip load-sharing [perpacket] [per-destination]

ip domain-name

ip route-cache flow

ip name-server
service nagle

3302
1300_05_2000_c2

2001, Cisco Systems, Inc.

93