Q&A

DYNAMIC MULTIPOINT VPN

Last Updated: July 2005

INTRODUCTION TO DYNAMIC MULTIPOINT VPN: POSITIONING AND ROADMAP

This section introduces Dynamic Multipoint VPN (DMVPN) and discusses positioning and advantages of DMVPN over those of native IPsec,
Generic Routing Encapsulation (GRE), easy VPN, the roadmap, and a description of future DMVPN projects.

Introduction to DMVPN
What is DMVPN?
DMVPN provides an easy and scalable way to create large and small IPsec VPNs by combining GRE tunnels, IPsec encryption, and Next Hop
Resolution Protocol (NHRP).

NHRP
In NHRP, the hub maintains a database of all spokes’ real (publicly reachable) Internet addresses. Spokes in a DMVPN network register their real IP
address with the hub during boot session. Spokes query the NHRP database on the hub for the real address of destination spokes when building
direct tunnels to them.

Multipoint Generic Routing Encapsulation

A multipoint GRE (mGRE) tunnel interface allows a single GRE tunnel to support multiple IPsec tunnels. This results in simplifying the size and
complexity of configuration.

What are the types of DMVPN deployments?

DMVPN can be configured in two ways. It can be configured as a DMVPN hub-to-spoke deployment or as DMVPN virtual full mesh with
spoke-to-spoke deployment. The differences between the two are configuration changes required at the hub and spoke.

Benefits of DMVPN Hub to Spoke

• Simplified and smaller configurations for hub and spoke
• Zero-touch provisioning for adding new spokes to the VPN
• Support for dynamically addressed spokes
• Support for multicast traffic from hub to spokes

Benefits of DMVPN Spoke to Spoke

Everything from DMVPN hub and spoke, plus:

• Direct dynamic spoke-to-spoke tunnels

• Ability of smaller spokes to participate in the virtual full mesh

All contents are Copyright © 1992–2005 Cisco Systems, Inc. All rights reserved. Important Notices and Privacy Statement.
Page 1 of 30
 When would a DMVPN hub-to-spoke or spoke-to-spoke topology be deployed?
DMVPN was designed to follow the 80:20 rule, where eighty percent of the traffic would go through the hub, and twenty percent of the traffic
would go through the spokes.

DMVPN hub-to-spoke network topology is usually preferred to DMVPN spoke-to-spoke topology for networks with a high volume of voice and
multicast traffic.

If a customer wants to deploy DMVPN architectures where 100 percent of the traffic will be spoke-to-spoke with voice and multicast requirements,
please contact the DMVPN core (dmvpn-core@cisco.com) for more detailed network design information and guidance.

For additional information, please visit:

• DMVPN topologies and configurations

http://www.cisco.com/go/dmvpn/

• DMVPN Design Guide

http://www.cisco.com/warp/public/779/largeent/it/ese/DMVPN_bk.pdf

DMVPN versus Other IPsec Solutions

What are the differences between DMVPN and native IPsec?
DMVPN has the following advantages over native IPsec:

• DMVPN avoids the need for crypto maps on the physical interface.
• Adding new spokes in a DMVPN network does not require any extra configuration on the hub side, making the maintenance much easier.
• DMVPN provides routing-based failover. This makes configuration of redundant hubs easier.
• DMVPN makes it easier to configure split tunneling or nonsplit tunneling. A configuration change on the hub can change the split-tunneling
behavior. In IPsec, all the spokes will have to be modified.
• DMVPN has dynamic spoke-to-spoke connectivity support. Plain IPsec needs additional configuration in the spoke to support spoke-to-spoke
connectivity and also needs static IP addresses.
• Native IPsec supports only IP Unicast traffic. Use a GRE tunnel architecture to support multicast and dynamic routing.
What are the differences between DMVPN and IPsec plus GRE?
IPsec and GRE have the following limitations with respect to DMVPN:

• IPsec uses an Access Control List (ACL) to define what data is to be encrypted. Each time a new subnetwork is added behind a spoke or the hub,
the customer must change the ACL on both the hub and spoke routers. If the service provider manages the router, the customer must notify the
service provider in order to get the IPsec ACL changed and the new traffic will be encrypted.
• With large hub-and-spoke networks, the size of the configuration on the hub router can become very large, to the extent that it is unusable. For
example, a hub router would need up to 3900 lines of configuration to support 300 spoke routers. This is large enough that it would be difficult to
show the configuration and to find the section of the configuration that is relevant to a current problem that is being debugged. Also, this size
configuration may be too large to fit in NVRAM and would need to be stored on Flash memory.
• GRE plus IPsec must know the endpoint peer address. The spokes’ IP addresses are connected directly to the Internet using their own ISP, and
they are often set up so that their external interface addresses are not fixed. The IP addresses can change each time the site comes online (using
Dynamic Host Configuration Protocol [DHCP]).
• If the spokes need to communicate with each other directly over the IPsec VPN, the hub-and-spoke network must become a full mesh. Since it is
not already known which spokes will need to talk directly with each other, a full mesh is required, even though each spoke may not need to talk
directly with every other spoke. Also, it is not feasible to configure IPsec on a small spoke router so that it has direct connectivity with all other
spoke routers in the network; spoke routers may need to be more powerful routers.

© 2005 Cisco Systems, Inc. All rights reserved.

Important notices, privacy statements, and trademarks of Cisco Systems, Inc. can be found on cisco.com.
Page 2 of 31
 How does DMVPN resolve the problems described in the previous question?
Configuration is simple and scales well to large networks. The user does not need to touch the existing DMVPN network to add extra spokes,
because they can be dynamically added without changes to hubs and other spokes.

DMVPN uses mGRE on the hub router, because mGRE does not require a tunnel destination to be configured, but mGRE cannot work alone. NHRP
is used to find the mapping between the IP next hop out the tunnel interface and the tunnel destination to get to that IP next hop (spoke). In order to
start this system, the spoke routers are configured with the NHRP mapping for one or more hub routers. When the spoke routers boot, they build the
GRE plus IPsec tunnel to the hub routers. At this point the spoke router will send an NHRP registration packet (for example, a Frame Relay inverse
Address Resolution Protocol [ARP]) with its own mapping to the hub routers. After the hub receives NHRP registrations from the spokes, it can send
packets back to the spoke router. At this point the hub and spoke become routing neighbors and pass routing information to each other. Since GRE is
now clear, IPsec is used to protect the content of GRE. Tunnel protection has been introduced as part of DMVPN, and the tunnel profile serves the
same functionality as a crypto map. IPsec applies the tunnel profile on the tunnel interface, and Cisco IOS® Software treats this internally as a
crypto map.

What are the differences between DMVPN and Easy VPN?

Cisco® Easy VPN, a software enhancement for existing Cisco Systems® routers and security appliances, greatly simplifies VPN deployment for
remote offices and teleworkers based on the Cisco Unified Client Framework. Cisco Easy VPN centralizes VPN management across all Cisco VPN
devices, reducing the complexity of VPN deployments. Cisco Easy VPN enables an integration of VPN remotes—Cisco routers, Cisco PIX®
firewalls, and Cisco VPN concentrators or software clients—within a single deployment with a consistent policy and key management method,
simplifying remote side administration.

Table 1 indicates the differences between DMVPN and Easy VPN.

Table 1. DMVPN and Easy VPN Comparison

Service/Feature Name DMVPN Easy VPN

Support for Multicast Traffic Yes –
Spoke-to-Spoke Communication Yes –
Support for GRE/Quality of Service (QoS) Yes –
Support for Routing Protocols Yes –
Support for Certificates Yes –
Stateful Failover Depends on routing protocol for recovery Yes
Scalability per Hub Because of routing protocols, DMVPN Large number of spokes can be
hubs support fewer spokes per hub supported per hub
Identical Configuration for all Spokes – Yes
Cross-Platform Support – Yes
Support for Software/Hardware Client Hardware client support only Yes
Always up Tunnel to Hub Yes Not required

Under what circumstances should DMVPN be deployed?

DMVPN should be deployed if the network requires any of the following:

• Multicast
• GRE
• Routing protocols
• Simplified configurations

© 2005 Cisco Systems, Inc. All rights reserved.

Important notices, privacy statements, and trademarks of Cisco Systems, Inc. can be found on cisco.com.
Page 3 of 31
• Zero-touch provisioning
• Support for dynamically addressed spokes

If a physical interface needs to be kept free for other applications (for example, router management), a hub-to-spoke DMVPN deployment is
recommended.

If the customer is interested in all items in the preceding list, as well as direct (not through hub) spoke-to-spoke communication, then a spoke-to-
spoke DMVPN deployment is recommended.

Between the two, it is recommended that DMVPN networks be deployed with hub-to-spoke topologies first and then follow a migration path to a
DMVPN spoke-to-spoke topology, if spoke-to-spoke voice and multicast requirements exist.

When should an alternate Cisco VPN solution be recommended?

If a customer is interested in a VPN for basic connectivity without any of the features listed in the preceding section, then one of the following
would be appropriate:

• Easy VPN
• Plain IPsec
If a customer is interested in multicast and routing protocols, recommended deployments are:

• DMVPN
• IPsec with GRE (differences between DMVPN and IPsec with GRE have been outlined in the previous section)

What are the options for deploying DMVPN hubs and the typical size of installations they support?
The largest currently deployed customer base for DMVPN is 3000 spokes as of June 21, 2005. Cisco is working on customer deployments for
networks with more than 10,000 spokes.

The following devices are recommended for DMVPN headend devices. Details are included in the following sections.

• Cisco 7200 VXR router with VPN Acceleration Module (VAM) and VAM2 cards or Cisco 3700 Series multiservice access routers with Advanced
Integration Module (AIM) and AIM HP II cards
• Cisco Catalyst® 6500 Series Switches and Cisco 7600 Series Routers with Cisco Multiprocessor WAN Application Module (MWAM) card and
VPN Services Module (VPNSM) installed for aggregating large number of spokes to a single hub
• Cisco Catalyst 6500 Series and Cisco 7600 Series with Cisco Catalyst 6500 Series Supervisor Engine 720 and the VPNSM for high throughput

Cisco 7200 VXR Router with VAM2 Cards and Cisco 3700 Series with AIM and AIM HP II Cards
Cisco 7200 Series Routers will scale up to 700 spokes, with 350 spokes per mGRE interface using Extended Interior Gateway Routing Protocol
(EIGRP). The size of installations can be increased by having multiple hubs. Cisco 7206VXR Routers are very stable, with 700 spokes with two
mGRE interfaces being used by Cisco IT for the internal Enterprise-Class Teleworker deployment (see section 3 for additional details). The
performance of these routers with DMVPN has been demonstrated to be better than IPsec point-to-point GRE in hub-and-spoke mode.

Table 2, taken from the DMVPN design guide (link included below), shows the comparison chart for DMVPN and IPsec with point-to-point GRE
with voice calls.

Table 2. DMVPN Compared to Point-to-Point GRE in Hub-and-Spoke Mode

Bi-directional Bi-directional
Spokes # of Voice Calls Traffic (Mbps) Traffic (kPPS) CPU Utilization %
Cisco 3745 Multiservice Access 60 60 17.6 8.4 72

© 2005 Cisco Systems, Inc. All rights reserved.

Important notices, privacy statements, and trademarks of Cisco Systems, Inc. can be found on cisco.com.
Page 4 of 31
 Router (AIM-II) P2P GRE
Cisco 3745 Multiservice Access 65 65 24.36 10.81 44
Router(AIM-II) DMVPN
Cisco 7200 Series NPE-G1 P2P 500 240 78.1 36.7 80
GRE
Cisco 7200 Series NPE-G1 400 250 104.3 45.2 82
DMVPN

Additional information:

• Detailed scalability numbers for all Cisco 7200 Series and Cisco Access Routers:
http://www.cisco.com/warp/public/779/largeent/it/ese/DMVPN_bk.pdf

Cisco Catalyst 6500 Series Switch with MWAM and VPNSM Card
The Cisco Catalyst 6500 Series with the Cisco MWAM Card and VPNSM installed are recommended for aggregating large numbers of spokes to
a single hub.

The MWAM card runs multiple installations of Cisco IOS Software. The MWAM essentially acts as six independent NPE-G1 Cisco IOS Software
Routers, all located on one card, which can be used in the Cisco Catalyst 6500 Series or the Cisco 7600 Series. When the mGRE tunnels are
terminated on the MWAM processors, it is easy to create a very scalable DMVPN design. The MWAM performs the NHRP resolution and the
GRE functionality, leaving most of the encryption to the VPNSM.

The VPNSM is a Gigabit Ethernet IPsec cryptographic module that is installed in the Cisco Catalyst 6500 Series Switch and Cisco 7600 Series
Router. Configuring VPNs using the VPNSM is similar to configuring VPNs on routers running Cisco IOS Software. When configuring VPNs with
the VPNSM, crypto maps are attached to VLANs (using interface VLANs); when VPNs are configured on routers running Cisco IOS Software,
crypto maps are attached to individual interfaces.

Unlike modules for existing router platforms, the VPNSM performs the crypto ACL lookup processing and all crypto key swapping (moving the
relevant key material for a given packet into use for encrypting or decrypting that packet) in Ternary Content Addressable Memory (TCAM)
hardware. The router platforms perform these functions in their main CPUs, giving a design using the VPNSM, a definitive advantage in
performance.

Figure 1 is a high-level diagram of this deployment option.

© 2005 Cisco Systems, Inc. All rights reserved.

Important notices, privacy statements, and trademarks of Cisco Systems, Inc. can be found on cisco.com.
Page 5 of 31
Figure 1. GRE High-Concentration VPN Design

The detailed packet flow is further explained in “DMVPN Support, Scalability and Management.” Detailed design information concerning DMVPN
designs with the MWAM access module can be accessed at DMVPN on CAT6K-MWAM and Native Modes white paper in the technical documents
section at http://www.cisco.com/go/dmvpn.

Cisco Catalyst 6500 Series and Cisco 7600 Series Router with Cisco Catalyst 6500 Series Supervisor Engine 720
and VPNSM
The MWAM is typically used for aggregating a high volume of spokes into a headend device. The Cisco Catalyst 6500 Series Switch with native
Cisco Catalyst 6500 Series Switch Supervisor Engine 720 or the Cisco Catalyst 6500 Series Supervisor Engine 2 and VPNSM can be used for a
high-bandwidth spoke or a hub device. The number of spokes supported by the Cisco Catalyst 6500 Series and Cisco 7600 Series Router running in
native supervisor mode is limited by the routing protocol, and the number of spokes is approximately 350 spokes per mGRE interface with a total of
700 spokes. Figure 2 is a high-level deployment diagram for this option.

© 2005 Cisco Systems, Inc. All rights reserved.

Important notices, privacy statements, and trademarks of Cisco Systems, Inc. can be found on cisco.com.
Page 6 of 31
Figure 2. DMVPN Deployment with Cisco Catalyst 6500 Series and Cisco Catalyst 6500 Series Supervisor Engine 720 with VPSNSM

DMVPN Roadmap and DMVPN Phase 3

What is DMVPN Phase 3?
DMVPN Phase 3 is a set of features that will improve scalability, stability, and manageability of DMVPN networks. They are:

• Increased DMVPN cloud sizes with routing and scalability enhancements

• Enhanced QoS for traffic shaping per spoke
• Better support for multicast
• Better management support

These features can take DMVPN to the next level. A detailed description of the projects have been given in subsequent sections.

DMVPN Phase 3 is not currently available. For exact availability dates for these features, please contact dmvpn-core@cisco.com.

What is the scope of the Routing and Scalability Enhancements Project?

Current DMVPN networks have some limitations with routing and scalability:

• The routing protocol must be configured; it will advertise all individual specific routes for networks behind the spoke routers. The route
summarization cannot be done on the hub for routes advertised to the spokes.
• When using Open Shortest Path First (OSPF) as the routing protocol, only two hub routers can be supported, and all spokes must be dual
connected to both hub routers. This reduces the scalability of DMVPN spoke-to-spoke when using OSPF.
• The hub routers must be connected to each other (over DMVPN) in a loop or “daisy chain,” and NHRP mapping resolutions will traverse the
whole chain. This tends to limit the number of hub routers.

In DMVPN Phase 3, all of the preceding limitations will be removed. It will rewrite the interaction of NHRP, Cisco Express Forwarding switching,
and the routing table to allow:

• Route summarization for reduced routing load and routing table size on spoke routers.
• Allowing for more than two hub routers for DMVPN spoke-to-spoke communication when using the OSPF routing protocol. (Limitations exist
only for current DMVPN spoke-to-spoke networks.)

© 2005 Cisco Systems, Inc. All rights reserved.

Important notices, privacy statements, and trademarks of Cisco Systems, Inc. can be found on cisco.com.
Page 7 of 31
• Removing the requirement that the hub routers must be connected in a loop (daisy chain), allowing more scalable hub router interconnections such
as hierarchical tree structures. (Limitation exists only for current DMVPN spoke-to-spoke networks.)

The following diagrams help illustrate what the DMVPN Phase 3 enhancements will be.

Route Summarization for Reduced Routing Load and Routing Table Sizes on Spoke Routers
In a DMVPN network, the hub maintains an NHRP mapping database of all the spoke’s tunnel to real (public interface) addresses. Each spoke
registers its real address when it boots. Spokes query the NHRP database for real addresses of destination spokes to build direct tunnels.

The current model for NHRP to interact with Cisco Express Forwarding is for NHRP to have ARP-like functionality only. This requires the routing
table to contain unique specific subnetwork entries with an IP next hop for each tunnel IP address of the remote spoke routers on the DMVPN. This
means every spoke router in the DMVPN will need to have a subnetwork route for every subnetwork behind every other DMVPN spoke router. As
the size of a DMVPN network grows, the number of routes grows, this may lead to a large number of routes and scalability problems from small
spoke routers.

In Figure 3, the current model of operation requires all spokes to maintain the subnetwork information of all the other spokes. As the size of DMVPN
networks increases, the routing tables in the spokes continue to increase in size. Also, the hub would need to propagate all the network and tunnel
address pairs for all the spokes without summarization. Currently routes cannot be summarized, because the unique IP next hop on the routes on the
spoke router is the trigger information for building the dynamic spoke-to-spoke tunnels.

A small spoke, for example a Cisco 831 Ethernet Broadband Router in a 300-node DMVPN network, would have routing information for all the
subnetworks behind all 300 spokes even if it would not have use for all of them. It is prefered to summarize routes and follow the query model of
NHRP to request routes for other spokes as needed, prior to building the tunnel and populate the subnet/tunnel mappings on demand.

© 2005 Cisco Systems, Inc. All rights reserved.

Important notices, privacy statements, and trademarks of Cisco Systems, Inc. can be found on cisco.com.
Page 8 of 31
Figure 3. Present-Day DMVPN Networks Without Summarization

Table 3 shows the topology from Figure 3.

Table 3. Tunnel Address, Physical Address and Network Mapping

Node Physical Address Tunnel Address Network

Hub 1 PH1 TH1 NO
Hub 2 PH2 TH2 NO
Hub 3 PH3 TH3 NO
Spoke 1 PS1 TS1 N1
Spoke 2 PS2 TS2 N2
Spoke 3 PS3 TS3 N3

In Figure 3, the spokes have subnetwork entries to every spoke. This is the behavior of DMVPN networks today. After DMVPN Phase 3, the routing
updates at the spokes will be summarized as shown in Figure 4. In this case the subnetwork routing entries will have an IP next hop of the NHRP
hub router(s), and NHRP will be triggered by “snooping” on the destination IP address of packets that Cisco Express Forwarding is switching to
search for a shorter route. Also, in this case, we can use summary routes instead of needing explicit subnetwork routes. If/when NHRP finds a shorter
route, NHRP will install a specific subnetwork route in the routing table with an IP next hop of the remote spoke router behind where that
subnetwork resides. Cisco Express Forwarding will then automatically pick up that route and install a corresponding Forwarding Information Base

© 2005 Cisco Systems, Inc. All rights reserved.

Important notices, privacy statements, and trademarks of Cisco Systems, Inc. can be found on cisco.com.
Page 9 of 31
(FIB) entry and resolve the adjacency with the help of NHRP. When the NHRP mapping entry corresponding to this route times out (because there is
no more data traffic), NHRP will remove the route from the routing table. The details of these interactions are shown in Figure 4.

Figure 4. DMVPN Route Summarization After Proposed NHRP Cisco Express Forwarding Rewrite

The routing entries for the spoke tunnel from N1 to N2 (that is, [N1-TS2] to [N1-TS1] are created on demand when the spoke-to-spoke
tunnel is created.

Allow More than Two Hub Routers for DMVPN Spoke-to-spoke When Using OSPF
For complete spoke-to-spoke communication with OSPF, the network cannot grow beyond the maximum number of nodes supported by one hub.
In order to help ensure that the spokes carry subnetwork information for all other spokes with OSPF as the routing protocol, configure OSPF in
broadcast mode. This means the designated router and the backup designated router (the two hubs) need to be connected to every spoke (see Figure
5) and spoke-to-spoke networks cannot grow beyond these two hubs. This limitation will be removed after delivery of DMVPN Phase 3.

© 2005 Cisco Systems, Inc. All rights reserved.

Important notices, privacy statements, and trademarks of Cisco Systems, Inc. can be found on cisco.com.
Page 10 of 31
Figure 5. DMVPN Spoke-to-Spoke Network with OSPF

Remove Requirement that Hub Routers be Connected in a Loop for Large Spoke-to-Spoke Networks
To enable any-to-any spoke connection, the NHRP resolution packets must be able to be propagated among the hubs to find the NHRP database
information that is needed to answer the query. In Figure 6, to achieve any-to-any spoke communication between a spoke in cluster (1–300) to a
spoke in cluster (1501–1800), NHRP requests would travel across from hub 1 to hub 6, which are linked together in a daisy chain. The simplest form
of a daisy chain would be 1 –> 2 –> 3 –> 4 –> 5 –> 6 –> 1. The NHRP requests traverse the daisy chain, and the NHRP responses also traverse the
rest of the daisy chain. If a spoke in hub 1 wants to contact a spoke in hub 3, the request would be:

• NHRP request: 1 -> 2 -> 3

• NHRP response: 3-> 4 –> 5 -> 6 -> 1

This topology has the following limitations:

• Possibility of single hub failures

• Longer time lags between NHRP requests and responses as the number of hubs increases

© 2005 Cisco Systems, Inc. All rights reserved.

Important notices, privacy statements, and trademarks of Cisco Systems, Inc. can be found on cisco.com.
Page 11 of 31
Figure 6. DMVPN Multihub Topology for Deploying 1800-Node Spoke-to-Spoke Network

We have mechanisms that have a primary daisy chain going across all the hubs and a secondary daisy chain going across alternate hubs. The primary
daisy chain goes across all the routers, and the secondary daisy chain goes across every other router. This is illustrated in Figure 7 and shows that the
hubs are still connected in the event of failure of hub 3.

DMVPN Phase 3 would remove the requirement for daisy chains by:

• Dynamic discovery of the shortest path between hubs so NHRP requests and responses do not go along a single statically defined path.
• Resiliency against single and multiple hub failures. Failure of a hub should only isolate the part of the DMVPN cloud under that hub without
negatively affecting the rest of the DMVPN network.

Figure 7. Daisy Chaining in DMVPN Networks

© 2005 Cisco Systems, Inc. All rights reserved.

Important notices, privacy statements, and trademarks of Cisco Systems, Inc. can be found on cisco.com.
Page 12 of 31
 What is the scope of the Enhanced DMVPN QoS Project?
The Enhanced DMVPN QoS Project addresses the following:

• Traffic shaping at the hub for spoke overrun

• Shaping of multicast and Voice-Over-IP (VoIP) flows on a per-spoke basis
• Prioritization of routing updates to prevent drops and increase cloud sizes
• Solution for IPsec antireplay
• Solution for hub overrun

These are further explained in the following sections.

Traffic Shaping at the Hub for Spoke Overrun

Many IPsec topologies have heterogeneous router types. Some are very powerful, some are tiny. The large hubs can generate a lot of traffic and
sometimes overwhelm the smaller spokes. This happens because a user behind a “spoke” router (in hub-spoke scenarios) places a simple request
(for example, to a database), and a lot of data comes back very quickly. See Figure 8.

Figure 8. Small Spoke Overrun by Hub

Traffic Shaping for Hub Overrun

The opposite problem can also occur. A few high-powered spokes can consume a lot of bandwidth from the central site (either from the network
access or from the crypto engine) and deprive the other spokes of resources. The other spokes might not be able to perform as expected because of
the lack of resources (bad voice quality and slow applications may occur). See Figure 9.

© 2005 Cisco Systems, Inc. All rights reserved.

Important notices, privacy statements, and trademarks of Cisco Systems, Inc. can be found on cisco.com.
Page 13 of 31
Figure 9. One Spoke Using Too Much Bandwidth

Currently the order of packet processing results in encryption occuring prior to QoS at the outbound interface. DMVPN Phase 3 QoS enhancements
allow for complete QoS classification, scheduling, queuing, and traffic shaping support per security association prior to encryption of the packet.
There will be a separate queue per spoke for routing updates and a priority queue for voice and application traffic per spoke.

Figure 10 shows inbound and outbound traffic (ie: traffic that has to be encrypted and decrypted). Based on traffic selectors, the right security
association is chosen to encapsulate or decapsulate the packet. The associated QoS policy is then applied on the packet before it enters the crypto
engine for encryption.

© 2005 Cisco Systems, Inc. All rights reserved.

Important notices, privacy statements, and trademarks of Cisco Systems, Inc. can be found on cisco.com.
Page 14 of 31
Figure 10. DMVPN Phase 3 QoS per-Spoke Enhancements

What is the scope of the Enhanced Support for Multicast Project?

Current DMVPN networks have limited support for IP Multicast. Because DMVPN creates a nonbroadcast multiaccess (NBMA) network, the
hub routers must send a duplicate of the IP Multicast packet to each spoke router individually. Because the duplication is done before encryption,
this puts an extra load on the encryption hardware. For example, a 1-Mbps IP Multicast stream going to 20 spokes will mean 20*1 Mbps of
encryption bandwidth will be used.

© 2005 Cisco Systems, Inc. All rights reserved.

Important notices, privacy statements, and trademarks of Cisco Systems, Inc. can be found on cisco.com.
Page 15 of 31
Figure 11. DMVPN Network Topology for Multicast Propagation

In Figure 11, if spokes A, B, and C want to participate in a multicast stream from a multicast source M behind the hub, the hub will need to make
three copies of the multicast packet and encrypt each with the security associations for spoke A, B, or C. Extending the case to a 5-Mbps multicast
stream that needs to be sent to 200 spokes. It is very easy to note that this would result in the hub processing 5Mbps X 200 spokes, or 1 Gbps of
encrypted traffic.

In DMVPN Phase 3, using a common group key under the Group Domain of Interpretation (GDOI) protocol (RFC-3547), albeit is possible to
encrypt the IP Multicast packet first, then duplicate it to send to each individual spoke router that wants it. This will reduce the encryption bandwidth
utilization (back to 1 Mbps for the example).

With GDOI, it is possible to have a common group key to encrypt multicast flows where there would be a requirement for a single encrypt and
multiple copies. This mode scales better because the copying operation is far less computationally intensive than uniquely encrypting the multicast
flows for each spoke with its security association. This can be achieved by running a GDOI server on the DMVPN hub, where the hub uses a group
key to send routing updates/multicast traffic to the set of spokes that want/need it.

Figure 11 shows the protocol flows necessary for group members to participate in a group:

1. Customer-edge (CE) devices register with the key server. The key server authenticates and authorizes each CE and provides the IPsec policy and
keys necessary for the group member to encrypt and decrypt data packets for the group.

2. Group members exchange data packets encrypted by IPsec.

3. As needed, the key server pushes a “rekey” key management message to the group members. The rekey message contains new IPsec policy and
keys to use when old IPsec security associations expire. Rekey messages are sent in advance of security association expiration time to help
ensure that valid group keys are always available.

© 2005 Cisco Systems, Inc. All rights reserved.

Important notices, privacy statements, and trademarks of Cisco Systems, Inc. can be found on cisco.com.
Page 16 of 31
The GDOI protocol is described in detail in RFC-3547. The Cisco IOS Software implementation is documented in [GDOI-GM] and [GDOI-KS]
in the RFC.

Figure 12. DMVPN and GDOI Integration

What is the scope of the Better Management Support for DMVPN Project?
To analyze, manage, or troubleshoot the DMVPN network, four different relevant internal tables need to be correlated and examined:
NHRP mapping table, crypto socket table, crypto map table, and IPsec security association table.

The state of a DMVPN tunnel requires the examination of four different data structures.

1. NHRP mapping table, which maps the end destination IP host or network to the remote spoke tunnel IP address and the remote spoke physical
IP (NBMA) address. The NBMA address is then used to create/map to an entry in the crypto socket table. The end destination IP address to
tunnel IP address/NBMA address mapping could be many to one. The NBMA address to crypto socket table entry is one to one.

2. Crypto socket table, which is used by NHRP to create dynamic crypto map entries. It has the same basic information that is provided by a static
crypto map configuration (IPsec local and remote peer addresses, and IPsec proxies for the traffic that is to be encrypted/decrypted). This
information along with the IPsec profile information (transform set) is used to create a dynamic crypto map entry.

3. Crypto map table, which contains the IPsec peer addresses, IPsec proxies, and transform mode from the crypto socket and is used by IPsec to
trigger/control ISAKMP security associations and IPsec security associations.

4. ISAKMP security associations for each DMVPN peer and tunnel.

In order for a DMVPN tunnel to work, an NHRP mapping must point to a crypto socket entry pointing to a crypto map pointing to an IPsec security
association. Currently, four or five different commands need to be entered to show the state of a DMVPN tunnel, which also involves manually
matching entries from one table to entries in the next table. If the tunnel is not working, it is not clear which piece is not working. The following
linkage must be in place:

NHRP mapping –> crypto socket –> crypto map –> IPsec security association

For simplification, show and debug commands that will deal with a DMVPN tunnel as a single entity and look up and correlate the information in
these tables automatically for display will be provided. Also, MIBs for NHRP and mGRE tunnel interfaces as well as expanding the IPsec MIB to

© 2005 Cisco Systems, Inc. All rights reserved.

Important notices, privacy statements, and trademarks of Cisco Systems, Inc. can be found on cisco.com.
Page 17 of 31
include the crypto socket table will be provided. To summarize, DMVPN Phase 3 management would focus on embedded instrumentation in the
areas of:

• Show commands
• Syslogs
• Traps/informs
• Debugs
• MIBs

DMVPN SUPPORT, SCALABILITY, AND MANAGEMENT

This section covers:

• DMVPN support questions (multicast, VoIP, Multiprotocol Label Switching [MPLS], routing protocols supported, QoS, Network Address
Translation [NAT])
• DMVPN configuration questions
• DMVPN scalability questions (how many tunnels per mGRE, tunnels per router type)
• DMVPN management (MIBs, monitoring individual connections, enhanced command-line interface [CLI])
• DMVPN availability and resiliency questions (failover mechanisms, resiliency, dual hubs)

DMVPN Support Questions

What images are recommended for DMVPN?
As of July 26, 2005, the following images are recommended:

For Cisco 800 Series through Cisco 7200 Series:

• Cisco IOS Software Major Releases 12.3([9b, 9c, 9d]), 12.3([12a, 12b]), and 12.3(13a)
• Cisco IOS Software Releases 12.3(8)T6, 12.3(8)T7, 12.3(8)T8, and 12.3(11)T2

For Cisco Catalyst 6500 Series and Cisco 7600 Series:

• DMVPN hub only (mGRE, NHRP, RP on MWAM or back-end Cisco 7200 Series processor)
• Cisco IOS Software Release 12.2(18)SXD1

For Cisco Catalyst 6500 Series and Cisco 7600 Series:

• DMVPN hub or spoke (mGRE, NHRP, RP on MSFC and Cisco Catalyst 6500 Series Supervisor Engine 720)
• Cisco IOS Software Release 12.2(18)SXE (Rockies 2)

DMVPN was released in phases; the following outlines the first release and major enhancements added to DMVPN:

• Cisco IOS Software Release 12.2(13)T:

– This feature was introduced with basic hub-to-spoke and spoke-to-spoke functionality.
• Cisco IOS Software releases 12.3(6) and 12.3(7)T:
– (Phase 1) hub-to-spoke production-level functionality with new CLI introduced.
– VPN routing and forwarding (VRF) instance integrated into DMVPN
– NAT Transparency (NAT-T)–aware DMPVN
• Cisco IOS Software Releases 12.3(9a) and 12.3(8)T1:
– (Phase 2) spoke-to-spoke production functionality

© 2005 Cisco Systems, Inc. All rights reserved.

Important notices, privacy statements, and trademarks of Cisco Systems, Inc. can be found on cisco.com.
Page 18 of 31
 – Call admission control with DMVPN (Release 12.3(8)T1 only)

Running the latest Cisco IOS Software Release 12.3 or 12.3T, except for Cisco IOS Software releases 12.3(10) and 12.3(11)T, is recommended for
best functionality.

• Hub: Cisco IOS Software Release 12.3(9)a or 12.3(12)

• Spokes: Cisco IOS Software Release 12.3(8)T5 or 12.3(11)T2

Do not run any code release between the above-listed versions; there is a severe bug, CSCef77648, that will break DMVPN networks.

What platforms are recommended for DMVPN?

DMVPN is supported on the entire Cisco line of VPN router products, which range in application from small business to large VPN tunnel
aggregation points at a large enterprise central site. The following lists show the range of products available.

Hub Platforms
Cisco 1700 Series modular access routers; Cisco 1800, 2800, and 3800 series integrated services routers; Cisco 2600 Series multiservice platforms;
Cisco 3700 Series; Cisco 7206VXR Router; Cisco Catalyst 6500 Series Switches; Cisco 76000 Series Routers.

Spoke Platforms
Cisco 1700 Series; Cisco 3700 Series; Cisco 2600 Series; Cisco 800, 1800, 2800, and 3800 series integrated services routers; and Cisco 7206VXR
Router.

How does DMVPN support spoke-to-spoke tunnels?

Spoke-to-spoke tunnels are created on demand. The network can achieve greater scalability by creating tunnels on demand, in case some sites
need permanent spoke-to-spoke tunnels, which can also be created. The network should be in split-tunneling mode to create spoke-to-spoke tunnels.

Does DMVPN support split tunneling?

DMVPN supports both split tunneling and nonsplit tunneling. Spokes forward traffic based on the routes advertised by the hub. By pushing
appropriate routes to the spokes, it can perform split tunneling or not. To change from split tunneling to nonsplit tunneling (or vice versa), only the
hub configuration needs to be changed. The routing of the tunnel destinations (tunnel source addresses of the hubs) on the spoke needs to be checked
to make sure that they will still be correctly forwarded out the physical interface and not forwarded out the tunnel interface.

What are the concerns for DMVPN with multicast?

Multicast traffic needs to be replicated for all the spokes in the DMVPN network. In a network of 1000 nodes, a single multicast packet would
need to be encrypted 1000 times with 1000 unique security associations. A 5-Mbps feed could require 5 Gbps of encryption power. This problem
should be resolved in DMVPN Phase 3. Please refer to section 1.3.4 for additional information.

How is DMVPN with MPLS VPN configured?

Please refer to Appendix A in the “Enterprise Class Teleworker Deployment Guide” in the Deployment Guides section at
http://www.cisco.com/go/ect.

What routing protocols are supported for DMVPN?

EIGRP, On-demand Routing (ODR), and OSPF are supported for DMVPN. Details of DMVPN and ODR can be accessed at the white paper
Configuring DMVPN with On-Demand Routing in the Technical Documents section at http://www.cisco.com/go/dmvpn.

Intermediate System-to-Intermediate System (IS-IS) is not supported because it does not use IP as its network/transport protocol.

Border Gateway Protocol (BGP) is supported, but requires specification of all the neighbors individually in the configuration.

© 2005 Cisco Systems, Inc. All rights reserved.

Important notices, privacy statements, and trademarks of Cisco Systems, Inc. can be found on cisco.com.
Page 19 of 31
 What are the restrictions and caveats for running DMVPN directly on Cisco Catalyst 6500 Series Switches and Cisco 7600 Series Routers?

1. Only Cisco Catalyst 6500 Series Supervisor Engine 720 (not Cisco Catalyst 6500 Series Supervisor Engine 2) can be used as a DMVPN hub
or spoke.

2. Tunnel key should not be configured. If configured, neither EARL7 nor VPNSM will take over the tunnel, and the tunnel will be process-
switched.

3. Platforms other than the Cisco Catalyst 6500 Series must upgrade to Cisco IOS Software releases (12.3T or later) that relax tunnel key
requirements, if interacting with Cisco Catalyst 6500 Series using mGRE.

4. mGRE tunnels in non-VRF mode should not share the same tunnel source.

5. GRE tunnels in different VRFs cannot share the same tunnel source.

6. The commands below are not supported under mGRE:

a. ip tcp adjust-mss
b. qos pre-classify
c. tunnel vrf vrf_name
d. tunnel path-mtu-discovery

7. Because of CSCed21558:

a. The Cisco Catalyst 6500 Series cannot be a hub behind NAT.

b. If the Cisco Catalyst 6500 Series is a spoke, the hub cannot be behind NAT.

8. Because of CSCef95695:

a. If the Cisco Catalyst 6500 Series is a hub, then the spoke behind NAT must be a Cisco Catalyst 6500 Series, or the box must run an image
with the fix.
b. If the Cisco Catalyst 6500 Series is a spoke behind NAT, the hub must be a Cisco Catalyst 6500 Series, or the box must run an image with
the fix.

9. Officially multicast streaming is not supported across DMVPN on Cisco Catalyst 6500 Series and Cisco 7600 Series. Only multicast packets
from the control plane such as routing protocols are supported.

10. In a VRF-aware DMVPN scenario, mls mpls tunnel-recir must be configured in global on the PE/DMVPN hub if CE/DMVPN spokes need to
talk to other CEs across the MPLS cloud.

11. In a VRF-aware DMVPN scenario, the mGRE interface should be configured with a large enough IP Maximum Transmission Unit (MTU)
value to avoid RP doing fragmentation. See bug CSCeh37078.

12. In a VRF-aware DMVPN scenario, EIGRP should be avoided because of bug CSCsa62127 (this bug may be fixed before official release).

13. Officially DMVPN does not support blade-to-blade switchover on the Cisco Catalyst 6500 Series and Cisco 7600 Series.

How is DMVPN deployed on Cisco Catalyst 6500 Series switches with MWAM?
DMVPN as a hub on a Cisco Catalyst Switch has a slightly different implementation than configuring DMVPN on a Cisco 7200 Series. A brief
summary is outlined below.

© 2005 Cisco Systems, Inc. All rights reserved.

Important notices, privacy statements, and trademarks of Cisco Systems, Inc. can be found on cisco.com.
Page 20 of 31
1. A spoke initiates an Internet Key Exchange (IKE) session and uses the loopback address on the supervisor engine (SUP) as its peer address.
Multilayer Switch Feature Card (MSFC) on Cisco Catalyst 6500 Series or Cisco 7600 Series SUP terminates the IKE session. The loopback
address on SUP and Cisco MWAM is the same.

2. The MSFC creates the security associations on the VPNSM blade. An encrypted packet from the spoke is first decrypted by the VPNSM
module on the Cisco Catalyst 6500 Series.

3. The decrypted packet is the GRE packet. The Server Load Balancer (SLB) on SUP then load balances the GRE packet on the less loaded
MWAM processor. SLB contains the five processors of the MWAM module in its server farm.

4. MWAM processor is like the headend of the DMVPN setup except that it doesn’t have to decrypt anything. MWAM processor accepts the GRE
packets and accepts the NHRP registrations in its database.

5. The mGRE tunnel source interface on MWAM processor is the loopback address.

6. After the NHRP adjacency is established on a MWAM processor, the MWAM processor sends the EIGRP hellos down the new tunnel.

7. MWAM processor just encapsulates the packet toward the spoke from the hub, and VPNSM module encrypts the packet. The SUP then sends
the encrypted packet using an appropriate route to the spoke.

Detailed design information on DMVPN designs with the MWAM Access Module can be accessed at DMVPN Deployment on Cisco Catalyst 6500
Switches-MWAM and Native Modes white paper in the Technical Documents section at http://www.cisco.com/go/dmvpn.

What equipment is needed to deploy a DMVPN solution on Cisco Catalyst 6500 Series Switch with the MWAM Access Module?
The following equipment will be needed:

• Cisco Catalyst 6500 Series chassis with Cisco Catalyst 6500 Series Supervisor Engine 720
• VPNSM module for encryption/decryption
• MWAM module for the headend (terminating GRE tunnels and maintaining NHRP database)

Detailed design information on DMVPN designs with the MWAM Access Module can be accessed at “DMVPN Deployment on Cisco Catalyst 6500
Switches-MWAM and Native Modes” white paper in the Technical Documents section at http://www.cisco.com/go/dmvpn.

How is voice supported in a DMVPN network?

VoIP is an application that is very sensitive for end-to-end delay. DMVPN provides VoIP with an optimized path across the network using the
spoke-to-spoke functionality. With direct spoke-to-spoke connectivity, the voice packet has to cross the IP network only once and does not have to
pass by the hub; this provides for optimum path and lower delay for the voice packets. DMVPN provides fast setup time (less than 3 seconds) for the
direct path between the spokes, and traffic between the spokes is forwarded through the hub router during the setup time; which makes it transparent
for the end user.

Currently, during spoke-to-spoke tunnel setup, the voice packets are directed by the hub. These packets are process switched. This may lead to some
latency of voice traffic during initial call setup. This will be removed with the Routing Scalability Project in DMVPN Phase 3. Details are outlined in
section 1.3.2.

How are QoS parameters configured in a DMVPN network?

The DMVPN application runs on the router to provide dynamic IPsec tunnel creation and management mechanism. The same QoS concept
occurs when using IPsec apply with DMVPN.

© 2005 Cisco Systems, Inc. All rights reserved.

Important notices, privacy statements, and trademarks of Cisco Systems, Inc. can be found on cisco.com.
Page 21 of 31
Currently:

1. If the GRE tunnel addresses for the spokes are static and known, they can be used to classify traffic per spoke for traffic shaping, and then the IP
TOS field can be used to classify traffic within this shaping for policing. The QoS classification is statically defined and applied on the physical
interface.

2. If the data networks behind the spoke are known, then that can be used to classify unicast traffic that is destined for that spoke. This
classification can be used in shaping on the outbound physical interface, and a “child” policy can be used to police traffic within this shaping.
The policy will not be able to classify any multicast traffic per spoke because all multicast traffic would have the same source and destination
IP address no matter which spoke it was destined for.

When configuring QoS with IPsec and with DMVPN, it is important to use the QoS preclassify command on the tunnel interface to enable
QoS to apply the appropriate QoS service before the data is encrypted.

Another QoS mechanism with IPsec is LLQ-before-crypto. This feature aids only when and if the crypto engine on the router is congested.
Check the following link for an example on QoS with DMVPN:
http://www.cisco.com/en/US/tech/tk583/tk372/technologies_configuration_example09186a00801e6206.shtml.

Also, using the call admission control for IKE feature is important to limit the number of simultaneous IKE security associations that a router can
establish. This is especially important on the spoke router and to prevent overloading the DMVPN connection. See
http://www.cisco.com/en/US/products/sw/iosswrel/ps5207/products_feature_guide09186a0080229125.html.

For details on configuring Cisco QoS, check the Cisco IOS Software Quality of Service Solutions Configuration Guide:
http://www.cisco.com/univercd/cc/td/doc/product/software/ios123/123cgcr/qos_vcg.htm#wp998201.

What is the proposed QoS Functionality in DMVPN Phase 3?

DMVPN Phase 3 QoS functionality has been explained in detail in section 1.3.3. The proposed QoS functionality will be divided into two
phases:

• DMVPN QoS Phase 0

• DMVPN QoS Phase 1

Please contact dmvpn-core@cisco.com for exact delivery dates of DMVPN QoS Phase 0 and DMVPN QoS Phase 1.

With DMVPN QoS Phase 0 Enhancements

The outbound physical interface can have up to 255 shaping and policing classes. The classes must be statically configured by ACL or QoS group.
Up to 255 spokes can be assigned, 1 per group. For more than 255 spokes, multiple spokes will have to be assigned to the same group.

For QoS Prior to Encryption

Priority class information from QoS classes on the outbound physical interface is used to place packets in the LLQ before encryption. Also, packet
priority or precedence 6 packets are placed in a packet priority queue before encryption. All other packets are spread across 255 fair queues based on
a security association unique ID before encryption. Packets are encrypted in order from the LLQ, then the packet priority queue, then the 255 fair
queues.

With DMVPN QoS Phase 1 Enhancements

ISAKMP security association (spoke) shaping and policing with dynamic QoS classes per spoke prior to the encryption engine will be availble.
The dynamic QoS classes are cloned from a template QoS class when the spoke connects to the hub.

© 2005 Cisco Systems, Inc. All rights reserved.

Important notices, privacy statements, and trademarks of Cisco Systems, Inc. can be found on cisco.com.
Page 22 of 31
 What are the concerns with DMVPN and NAT?
In a hub-and-spoke DMVPN configuration, IPsec tunnels between hub and spoke are completely supported, and spoke-to-spoke communication
is possible through the hub (traffic goes from one spoke to the hub and then to the second spoke). If only the hub is using NAT or Port Address
Translation (PAT), then spoke-to-spoke tunnels are supported. Spoke-to-spoke tunnels are not supported at this time if NAT or PAT is used at the
spoke.

DMVPN supports NAT-T with the following restrictions:

DMVPN spokes may be behind dynamic NAT, and/or the DMVPN hub may be behind static-NAT. For DMVPN spokes behind dynamic NAT,
Cisco IOS Software Release 12.3(6) or 12.3(7)T is needed. If the DMVPN hub is behind NAT, then all DMVPN routers must run at least Cisco IOS
Software Release 12.3(9a) or 12.3(11)T.

DMVPN spoke routers behind the same NAT box must use NAT to communicate with unique outside (post-NAT) IP addresses. Two spokes with the
same IP address that they present to the DMVPN hub cannot be supported. NAT-T can handle this scenario (PAT), NHRP cannot.

With DMVPN spoke or hub routers behind NAT, IPsec transport mode must be used. IPsec transport mode is the recommended mode for all
DMVPN networks.

Dynamic direct spoke-to-spoke tunnels to/from DMVPN spokes that are behind NAT are not supported. Spoke-to-spoke tunnel may not come up in
all of the NAT-T scenarios:

• Neither spoke behind NAT: This scenario will work.

• One spoke behind NAT: This scenario may work.
• Both spokes behind NAT: This scenario may not work.
In the last case we can end up black-holing the spoke-to-spoke traffic. We therefore force packets to/from spokes behind NAT to use the hub. The
hub will send back its own tunnel IP address in the NHRP resolution response for an NHRP resolution query from a spoke that it knows is behind
NAT or about a spoke that it knows is behind NAT. There are two requirements for the hub to be able to do this:

• The spoke must do Cisco Express Forwarding switching; it will send an NHRP resolution request for the remote spoke tunnel IP address, not for
the network behind the remote spoke.
• IPsec transport mode must be used; otherwise, the hub cannot detect spokes that are behind NAT.

For Cisco IOS Software Release 12.3(13), 12.3(14)T, or 12.3(11)T3 , if the hub is behind NAT, dynamic direct spoke-to-spoke tunnels between
spokes that are not behind NAT are supported.

IPsec tunnels generated using DMVPN between hub and spoke are not supported with the hub behind NAT/PAT when using the Cisco
Catalyst 6500 Series or Cisco 7600 Series with VPNSM. If this is a design requirement, using the Cisco 7200 Series Router or other Cisco IOS
Software Router at the hub is recommended.

Can a remote DMVPN router be behind a NAT (with a private address)?

This is supported, as well as two spoke routers each behind NAT and having the pre-NAT address (the post-NAT address must be different).

DMVPN Configuration Questions

How are DMVPN timings configured?
Three different timers control the DMVPN functionality:

• NHRP Holdtime—The hub and spoke must agree on an NHRP holdtime (the recommendation is 10 minutes, or roughly three times the NHRP
registration request interval).

© 2005 Cisco Systems, Inc. All rights reserved.

Important notices, privacy statements, and trademarks of Cisco Systems, Inc. can be found on cisco.com.
Page 23 of 31
• Routing Protocol Hellos—This timer controls the failover and recovery of the DMVPN spokes. The default of each routing protocol varies and
is suitable in many cases.
• ISAKMP Keepalive—This allows users to configure the router to query the liveliness of its IKE peer at regular intervals.

How are dual hubs configured?

It only takes a few additional configuration lines to the spoke routers to set up dual (or multiple)-hub routers, for redundancy. There are two
ways to configure dual-hub DMVPNs:

• A single DMVPN network with each spoke using a single multipoint GRE tunnel interface and pointing to two different hubs as its Next Hop
Server (NHS). The hub routers will only have a single multipoint GRE tunnel interface.
• Dual DMVPN networks with each spoke having two GRE tunnel interfaces (either point to point or multipoint) and each GRE tunnel connected
to a different hub router. Again, the hub routers will only have a single multipoint GRE tunnel interface.

Details of DMVPN topologies and configurations can be accessed in the DMVPN white paper in the Technical Documents section at
http://www.cisco.com/go/dmvpn and the DMVPN design guide (SRND) at http://www.cisco.com/warp/public/779/largeent/it/ese/DMVPN_bk.pdf.

How is dial backup with DMVPN configured?

In a design with dial backup for redundancy, when a VPN tunnel, as primary path to the hub, fails, the router needs to bring the backup link up,
to maintain the user connectivity. There are two different approaches for implementing dial backup with DMVPN.

First Approach
This approach uses the following Cisco IOS Software features:

• Reliable Static Routing Backup Using Object Tracking

• Policy-Based Routing
• Dial Backup

With the Reliable Static Routing Backup Using Object Tracking feature, the spoke router uses Internet Control Message Protocol (ICMP) pings to
identify when the hub has become unavailable. A floating static route allows the initiation of the backup connection using the alternative path.
Policy-based routing is employed to direct the ICMP messages from the spoke router to the hub router using the primary interface. Check the
following document for a sample configuration:
http://www.cisco.com/en/US/tech/tk583/tk372/technologies_white_paper0900aecd801d5004.shtml

Second Approach
This approach uses the dialer watch feature. With the dialer watch feature, the router keeps track of a specific route learned from the primary hubs.
When this route disappears, because of a failure of the primary path, dialer watch initializes the dial link and brings up a second GRE tunnel. The
second GRE tunnel, when it is established, enables learning of all the routes, except the specific route, indicating that the backup path is active.
When the primary path is recovered from failure, the specific path is learned again, causing the dial backup to tear down.

The second approach support is for dynamic IP addresses at the spoke and for spoke-to-spoke functionality, for tunnel protection configuration
and may be used in many configuration scenarios. Details can be accessed at:
http://www.cisco.com/en/US/netsol/ns340/ns394/ns171/ns142/networking_solutions_white_paper0900aecd8024e9cc.shtml

What is tunnel protection, and how is it used in DMPVN?

In early versions of IPsec configurations, dynamic or static crypto maps are configured using the router CLI. These crypto maps specify which
IPsec transform set (encryption strength and Diffie-Hellman group) and Perfect Forward Secrecy (PFS) group is used and specify a crypto access list,
which defines interesting traffic for the crypto map. As of Cisco IOS Software Release 12.2(13)T, the concept of an IPsec profile was developed.

© 2005 Cisco Systems, Inc. All rights reserved.

Important notices, privacy statements, and trademarks of Cisco Systems, Inc. can be found on cisco.com.
Page 24 of 31
The IPsec profile shares most of the same commands as the crypto map configuration, but only a subset of the commands is valid in an IPsec profile.
Only commands that pertain to an IPsec policy can be issued under an IPsec profile; the IPsec peer address or the ACL to match the packets that are
to be encrypted cannot be specified.

The following valid commands can be configured under an IPsec profile:

• set-transform-set—Specifies a list of transform sets in order of priority.

• set pfs—Specifies PFS settings.
• set security-association—Defines security association parameters.
• set-identity—Specifies identity restrictions.

The IPsec profile is associated with a tunnel interface with the tunnel protection IPsec-profile command, also first introduced in
Cisco IOS Software Release 12.2(13)T. The tunnel protection command specifies that IPsec encryption will be performed after the GRE
encapsulation has been added to the tunnel packet. The tunnel protection command can be used with mGRE and point-to-point GRE (p-
pGRE) tunnels. With p-pGRE tunnels, the tunnel destination address will be used as the IPsec peer address. With mGRE tunnels, multiple IPsec
peers are possible; the corresponding NHRP mapping NBMA destination addresses will be used as the IPsec peer addresses.

If more than one mGRE tunnel is configured on a router (for instance, on a hub router), it is possible to reference the same tunnel source address on
each tunnel interface. In this case, the keyword shared is used in the tunnel protection command on both interfaces. This does not mean
that the two mGRE tunnels are hosting the same DMVPN “cloud”—each tunnel interface requires a unique tunnel key, NHRP network ID, and IP
subnet.

The tunnel protection command must be configured on both the hub and spoke tunnel interfaces.

How is tunnel protection achieved?

Define a transform set:
crypto IPsec transform-set ts esp-sha-hmac esp-3des
mode transport
• Use IPsec profile instead of crypto map:
crypto IPsec profile prof
set transform-set ts

(The IPsec profile is like a crypto map without set peer and match address. Internally Cisco IOS Software will treat this as a crypto map and
“guess” the set peer and match address parameters from the tunnel parameters and the NHRP cache.)

• The profile must be applied on the tunnel:

tunnel protection IPsec profile prof

What is the difference between tunnel protection and crypto map?

Tunnel protection uses IPsec profiles and is applied on tunnel interfaces. IPsec profile is almost the same as crypto map except that it doesn’t
need to know its peer’s address beforehand. Cisco IOS Software obtains these parameters from tunnel parameters and the NHRP cache.

Tunnel protection uses crypto sockets on Cisco IOS Software to dynamically create crypto maps internally as and when the need arises.

Why does an incomplete (negative) NHRP mapping entry get created?

When NHRP sends an NHRP resolution request, it inserts an incomplete (negative) NHRP mapping entry for the address in the resolution
request. This is to keep from triggering more NHRP resolution requests while this NHRP resolution request is being resolved and the IKE/IPsec
tunnel created. There already is a bug id (CSCsa43135 NHRP: Incomplete mapping entry stays in table too long) that reduces the time for the
incomplete (negative) NHRP mapping entry so that it will go away at the same time that NHRP gives up on its retransmits of the NHRP resolution

© 2005 Cisco Systems, Inc. All rights reserved.

Important notices, privacy statements, and trademarks of Cisco Systems, Inc. can be found on cisco.com.
Page 25 of 31
request rather than last for one hour. This will allow the next data packet to trigger another NHRP resolution request (with retransmits) to try to
complete the mapping.

For information on tracking defects, please access http://www.cisco.com/cgi-bin/Support/Bugtool/launch_bugtool.pl.

DMVPN Scalability Questions

How many spokes per tunnel interface and total spokes per platform are supported?
In solutions testing on the Cisco 7206VXR Router, it’s been observed that the number of spokes that can safely be paired to a given mGRE
tunnel interface is 350. If hub-and-spoke mode is in use, a second mGRE tunnel interface can be configured, for a total of 700 routing peers per
headend device, effectively creating a new DMVPN region or domain. If spoke-to-spoke connectivity between all spokes on a given hub is required,
then only a single mGRE interface, terminating 350 routing peers, is possible.

How is DMVPN spoke-to-spoke with daisy chaining configured?

Scalable spoke-to-spoke designs are more complicated to achieve. To achieve resiliency in the design, it is recommended that each spoke have
a link to two hubs, but these links do not belong to different subnets. One DMVPN (and therefore one subnet) must be spread out across:

• At least two hubs for redundancy

• Multiple hubs for scalability

To build spoke-to-spoke tunnels between spokes located on different hubs, the hubs must have NHRP maps to each other, just as the spokes have
NHRP maps to the hubs.

Groups of spokes are mapped to different hubs as primary and secondary. Because each hub has only one mGRE interface to aggregate the primary
and secondary tunnels from the spokes, the spoke fanout is limited to 175 routers per group (totaling 350 spokes per hub).

The hubs have NHRP maps to each other:

• Intrahub communications must flow over mGRE tunnels.

• Hubs must be routing peers of each other.

Figure 13 shows a spoke-to-spoke multihub topology.

© 2005 Cisco Systems, Inc. All rights reserved.

Important notices, privacy statements, and trademarks of Cisco Systems, Inc. can be found on cisco.com.
Page 26 of 31
Figure 13. Spoke-to-Spoke Multihub Topology

How is spoke-to-spoke tunnel disabled from certain sites?

When using an mGRE tunnel on the spoke, also configure the following commands to block spoke-to-spoke tunnel creation:
interface tunnel<x>
ip nhrp interest <acl>
access-list <acl> deny ip any any

Or to guarantee no spoke-to-spoke tunnels, configure the spokes to use p-pGRE tunnels with the tunnel destination pointing at the hub. If
two hubs are used, then there will be two p-pGRE tunnels on the spoke.

DMVPN Management Questions

What are the current management tools for DMVPN?
DMVPN can be configured and monitored using the following network management tools:

• Cisco IP Solution Center Version 3.1—Cisco IP Solution Center (ISC) Security Management enables customers to manage their network
security from a single or multiservice application. The following is a sample for configuring DMVPN using Cisco IP Solution Center:
http://www.cisco.com/en/US/products/sw/netmgtsw/ps4748/products_user_guide_chapter09186a00801e1990.html#1138244
• Cisco Router and Security Device Manager (SDM)—Cisco SDM provides two modes for configuring DMVPN: wizard mode and configuration
mode. SDM 1.2 and earlier provided support for hub and spoke configuration with SDM. Starting with SDM Version 2.0, the spoke-to-spoke
configuration was added to SDM. For configuring DMVPN using SDM, please refer to Configuring DMVPN Spoke Router in Full Mesh IPSEC
VPN using SDM in the Technical Documents section at http://www.cisco.com/go/dmvpn. The complete SDM Configuration guide can be accessed
at http://www.cisco.com/en/US/products/sw/secursw/ps5318/tsd_products_support_series_home.html.
• CiscoWorks Router Management Center (Router MC) starting with Version 1—Router MC is a Web-based application designed for large-
scale management of VPN and firewall configurations on Cisco routers. Router MC enables the setup and maintenance of VPN connections
between multiple Cisco VPN router devices:
http://www.cisco.com/en/US/products/sw/cscowork/ps3994/products_user_guide_chapter09186a00801f5966.html

© 2005 Cisco Systems, Inc. All rights reserved.

Important notices, privacy statements, and trademarks of Cisco Systems, Inc. can be found on cisco.com.
Page 27 of 31
 How are individual connections at the headend monitored?
Because of the DMVPN combination of multiple features working together (IPSec, IKE, GRE, NHRP, routing), the commands used to monitor
these features are mainly used to monitor DMVPN. Some examples of commands are show crypto session, show ip nhrp, show
crypto isakmp sa, show ip route, and show ip eigrp neighbors.

Also using SDM, monitor DMVPN tunnels in the monitor mode–> VPN Status –> DMVPN tunnels.

What is the support for DMVPN and MIBs?

DMVPN is a combination of multiple features working together. There is currently no one MIB library that manages all the DMVPN features.
However, the DMVPN Phase 3 Management Enhancements envision a group of MIBs for crypto sockets, GRE tunnels, and NHRP for completing
capturing all the variables required for DMVPN.

DMVPN High-Availability/Failover Mechanisms

What configuration is supported to provide DMPVN with resiliency and failover?
There are two basic topologies to be employed in DMVPN: dual-hub single DMVPN and dual-hub dual DMVPN; either of these is feasible
with either p-pGRE on the spokes (hub-and-spoke mode only) or mGRE on the spokes (allowing spoke-to-spoke tunnel setup). In general, dual-hub
dual DMVPN is recommended for hub-and-spoke networks, dual-hub single DMVPN is recommended for networks where a large percentage of
traffic is expected to flow through spoke-to-spoke tunnels.

What is the DMVPN dual-hub dual-DMVPN design?

With the dual-hub dual-DMVPN design, each DMVPN hub connects a tunnel interface with a different IP subnet, NHRP network ID, and GRE
tunnel key. Each spoke has an IPsec connection to both hubs at all times, and routing is used to control which tunnel is active and which is backup,
and how the spokes are load-balanced across the hubs. In this configuration all traffic is flowing through the primary IPsec tunnel, and the secondary
tunnel is being used only for resiliency.

Details of DMVPN topologies and configurations can be accessed in the DMVPN white paper in the Technical Documents section at
http://www.cisco.com/go/dmvpn and the DMVPN design guide (SRND) at http://www.cisco.com/warp/public/779/largeent/it/ese/DMVPN_bk.pdf.

What is the DMVPN dual-hub single-DMVPN design?

With the dual-hub single-DMVPN design, both DMVPN hubs connect a tunnel interface with the same IP subnet, NHRP network ID, and GRE
tunnel key. Each spoke has an IPsec connection to both hubs at all times. The static NHRP mappings from the spokes to the hubs define the tunnels
over which the routing protocol runs. If spoke-to-spoke tunnels are permitted by the configuration and are created because of traffic needs, the
dynamic routing protocol does not run over the spoke-to-spoke tunnels. Since the spoke routers are routing neighbors with the hub routers over the
same mGRE tunnel interface, it is not possible to use link or interface parameters (such as delay, cost, bandwidth, etc.) to prefer one hub over
another.

Details of DMVPN topologies and configurations can be accessed in the DMVPN white paper in the Technical Documents section at
http://www.cisco.com/go/dmvpn and the DMVPN design guide (SRND) at http://www.cisco.com/warp/public/779/largeent/it/ese/DMVPN_bk.pdf.

How much time does it take for failover with DMVPN?

The failover time with DMVPN is dependent on the routing protocol and timers in use. When the routing protocol detects a failure from a
neighbor peer, it causes rerouting of the routing table, and recovery of the network connectivity. OSPF, for example, detects a failure after detecting
a failure of three hellos from any peer; by default; the hello time of OSPF is 10 seconds. After the rerouting occurs, the traffic is forwarded
immediately on the preestablished IPsec tunnel with the backup DMVPN hub.

© 2005 Cisco Systems, Inc. All rights reserved.

Important notices, privacy statements, and trademarks of Cisco Systems, Inc. can be found on cisco.com.
Page 28 of 31
 Is stateful failover supported with DMVPN?
Stateful failover is not currently supported with DMVPN. Stateful failover relies on Hot Standby Router Protocol (HSRP) to provide detection
of failure. HSRP requires the dual hubs to be directly connected. Most DMVPN scenarios require the hubs to be local in geographically dispersed
locations. Failover still can be achieved by reasonably tuning the routing protocol’s hello.

What is needed to provide redundancy at the spoke routers?

HSRP with two spoke routers can be used to provide redundancy at the spoke site.

Enterprise-Class Teleworker Solution

What is the ECT solution?
Enterprise-Class Teleworker (ECT) is the Cisco IT implementation of an internal teleworker solution for Cisco employees to work from the
comfort of their homes with complete access to corporate resources. ECT is currently used by over 2000 Cisco employees and spans multiple Cisco
locations in the United States and overseas. ECT offers several unique features such as providing complete support for IP telephony, layers of
telephony, and access to video on demand (multicast applications) with layers of security.

For complete details on ECT, please visit the ECT Website at http://www.cisco.com/go/ect. The following white papers outline the solution and can
be accessed from http://www.cisco.com/go/ect.

• DMVPN Enterprise Class Teleworker Guide—This document describes the deployment of DMVPN and various aspects of the ECT solution in a
consolidated way.
• Layered Security in a VPN Deployment—This document describes the deployment of different aspects of layers of security in the ECT network.
• Deployment of Secure Sockets Layer VPNs—Gives topology/configuration guidance and tried and tested scenarios of DMVPNs with Secure
Sockets Layer (SSL) VPNs.
• Cisco IOS Software IPsec High Availability (HA)—Gives topology/configuration guidance and tried and tested scenarios of IPsec HA. IPsec HA
has been used in management gateways to provide transparent connectivity to management networks.
• Secure Voice and Wireless in a VPN Deployment—This document describes the deployment of voice and wireless applications in an ECT
network.
• Integrated Easy VPN and Dynamic Multipoint VPN—This document describes the deployment of an Easy VPN client on the same hub as
DMVPN hubs used in an ECT solution.

What are the layers of security in the ECT solution?

• Router antitheft protection

• Maintaining Customer Premises Equipment (CPE) configuration integrity; Cisco AutoSecure; supported platforms: Cisco 831 Ethernet Broadband
Router, Cisco 836 ADSL over ISDN Broadband Router, and Cisco 837 ADSL Broadband Router
• Maintaining client integrity: “loss of private key”
• Perimeter integrity: firewall access control and stateful inspection
• Client (router) authentication: public key infrastructure authentication, authorization, and accounting (PKI-AAA) integration
• User authentication: based on Authentication Proxy (Auth-Proxy) feature, integrated with AAA and Secure-ARP (in DHCP)
• Port authentication: 802.1x VLAN authentication for Cisco 831 Ethernet Broadband Router, Cisco 836 ADSL over ISDN Broadband Router, and
Cisco 837 ADSL Broadband Router; Cisco 1701 ADSL Security Access Router; Cisco 1711 and 1712 Security Access Routers; Cisco 1721,
1751, 1751-V, and 1760 modular access routers
• Data authentication and confidentiality
• Host protection: NAC (antivirus) protection
• Intrusion Prevention System (IPS): dynamic IPS
• Encrypted RSA private key

© 2005 Cisco Systems, Inc. All rights reserved.

Important notices, privacy statements, and trademarks of Cisco Systems, Inc. can be found on cisco.com.
Page 29 of 31
• Easy secure device deployment using transitive trusted introduction
Please refer to Layered Security in a VPN Deployment under Deployment Guides for detailed explanation of the ECT layers of security at
http://www.cisco.com/go/ect.

What are the authentication methods in the ECT solution?

802.1x and authproxy are some of the authentication methods. This is further described in the ECT Deployment Guide under Deployment
Guides at http://www.cisco.com/go/ect.

What components are involved in management of ECT?

From a provisioning/management perceptive:

• AAA for device authentication for PKI-AAA integration

• Cisco IOS Software Certificate Server for PKI deployment
• Cisco CNS 2100 Series Intelligence Engine as an event notification engine and configuration engine
• Cisco IP Solution Center for provisioning and management of the network
• Easy secure device deployment registrar for easy secure remote bootstrapping of new spokes

The details can be accessed at http://www.cisco.com/warp/public/732/Tech/security/ipsec/docs/ectmanagement.pdf.

Corporate Headquarters European Headquarters Americas Headquarters Asia Pacific Headquarters

Cisco Systems, Inc. Cisco Systems International BV Cisco Systems, Inc. Cisco Systems, Inc.
170 West Tasman Drive Haarlerbergpark 170 West Tasman Drive 168 Robinson Road
San Jose, CA 95134-1706 Haarlerbergweg 13-19 San Jose, CA 95134-1706 #28-01 Capital Tower
USA 1101 CH Amsterdam USA Singapore 068912
www.cisco.com The Netherlands www.cisco.com www.cisco.com
Tel: 408 526-4000 www-europe.cisco.com Tel: 408 526-7660 Tel: +65 6317 7777
800 553-NETS (6387) Tel: 31 0 20 357 1000 Fax: 408 527-0883 Fax: +65 6317 7799
Fax: 408 526-4100 Fax: 31 0 20 357 1100

Cisco Systems has more than 200 offices in the following countries and regions. Addresses, phone numbers, and fax numbers are listed on
the Cisco Website at www.cisco.com/go/offices.

Argentina • Australia • Austria • Belgium • Brazil • Bulgaria • Canada • Chile • China PRC • Colombia • Costa Rica • Croatia • Cyprus
Czech Republic • Denmark • Dubai, UAE • Finland • France • Germany • Greece • Hong Kong SAR • Hungary • India • Indonesia • Ireland • Israel
Italy • Japan • Korea • Luxembourg • Malaysia • Mexico • The Netherlands • New Zealand • Norway • Peru • Philippines • Poland • Portugal
Puerto Rico • Romania • Russia • Saudi Arabia • Scotland • Singapore • Slovakia • Slovenia • South Africa • Spain • Sweden • Switzerland • Taiwan
Thailand • Turkey • Ukraine • United Kingdom • United States • Venezuela • Vietnam • Zimbabwe

Copyright 2005 Cisco Systems, Inc. All rights reserved. CCSP, CCVP, the Cisco Square Bridge logo, Follow Me Browsing, and StackWise are trademarks of Cisco Systems, Inc.;
Changing the Way We Work, Live, Play, and Learn, and iQuick Study are service marks of Cisco Systems, Inc.; and Access Registrar, Aironet, ASIST, BPX, Catalyst, CCDA, CCDP,
CCIE, CCIP, CCNA, CCNP, Cisco, the Cisco Certified Internetwork Expert logo, Cisco IOS, Cisco Press, Cisco Systems, Cisco Systems Capital, the Cisco Systems logo, Cisco Unity,
Empowering the Internet Generation, Enterprise/Solver, EtherChannel, EtherFast, EtherSwitch, Fast Step, FormShare, GigaDrive, GigaStack, HomeLink, Internet Quotient, IOS, IP/TV, iQ
Expertise, the iQ logo, iQ Net Readiness Scorecard, LightStream, Linksys, MeetingPlace, MGX, the Networkers logo, Networking Academy, Network Registrar, Packet, PIX, Post-
Routing, Pre-Routing, ProConnect, RateMUX, ScriptShare, SlideCast, SMARTnet, StrataView Plus, TeleRouter, The Fastest Way to Increase Your Internet Quotient, and TransPath are
registered trademarks of Cisco Systems, Inc. and/or its affiliates in the United States and certain other countries.

© the
All other trademarks mentioned in this document or Website are 2005 Cisco
property of Systems, Inc.owners.
their respective All rights reserved.
The use of the word partner does not imply a partnership relationship between
Cisco and any other company. (0502R) notices, privacy statements, and trademarks of Cisco Systems, Inc. can be found on cisco.com.
Important 205297.G_ETMG_SH_6.05
Page 30 of 31
Printed in the USA
 © 2005 Cisco Systems, Inc. All rights reserved.
Important notices, privacy statements, and trademarks of Cisco Systems, Inc. can be found on cisco.com.
Page 31 of 31