Beruflich Dokumente
Kultur Dokumente
The Cisco Cloud Services Router (CSR) 1000V is software appliance version of the
Cisco ASR 1000 Series routers. It can be used to extend advanced enterprise network
and security services such as IPSec VPNs, NAT, FW, application visibility, and SLA
monitoring into an AWS VPC environment.
The CSR 1000v runs as an EC2 instance and is launched from the AWS market place.
Figure 1 shows a notional view of the CSR 1000v in an AWS VPC. By using the VPC
routing table, traffic from the EC2 instances will be forwarded through the CSR 1000v so
that services can be applied.
CSR 1000v
AWS igw
.254
Since the CSR 1000v runs as an EC2 instance, it can rely on native EC2 high availability
mechanisms in the event of underlying compute hardware issues. In this case, the CSR
would be restarted and recovery times would be on the order of minutes. For designs
that require fast convergence, the CSR 1000v can be deployed in a redundant pair with
failover between them.
Solution Overview
The logic of the solution is as follow:
1. A pair of CSR 1000vs are deployed into an AWS VPC.
2. A GRE tunnel is configured between the CSRs.
3. Bi-directional Forwarding Detection (BFD) and a routing protocol (EIGRP or BGP)
are enabled on the GRE tunnel for peer failure detection.
4. Each CSR 1000v is configured with an Embedded Event Manager (EEM) applet
that will monitor BFD peer down events
5. When a BFD peer down event is detected, the EEM applet will use the AWS EC2
VPC API to modify the VPC route table to redirect traffic around the failure.
Solution Details
The topology in figure 2 is an example of a VPN gateway configuration.
This topology uses a single availability zone and four VPC subnets. Other topologies,
including multiple availability zones, single or multi subnet VPCs, multiple VPN tunnels,
and multiple CSR Ethernet interfaces, are possible and would be applicable to this
solution.
Each CSR also has a VPN tunnel to Internet. These tunnels would typically terminate at
another VPN device located on the enterprise network or another VPC.
To support the high availability solution, a GRE tunnel is configured between the local
CSRs. These GRE tunnels allow the CSRs to exchange BFD control packets that are
used for peer failure detection.
The EC2 instances reside in a private subnet, Private Subnet-3, in the topology diagram.
If the CSR is not directly connected to this private subnet, it is recommended to add a
static route for the private subnet to each CSR. This static route points to the address of
the VPC router on the public subnet. This address will always be the first usable address
of a subnet. For example, the VPC router address for the subnet 172.24.2.0/25 will
172.24.2.1.
EIGRP is used as the routing protocol, though other routing protocols could be used. The
primary purpose of the routing protocol is to register as a BFD client. BFD requires at
least one client protocol before it will initiate neighbor discovery. An additional benefit of
the GRE tunnel and the routing protocol is that they can be used to establish a back-up
path in case of VPN tunnel failures.
The EC2 private subnet, Public Subet-3, has its own VPC route table. The default route
for this subnet will have a target of the public subnet network interface (GigabitEthernet1)
of one of the CSRs. Because the VPC route table only allows for one active target per
route, only one CSR is in the egress traffic path for this subnet. Ingress traffic flow over
the VPN tunnels is determined by the remote VPN devices, so it is possible that CSR-B
is the active ingress path or that load sharing is being done between CSR-A and CSR-B.
In this example, ingress and egress traffic is initially being forwarded through CSR-A, as
shown in Figure 3.
CSR-A then fails, as shown in Figure 4. The goal is to shift traffic so that it will egress
through CSR-B and no longer ingress through CSR-A.
For the ingress traffic flow, the remote VPN device will detect that the VPN tunnel
terminated at CSR-A is no longer available. This is done using traditional VPN tunnel high
availability techniques such as routing protocols (with or with out BFD) and IKE dead peer
detection.
When BFD times out on CSR-B, a log message similar the following is generated.
EEM is an event detection and automation technology available on the CSR. The EEM
applet is configured to run whenever the BFD peer down log message is generated.
When triggered, the EEM applet will use the AWS API ec2-replace-route command to
modify the VPC route table to make itself the new target for the default route.
The CSR cannot access the AWS EC2 API directly. This requires use of a helper VM with
the AWS EC2 CLI tools installed. CSR-B will SSH into the helper VM and run the ec2-
replace-route command. See the section titled Setting up the Helper VM for more
details on configuring this VM.
An example EEM applet is found in figure 6. The EEM configuration on CSR-A and CSR-
B will be nearly the same. This is covered in more detail in the deployment procedure
section.
Figure 7 shows CSR-B modifying the VPC Route table for the default route.
Once the VPC route table is modified, the VPC will begin directing egress traffic to the
CSR-B, as show in Figure 8.
For more information on deployment steps and CSR configuration, please consult the
following documents.
CSR-A(config)#end
CSR-A#wr mem
Building configuration...
[OK]
CSR-A#reboot
A private subnet is required from each CSR for connectivity to the helper VMs.
First, you will need to create two private subnets for the helper VM using the VPC
dashboard. You can use a /28 subnet mask since these subnets will only be used by the
CSR and the helper VMs.
Once these subnets are created, navigate to the EC2 dashboard and create two network
interfaces, one in each of these subnets. Be sure to note the ENI values and IP
addresses for these newly created interfaces.
Then attach the network interfaces to the two CSRs. These will show up as an additional
Gigabit Ethernet interface (GigabitEthernet2 in our example). Use the IOS CLI to
manually configure the IP address and no shut the new interfaces.
CSR-A(config)#int gigabitEthernet 2
CSR-A(config-if)#no shut
CSR-A(config-if)# ip address 172.24.2.22 255.255.255.240
Navigate back to the VPC dashboard. Then create two new route tables and add a
default route (0.0.0.0/0) to each, one with a target of the CSR-A GigabitEthernet2 ENI
value, and the other with a target of CSR-B.
Next, associate the new route tables to the new subnets. Be sure the default route target
ENI is local to that subnet, e.g. the route table with a 0.0.0.0/0 target of CSR-A
GigabitEthernet2 ENI is mapped to the subnet assigned to the CSR-A GigabitEthernet2
ENI.
Finally, NAT translation should be configured on the CSRs to allow the helper VMs to
access the AWS APIs to modify the routes. The NAT configuration on the CSR will
translate the helper VM IP address to the CSR interface address when it connects to
AWS API servers.
The following is a sample NAT configuration, where GigabitEthernet1 is the public
interface, GigabitEthernet2 is the helper VM subnet interface, and 172.24.2.25 is the
address of the Helper VM.
interface GigabitEthernet1
ip nat outside
!
interface GigabitEthernet2
ip nat inside
!
ip nat inside source list helper-vm-list interface GigabitEthernet1 overload
Please refer to the section titled Setting up the Helper VM in this document for the
deployment procedure of the helper VM. Each helper VM is associated with a CSR.
The BFD values are configurable and can be made more aggressive if faster
convergence is desired. However, this can lead to BFD peer down events during
intermittent connectivity. The above values, which will detect peer failure within 1.5
seconds, have been tested to be stable in an AWS VPC environment.
There is also a variable delay associated with the time the AWS API command is
executed and the time the VPC routing table changes go into effect. This is usually on
the order a few seconds.
Step 6: Collect the route table ID and network interface ID for each CSR.
The route table ID and network interface ID can be found using the AWS console.
3
2
2 3
To promote the reusability of this applet, local variables are separated out of the body of
the EEM applet and are defined as EEM environment variables. The variables used are as
follows:
The main differences between the CSR-A and CSR-B EEM applet should be the helper
VM and the ENI environment variables. These should be set to the local helper VM
address and local network interface ID of the CSR.
Step 8: Verification
First check that the BFD and EIGRP relationships are established and normal on both
peers. This example as shows the local peer on Tunnel 33, and also the remote peer on
Tunnel 98.
CSR-A#show bfd neighbors
IPv4 Sessions
NeighAddr LD/RD RH/RS State Int
172.24.33.2 4097/4104 Up Up Tu33
172.24.98.1 4098/4108 Up Up Tu98
Log into the Helper VM and verify the current VPC route table configuration.
[csr@ip-172-24-2-25 ~]$ ec2-describe-route-tables rtb-c41b78a5
ROUTETABLE rtb-c41b78a5 vpc-950467f4
ROUTE local active 172.24.2.0/24 CreateRouteTable
ROUTE i-6f3aef4c active 0.0.0.0/0 eni-060ce72d CreateRoute
ASSOCIATION rtbassoc-c6cc33a3 subnet-b00b68d1
For this example, the current active gateway is CSR-A, which has the ENI value of eni-
060ce72d.
To create a failure, the active CSR is rebooted. The new target for the default route
should be the ENI of CSR-B.
[csr@ip-172-24-2-85 ~]$ ec2-describe-route-tables rtb-c41b78a5
ROUTETABLE rtb-c41b78a5 vpc-950467f4
Full Configurations
CSR-A
version 15.4
service timestamps debug datetime msec
service timestamps log datetime msec
no platform punt-keepalive disable-kernel-core
platform console virtual
!
hostname CSR-A
!
boot-start-marker
boot-end-marker
!
!
!
no aaa new-model
!
!
!
!
!
!
!
!
!
!
!
!
!
!
!
!
!
!
!
subscriber templating
!
multilink bundle-name authenticated
!
!
crypto pki trustpoint TP-self-signed-941582656
enrollment selfsigned
subject-name cn=IOS-Self-Signed-Certificate-941582656
revocation-check none
rsakeypair TP-self-signed-941582656
!
!
crypto pki certificate chain TP-self-signed-941582656
certificate self-signed 01
30820229 30820192 A0030201 02020101 300D0609 2A864886 F70D0101 05050030
30312E30 2C060355 04031325 494F532D 53656C66 2D536967 6E65642D 43657274
69666963 6174652D 39343135 38323635 36301E17 0D313430 39313531 34303031
385A170D 32303031 30313030 30303030 5A303031 2E302C06 03550403 1325494F
532D5365 6C662D53 69676E65 642D4365 72746966 69636174 652D3934 31353832
36353630 819F300D 06092A86 4886F70D 01010105 0003818D 00308189 02818100
CF087FE6 23BA0724 A729B1D5 E0B4A492 0403C24A A6958DFF 1235127D A0842721
57552EFA F3A271EF B1463E17 A4F6AC36 67D215CF 3A12A199 06F17F4B 2194F7B9
4F28C684 9369E399 938C4DD5 B24BA518 2CB913F8 1E8DB5F5 8484D22F F69E2B0A
8716BFC2 D34C1110 C0B7A703 3A480AA1 E4E9F95D 5D71E7D6 03DC44EA B333471F
02030100 01A35330 51300F06 03551D13 0101FF04 05300301 01FF301F 0603551D
23041830 168014FD 2737051A B81CA2DD 31832423 04189629 DB272230 1D060355
1D0E0416 0414FD27 37051AB8 1CA2DD31 83242304 189629DB 2722300D 06092A86
4886F70D 01010505 00038181 00A7A0CE E3A90EA7 0DDE2E48 12AC5D98 DBE1075B
BE4C1DE0 C42DC4DA 8D8F9F51 F6290594 E8F66193 A250385E 8A1B476F 0F3ED5B1
D750A497 6DA564CD 8E0B4CB5 E2842365 95655D3F 44B762A0 81B888E2 1DA97373
605A15F9 2E27DBB5 23B3415A 9E98DC12 D69F0AFF B6E7B7D3 52E350F1 6D2D7FED
41C70835 2E264F64 BF61C3DE 89
quit
license udi pid CSR1000V sn 9J7C89PX47H
license boot level ax
spanning-tree extend system-id
!
username ec2-user privilege 15 secret 5 $1$00XX$BXyXX2Mpsdq3bLPdeJZFw1
!
redundancy
mode none
!
!
!
!
crypto ikev2 profile default
match identity remote fqdn domain cisco.com
identity local fqdn csr-a.cisco.com
authentication remote pre-share key cisco123
authentication local pre-share key cisco123
CSR-B
version 15.4
service timestamps debug datetime msec
service timestamps log datetime msec
no platform punt-keepalive disable-kernel-core
platform console virtual
!
hostname CSR-B
!
boot-start-marker
boot-end-marker
!
!
!
no aaa new-model
!
!
!
!
!
!
!
!
!
!
!
!
!
!
!
!
!
!
!
subscriber templating
!
multilink bundle-name authenticated
!
!
crypto pki trustpoint TP-self-signed-3549366697
enrollment selfsigned
subject-name cn=IOS-Self-Signed-Certificate-3549366697
revocation-check none
rsakeypair TP-self-signed-3549366697
!
!
crypto pki certificate chain TP-self-signed-3549366697
certificate self-signed 01
3082022B 30820194 A0030201 02020101 300D0609 2A864886 F70D0101 05050030
31312F30 2D060355 04031326 494F532D 53656C66 2D536967 6E65642D 43657274
69666963 6174652D 33353439 33363636 3937301E 170D3134 30393135 31343030
31335A17 0D323030 31303130 30303030 305A3031 312F302D 06035504 03132649
4F532D53 656C662D 5369676E 65642D43 65727469 66696361 74652D33 35343933
36363639 3730819F 300D0609 2A864886 F70D0101 01050003 818D0030 81890281
8100CEEC A9E186D4 93051637 C56B5C7D B3A42A89 A1EDF09B E394A952 FB029E8F
F960F953 6A8C643E D8096336 62D5EE20 F866E6BE 65DDB9F9 67E1D118 AB42A2E3
11A5B7F8 B6947315 93E9E0AF 5BC28553 C443494E 3DF1041F 1F029AB9 F14EAC56
7874EC32 67272303 896B5C50 D7E85A54 A3B8671A A878F422 E922A96C 02D7188D
BC410203 010001A3 53305130 0F060355 1D130101 FF040530 030101FF 301F0603
551D2304 18301680 1423492E BA79B04D 38488FF1 C1B2FB36 8CF9CBD2 45301D06
03551D0E 04160414 23492EBA 79B04D38 488FF1C1 B2FB368C F9CBD245 300D0609
2A864886 F70D0101 05050003 81810062 31209D60 92ED396B 268A11F8 543FAFB1
9A7D293F AF39182B 1A297A6A 57EEC65F 9513A99E 46D89EF1 D17F59EE 7BF88220
654A734F 703FC30B A105AC5D 10A5803F 593B1013 CDA07D43 B209BF88 C2404C0C
DB209648 DC559AED E19DA40A C54C94F5 B8EF3187 5961C2D7 CE55A438 2A61380C
D6D4108A 494A8FDE 19A59A1C 6F4CC3
quit
license udi pid CSR1000V sn 9TMLDL66HOH
license boot level ax
spanning-tree extend system-id
!
username ec2-user privilege 15 secret 5 $1$GMHV$rx2mhUL6iLtdnBRUID4Zr1
!
redundancy
mode none
!
!
!
!
crypto ikev2 profile default
match identity remote fqdn domain cisco.com
identity local fqdn csr-b.cisco.com
authentication remote pre-share key cisco123
authentication local pre-share key cisco123
!
crypto ikev2 dpd 10 2 on-demand
!
!
!
ip ssh rsa keypair-name ssh-key
ip ssh version 2
ip ssh pubkey-chain
username ec2-user
Browse to the IAM dashboard, and navigate to Roles> Create Role as shown below
Create a role with appropriate role name. In this example, we are using the name
ChangeRouteRole.
The helper VM is only used to run EC2 API commands, so a micro instance can be used.
Two helper VMs need to be created, one in each of the private subnet local to the CSR.
It is advised to configure an AWS security group that restricts access inbound to SSH
only, and ideally, the IP address of the CSRs and any administrative workstations. Initially,
an administrative workstation will be required to login to the ec2-user account and setup
the helper VM.
During the launch use the IAM role setup in the previous steps.
The following figures show the steps for launching the helper VM from the AWS console.
3
1
Since the helper VMs do not have public addresses, it may be required to first SSH into
an EC2 instance that does have direct internet access, and then log into the helper VM
hosts from that public host using the key pair that was assigned in the previous step.
Under the list of EC2 instances, find the helper VM and locate the IP address.
sudo vi /etc/ssh/sshd_config
Restart sshd.
ssh csr@172.24.2.25
Step 8: Set the default region environment variable based on your VPN location
Edit the .bashrc file to set the region for the local VPC.
export EC2_URL=https://ec2.us-east-1.amazonaws.com
Source the .bashrc again and run a API command to make sure the local region is set.
Make sure to record the private IP address of the VM and the created csr account
username and password so that they can be used in the EEM applet.
References
CSR 1000v for AWS Deployment Guide
https://supportforums.cisco.com/sites/default/files/deployment_guide_final_community_
support.pdf