IPE DC Lab

CCIE Data Center Lab Preparation Workbook
Chapter 21: Mock Lab

Challenge 2
Chapter 21: Mock Lab Challenge 2 is the second of 3 mock lab challenges that will test you on
all aspects of the CCIE Data Center Blueprint. This first lab will have an equal difficulty level as
the actual lab to get you familiar with the set-up and all aspects involved.
We highly recommend creating your own diagram at the beginning of each lab so you are able
to draw on your own diagram, making it much easier when you step into the real lab.
Multiple topology drawings are available for this chapter.
General Rules
Try to diagram out the task. Draw your own connections the way you like it
Create a checklist to aid as you work thru the lab
Take a very close read of the tasks to ensure you dont miss any points during grading!
Monitor your time. This is a Mock Lab. Verify how many points you earn in a given time
frame
Partial credit is not given. Any task should be completed 100% to receive credit
You require a score of 80 out of 100 points to have a passing score
Estimated Time to Complete: < 8 hours
Pre-setup
Connect to all devices within the topology

Use the central topology drawing at the start of this workbook
Load the pre-configuration associated to this lab
This lab is intended to be used with online rack access provided by our partner Proctor Labs
(www.proctorlabs.com). Connect to the terminal server and complete the configuration
tasks as detailed below
Copyright by IPexpert. All rights reserved.
Drawing 1: Physical Topology
Introduction
You are the engineer hired to build a dual datacenter set-up for a customer. These 2 data
centers will be interconnected via a Layer 2 connection. In this datacenter design a number of
key components are introduced including the Nexus 7000, Nexus 5000, Nexus 2000, UCS
system and MDS switches.
In this set-up as shown in the drawing all connections are directly connected, unless specifically
stated. SW1 is pre-configured with different VDCs.
All devices have management IP addresses and credentials pre-configured per the following
table. Please refer to this table when working through the devices. A console connection is also
available.
IP
Username
Password
SW1-1
10.10.210.71
admin
IPexpert123
SW1-2
10.10.210.72
admin
IPexpert123
SW1-3
10.10.210.73
admin
IPexpert123
SW1-4
10.10.210.74
admin
IPexpert123
SW2
10.10.210.51
admin
IPexpert123
SW3
10.10.210.52
admin
IPexpert123
MDS1
10.10.210.41
admin
IPexpert123
MDS2
10.10.210.42
admin
IPexpert123
ACE
10.10.210.39
admin
IPexpert123
UCS-1 (cluster)
10.10.210.81
admin
IPexpert123
Device
FI-A
10.10.210.82
admin
IPexpert123
FI-B
10.10.210.83
admin
IPexpert123
Blade KVM
10.10.210.91-98
n/a
n/a
Verify all pre-configuration as there is troubleshooting involved in this mocklab!
Section 1
Data Center Networking
Task 1: VDC allocations

VDC
Interfaces
SW1-1
Ethernet3/9, Ethernet3/11, Ethernet3/13, Ethernet3/15

Ethernet4/1-2, Ethernet4/9-12
SW1-2
Ethernet3/10, Ethernet3/12, Ethernet3/14, Ethernet3/16

Ethernet4/3-4,Ethernet4/13-16
SW1-3
Ethernet3/1, Ethernet3/3, Ethernet3/5, Ethernet3/7, Ethernet3/17, Ethernet3/19,

Ethernet3/21, Ethernet3/23
Ethernet4/5-6, Ethernet4/17-20
SW1-4
Ethernet3/2, Ethernet3/4, Ethernet3/6, Ethernet3/8, Ethernet3/18, Ethernet3/20,

Ethernet3/22, Ethernet3/24
Ethernet4/7-8,Ethernet4/21-24
1. SW1 is a Nexus 7000 with 4 VDCs. The interfaces should be allocated according to the
following details
2. Ensure the port allocations are as the following table and the Nexus 7000 VDCs will support
the required features and configuration
1 point
Task 2: DC 1 VLAN
VLAN
123
Name
USERS
Device
SW1-1, SW1-2, SW2, SW3
124
USERS2
SW1-1, SW1-2, SW2, SW3
125
SERVERS
SW1-1, SW1-2, SW2, SW3
126
SERVERS2
SW1-1, SW1-2, SW2, SW3
1011
EIGRP
SW2, SW3
1012
VRRP
SW2, SW3
1111
FCIP
SW2, SW3
1. Assign the following VLANs with names.

2. Ensure that the VLANs are only created on the specified devices
1 point
Task 3: Trunk interfaces

1.
2.
3.
4.
5.
On Ethernet1/5 and Ethernet1/6 on SW2 and SW3 a trunk link should be configured
All VLANs which are previously configured should be allowed to pass between the switches
Enable Spanning-Tree Bridge Assurance across this connection
Allow Jumbo frames in the best-effort QoS class
When one of the links would need to go in Spanning-Tree Blocking state it should be
interface Ethernet1/5
3 points
Task 4: Routing
Device
Link
IP address
SW1-1
Ethernet3/9
198.18.12.1/25
SW1-1
Ethernet3/11
198.18.21.1/24
SW1-1
Ethernet3/1
198.19.12.1/26
SW1-1
Ethernet3/5
198.19.13.1/30
SW1-2
Ethernet3/10
198.18.12.2/25
SW1-2
Ethernet3/12
198.18.21.2/24
SW1-2
Ethernet3/2
198.19.22.1/30
SW1-2
Ethernet3/6
198.19.23.1/30
SW2
Ethernet1/1
198.19.12.2/26
SW2
Ethernet1/2
198.19.22.2/30
SW2
Vlan1011
198.19.223.1/24
SW3
Ethernet1/1
198.19.13.2/30
SW3
Ethernet1/2
198.19.23.2/30
SW3
Vlan1011
198.19.223.2/24
1. Configure routed links according to the IP addressing information in the table stated above.
2. Configure OSPF on all interfaces except the SVIs
3. Use the optimal network type on links with a /30 subnetmask and ensure the highest
numbered switch becomes the DR on the other links
4. Assign Loopback interfaces with an IP address in the range of 198.18.0.0/32 with a host
address equal to the switch number.
5. Advertise Loopback interfaces into OSPF
6. Configure EIGRP using the SVI on SW2 and SW3 and ensure that this link can be used as
backup when the links to SW1-1 and SW1-2 fail.
5 points
Task 5: vPC
1. On Ethernet1/15 on both SW2 and SW3 a server is connected. Ensure that this server has a
single link towards the switches, where the switches act as a single switch
2. Ensure the ports transition to the forwarding state immediately after they come online, the
server port should be a trunk
3. Use the Loopback interfaces for keepalives
4. Use the LACP protocol towards the server where the vPC switches advertise themselves
with a priority value of 100 and MAC address 12:34:56:78:ab:cd
5. You should be certain that SW2 has the primary role

6. Use Ethernet1/6 as the inter-link for this configuration, do not use LACP
7. Ensure that after an outage, the vPC working is fully recovered automatically after a delay of
5 minutes
5 points
Task 6: FEX
1. Both Fabric Extenders should be dual-homed to SW2 and SW3
2. FEX1 should have identifier 105 and FEX2 should have identifier 106
3. Use vPC numbers 100 for FEX1 and 101 for FEX2
3 points
Task 7: EvPC
1. Ensure the first port of both fabric extenders is using a single logical connection without
LACP
2. Ensure the ports transition to the forwarding state immediately after they come online
3. Allow only VLAN 125 and 126 towards the connecting router
1 point
Task 8: First Hop Redundancy
1. The first hop for VLAN 1012 should be made redundant on SW2 and SW3 using a standards
based FHRP
2. Use the 172.22.12.0/24 subnet for VLAN 1012 where the last address of the subnet is the
virtual address and the first and second address are the phyiscal IPs of the switches
3. VLAN 125 should also be made redundant using a Cisco proprietary protocol
4. Use the 172.22.125.0/24 subnet for VLAN 125 where the first address of the subnet is the
virtual address and the second and third address are the phyiscal IPs of the switches
5. SW2 should be the primary default gateway for VLAN1012 and SW3 should be the primary
for VLAN 125
6. When one of the 2 OSPF uplinks fails on SW3 it should not be forwarding traffic for traffic
coming from a vPC on VLAN 125
7. When both the OSPF uplinks fail on SW3, SW2 should acquire the primary role on VLAN 125
8. VLAN 125 should fail-over after 3 hello packets are missed in a 750ms timeframe
5 points
Task 9: FabricPath
1. Prepare SW2 and SW3 to support FabricPath on VLAN 123 and VLAN 124
2. Enable FabricPath on the interfaces Ethernet4/11 and Ethernet4/12 on SW1-1 and
Ethernet4/15 and Ethernet 4/16 on SW1-2.
3. Adjacencies will only come online after DC2 has been configured
4. Allow VLAN 123 and VLAN 124 to cross between DC1 and DC2
5. FabricPath adjacencies should be terminated when no hello packets are received for 12
seconds
6. The lowest numbered link should always be preferred from DC1 to DC2, where traffic from
DC2 to DC1 should use the highest numbered link
7. Configure Switch IDs according to the Switch number (SW1-1 is SID 11)
8. FabricPath authentication using a text password of FPauth should automatically be
enabled on all current links and automatically on future links
3 points
Task 10: OTV
1. Create VLANs 201, 202 and 203 on SW1-2, SW1-3, SW1-4 and SW3
2. Create SVIs on SW3 and SW1-3 for VLAN 201 and 202 using an IP address in the range of
198.0.X.Y/24 where X is the VLAN number and Y is the switch number (SW1-3 = 13).
3. Create a standard Layer 2 trunk without Spanning-Tree Bridge Assurance on Ethernet1/4 on
SW4 and Ethernet3/8 on SW1-2 allowing VLANs 201, 202 and 203
4. Create a Layer 2 trunk with Spanning-Tree Bridge Assurance on Ethernet3/19 on SW1-3 and
Ethernet3/20 on SW1-4 allowing VLANs 201, 202 and 203
5. Configure an IP address of 198.1.24.1/29 on SW1-2 interface Ethernet3/14 and
198.1.24.2/29 on SW1-4 interface Ethernet3/22
6. Use this Layer 3 port as a source to create a Layer 2 connection using OTV
7. Use VLAN 203 as the site-vlan on both OTV sites
8. Use a site number of 12 in DC1 and 14 in DC2
9. Use a solution which utilizes multicast, you are free to choose Multicast IP addressing
10. At the end of the task the SVIs created for VLAN 201 and 202 should be able to ping each
other
4 points
10
Task 11: DC2 VLANs

VLAN
Name
Device
USERS
SW1-3, SW1-4
124
USER2
SW1-3, SW1-4
301
UCS1
SW1-3, SW1-4
303
UCS2
SW1-3, SW1-4
123
1. Create the VLANs using the names stated in the table above.
1 point
Task 12: FabricPath #2

1. Finish the configuration of Task 9
2. Enable FabricPath on the interfaces Ethernet4/17 to 4/20 on SW1-3 and Ethernet4/21 to
4/23 on SW1-4.
3. Allow VLAN 123 and VLAN 124 to cross between DC1 and DC2
4. FabricPath adjacencies should be terminated when no hello packets are received for 12
seconds
5. The lowest numbered link should always be preferred from DC1 to DC2, where traffic from
DC2 to DC1 should use the highest numbered link
6. Configure Switch IDs according to the Switch number (SW1-3 is SID 13)
11
7. FabricPath authentication using a text password of FPauth should automatically be

enabled on all current links and automatically on future links
3 points
Task 13: QoS

1. Ensure Jumbo frames are allowed on all links in DC2
2. Configure a Gold class with a bandwidth reservation of 20% mapping CoS value 1 and 2
3. Configure a Silver class with a bandwidth reservation of 35% mapping CoS value 4 and 5
3 points
Task 14: Port-Channel

1. Configure the links to the UCS Fabric Interconnects to be a port-channel
2. Configure Ethernet4/5 on SW1-3 in Port-channel45 and Ethernet4/6 on SW1-4 in Port-
channel46
3. Ensure the links transition to forwarding state immediately
4. Only allow the FabricPath VLANs and the DC2 VLANs
2 points
12
13
Section 2
Storage Networking
Task 1: FCoE
1. Ensure that the C-series server connected to Ethernet1/15 on SW2 and SW3 can set-up an
FCoE connection towards the switches
2. Configure VSAN 2000 named FCoE_VSAN1 on SW2 and VSAN 2001 on SW3
3. Use VLAN numbers equal to the VSAN number
4. Use interface vfc1 on SW2 and vfc2 on SW3
5. Configure a multi-hop FCoE connection across the trunk between SW2 and SW3 where
VSAN 188 and VSAN 299 will be trunked between the 2 switches. Use VLAN numbers equal
to the VSAN numbers plus 2000 (example: VSAN 100 = VLAN 2100)
2 points

Task 2: JBOD
Host
VSAN
Name
MDS1
188
ML2_VSAN1
MDS1
299
ML2_VSAN2
MDS2
188
ML2_VSAN1
MDS2
299
ML2_VSAN2

1. Create VSAN 188 and VSAN 299 on both MDS switches
2. Configure VSAN assignments according to the table above
1 point
Task 3: Access interfaces #1

1. Ensure that JBOD1 is configured in VSAN 188
2. JBOD2 should be configured in VSAN 299
1 point
14

Task 4: ISL
1. Interface fc1/13 and fc1/14 on MDS2 should be configured to run as a single logical link to
fc1/31 and fc1/32 on SW3.
2. Enable a protocol to negotiate the status of the port-channel
3. Only allow the 2 VSANs previously created
4. Ensure you see the FCNS entries of JBOD1 on SW2 and SW3 in VSAN 299
2 points
Task 5: FC security
1. Ensure that all switches in the network authenticate each other on all E-port links
2. The switch should use an SHA-1 hash of <hostname>securehash, for example:
MDS1securehash
3. Ensure that all E-ports are enforcing authentication before coming online
3 points

Task 6: FCIP
Host
IP address
Subnetmask
VLAN
MDS1
198.18.111.1
255.255.255.128
1111
MDS2
198.18.111.2
255.255.255.128
1111
MDS1
172.22.12.101
255.255.255.0
1012
MDS2
172.22.12.102
255.255.255.0
1012

15
1. Configure SW2 and SW3 to enable communication in the mentioned VLANs above.
2. Use the first GigabitEthernet connection for VLAN 1111 traffic which should include a
802.1Q tag and the second GigabitEthernet connection for VLAN 1012, where the MDS
switches will send traffic without a 802.1Q tag.
3. Ensure the switch ports (Ethernet1/11 and Ethernet1/12) transition into forwarding
immediately
4. Use IP addressing as the table provides
5. Ensure that the failure of a single GigabitEthernet connection and therefore FCIP tunnel, will
not cause an FSPF re-calculation
6. Allow both VSAN 188 and VSAN 299 on this link
7. Ensure that R_RDY frames are sent locally by the MDS switch to enhance the performance
of write actions
5 points

Task 7: Zoning
1. Create a device-alias for each FC target currently present.
2. Create a device-alias for each UCS initiator based on the UCS pool section
3. The device-alias database may only be configured from MDS1. Ensure all other switches
contain the same copy of the device-alias database.
4. Ensure device-alias names will be kept in the zoning configuration and will not be
overwritten by the WWPN.
5. Ensure that zoning is created to support the UCS section. Create a separate zone per
initiator and target. (Initiators will be known from the UCS section).
6. The target which should be used in the UCS zoning is the disk with WWPN:
22:00:00:11:c6:a6:27:4c and 21:00:00:11:c6:a6:27:4c
7. Use zones with ML2 in the name
8. Points in this task are only awarded if the zoning works successfully in the UCS section as
well
4 points
16
1. Prepare fc1/9 and fc1/10 on both MDS switches to be access ports for the UCS system
which will run in End-Host Mode.
2. These connections should be bundled into a single logical connection
3. VSAN 188 should be the native VSAN
4. Use number 102 on MDS1 and number 103 on MDS2 for this connection
5. Both VSAN 188 and 299 should be able to have a connection on both Fabric Interconnects
2 points
17
Section 3
Unified Computing
Task 1: Chassis initialization

1. Assign ports 1, 3, 5 and 7 on both Fabric Interconnects to be Server Ports
2. Chassis should be discovered with any link configuration, but should use all 4 links when
fully initialized and use the best load-balancing possible when supported by the chassis IO
modules
3 points

Task 2: VLANs and Uplinks
1. Create VLANs according to the DC2 VLAN list which has been previously configured on the
SW1-3 and SW1-4 interfaces
2. Create port-channels for the Ethernet uplink traffic on the UCS system
3 points
Task 3: VSANs and Uplinks

1. Create VSAN 188 and VSAN 299 on both Fabric Interconnects
2. Configure the port VSANs according to the task in the Storage section
3. Create port-channels for the Fibre Channel uplink traffic on the UCS system
3 points
18
Task 4: Pools
Name
Prefix
Size
Pool
WWPN
DC_WWPN_A
20:00:00:25:B5:A0:00:00
WWPN
DC_WWPN_B
20:00:00:25:B5:B0:00:00
WWNN
DC_WWNN
20:00:00:25:B5:00:00:00
MAC
DC_MAC_A
00:00:25:B5:AA:00
25
MAC
DC_MAC_B
00:00:25:B5:BB:00
25
UUID
DC_UUID
Default
Management
n/a
10.10.210.91/24 (GW: 10.10.210.254)

1. Create pools according to the table above
2 points
Task 5: Server pools

1. Create a server pool which automatically adds all servers based on their memory
2. Create a server pool which automatically adds all servers with a Cisco M81KR mezzanine
card
2 points
19
Task 6: vNIC template

1. All vNICs in the system should be based on templates which always update the service
profile also after associating to the Service Profile
2. Create a vNIC template for Management traffic. This traffic will be send untagged from the
blade. Map this traffic to VLAN 301. Ensure this vNIC is used primarily on Fabric A, but when
all uplinks fail, the blade will not notice a failover to Fabric B
3. Ensure that this traffic is marked as Gold traffic with CoS value 1. Ensure the UCS system
compliments the QoS settings in the switches
4. Create 2 vNIC templates for Fabric A and Fabric B without supporting failover. Allow all DC2
VLANs on this template. Be sure that traffic previously marked with CoS values is trusted on
these vNICs
5. Be sure to use the MAC Pools accordingly as previously created
4 points

Task 7: vHBA template
1. Create a vHBA template which only applies settings when its applied to the service profile
for Fabric A in VSAN 188
2. Create another vHBA template which updates the settings when changed to all associated
service profiles for Fabric B in VSAN 299.
3. Be sure to use the WWN Pools accordingly as previously created
2 points

Task 8: Policies
In this task policies will be created, which will be applied when the building the Service Profile
1. The local disks in the blade should not be used and should not be changed, create a policy
for this
2. Make sure that a blade runs the latest UCS firmware versions, independent of the current
running version
3. Users should acknowledge changes to a Service Profile. You are not allowed to use the
default policy
4. Ensure that disks and BIOS settings are retained on the blade when a Service Profile
association is removed
4 points

20
Task 9: Service Profiles

1.
2.
3.
4.
5.
Create a new service profile which can be re-applied to multiple blades automatically
Settings should only be initially pushed to the Service Profile
Use pool assignments from previously created pools wherever possible
Create 3 vNICs. Based on the previously created vNIC templates
Create a new boot policy to support Boot from SAN. The UCS should boot primarily on
WWPN: 22:00:00:11:c6:a6:27:4c across Fabric A. If this WWPN is not available it should use:
21:00:00:11:c6:a6:27:4c across Fabric B.
6. Configure zoning on the MDS switches based on the task in the Storage section.
7. Ensure all policies from the previous task are applied
8. Do not assign the profile template yet
6 points
Task 10: Server boot

1. Create sufficient Service Profiles to allocate 3 servers. Use a naming convention of ML2-SP
2. Successfully associate the Service Profile and ensure the servers are booted
2 points
Task 11: Cloning
1. Clone one of the previously generated Service Profiles to a new profile to support the fourth
server.
2. Change the Service Profile so it supports a configuration where the UCS blade would not
be a Cisco VIC card. This profile should use local storage to boot from.
3 points
Task 12: Management

Setting
Value
IP address
10.10.210.222
Bind DN
cn=bind_user, cn=users, dc=cciedc, dc=local
Base DN
cn=users, dc=cciedc, dc=local
Filter
sAMAccountName=$userid
Password
IPexpert123
Group Authorization
Yes
Group Recursion
Yes

1. Configure Active Directory based authentication according to the table above.
2. There is no Domain Controller in the lab, but assume there is one
21
3. Users should be able to select Active Directory authentication when logging in to the UCS by
using the dropdown box. By default authentication should use the local database
Group
Mapping
DomainAdmins
admin
ServerAdmins
server-equipment, server-profile,server-security
StorageAdmins
storage
NetworkAdmins
network

1. Map the groups to the roles according to the table above
6 points
22
CCIE Data Center Mock Lab Challenge Chapter 22

Challenge
This lab is heavily layer 2, UCS and Storage focused and is designed to be quite challenging in
these particular areas. It covers most of the configuration you can do with vPC and FabricPath.
The lab is also quite long so allow enough time to do all the setup activities and then still have 8
hours remaining to complete the lab. If you can finish this lab within the allocated time you will
know you have good speed!
Just like the real lab, some topics from the CCIE DC Blueprint may be missing from this lab: its
important in your preparation that you attempt multiple Mock Labs so that you cover the entire
range of topics and are truly the expert the CCIE DC requires you to be.
General Rules
Basic IP addressing, switching configuration and storage has already been pre-configured
for you
Troubleshooting is a HUGE part of this practice lab! If you can troubleshoot, during your real
exam you will have a much better chance of passing.
The tasks in this lab can be completed but you will need to troubleshoot if you run into
problems as there are errors in the initial configuration.
NOTE: Static/default routes are NOT allowed unless otherwise stated in the task
NOTE: Do not create VLANs on devices not specified for those VLANs
Estimated Time to Complete:
8-10 Hours
Pre-setup
This lab is intended to be used with online rack access provided by our partner Proctor Labs
(www.proctorlabs.com).
Please log in to your Data Center vRack at ProctorLabs.com
A file should be available with this workbook in your eBooks/Download section of your
ipexpert.com login. The file is called InitialConfigLab22.txt. Follow the instructions in this
file to load the initial configuration.


1.0 Data Center Configuration
(32 points)
Note: Ensure you have loaded the initial setup as per the Pre-setup instructions
Task 1.1: Initial Setup (2 Points)

Configure the switches with the following VLANs and be sure to name them as per the table
below


VLAN
110
Switch
SW1-1,SW1-2,SW2,SW3
Name
AcmeCorp-Data
120
SW1-1,SW1-2,SW2,SW3
AcmeCorp-Voice
130
SW1-1,SW1-2,SW2,SW3
AcmeCorp-DMZ
210
SW1-1,SW1-2, SW2,SW3
MegaCorp-Data
220
MegaCorp-Voice
230
MegaCorp-DMZ
500
SW1-1,SW1-2,SW1-3,SW1-4
Spine1
600
SW1-1,SW1-2,SW1-3,SW1-4
Spine2
10
SW1-1,SW1-2,SW1-3, SW1-4,SW2,SW3
NFS
100
SW1-1,SW1-2,SW1-3, SW1-4,SW2,SW3
iSCSI-Network
Task 1.2: L3 Initial configuration (2 Points)
Configure the following L3 Interfaces
VLAN
100
Switch
SW1-3
IP Address
10.0.100.1/24
10
SW1-4
10.0.10.1/24
110
SW2
10.100.10.1/24
210
SW3
10.200.10.1/24

Task 1.3: vPC Configuration (3 Points)
Configure vPC between SW1-1 and SW1-2 using only the following interfaces for the vPC
peer link

Switch
SW1-1
Interface
Eth3/9
SW1-2
Eth3/10
Switch
SW1-1
Interface
Eth3/11
SW1-2
Eth3/12

Use any IP addressing information you desire for this keepalive link, but ensure it is
located within its own dedicated VRF. Name the VRF IPExpertVRF
Ensure that in the event of both switches failing, but only one rebooting successfully and
turning on successfully that after 240 seconds the switch will restore vPC functionality.

You may use any vPC domain ID you choose.

Configure a keepalive mechanism between SW1-1 and SW1-2 using a dedicated L3
interface on each switch as per the table below

Configure vPC between SW2 and SW3 using a domain ID of your choosing.
Use mgmt0 for keepalive mechanism
Ensure SW2 is the vPC Primary
Use all available links between SW2 and SW3 for the vPC Peer link.
Configure a back to back vPC from Sw2 and SW3 to SW1-1 and SW1-2
Ensure that this back to back vPC forms port channels using a negotiation protocol

Task 1.5: FabricPath Configuration (6 Points)
Configure SW1-3 and SW1-4 for fabric path and enable Fabric Path on the interfaces
connecting these two switches
Configure Fabric Path on SW1-2 and SW1-1, ensuring all F-Line-card ports facing
towards SW1-3 and SW1-4 are enabled for fabric path
To make identification of these switches easier, ensure the switches are assigned the
following Switch IDs:

Switch
SW1-3
Switch-ID
130
SW1-4
140
SW1-2
120
SW1-1
110

The following VLANs should be set to FabricPath VLANs

Switch
500
Switch-ID
FabricPath
600
FabricPath
100
FabricPath
10
FabricPath

SW1-1 and SW1-2 are the leaf switches in this configuration, configure spanning-tree as
appropriate in such a design bearing in mind that SW1-1 and SW1-2 are vPC Peers and
that we want to avoid any STP convergence issues should the vPC primary switch fail
(I.E. Both switches should be sending BPDUs)
All areas of FabricPath should be authenticated including Adjacencies and updates using
the key CCIEDC-IPEXPERT

Task 1.6: FabricPath Traffic Engineering (4 Points)
The E4/19 and E4/11 interface on SW1-3 and SW1-1 respectively is a high-cost link that
should not be used if the E4/20 and E4/12 link is available, use traffic engineering to
meet this requirement
Ensure that the broadcast traffic tree used by Fabric Path is rooted at SW1-4 switch.
Task 1.7: vPC enhancement configuration (4 Points)
Configure the following ports On SW2 and SW3 to face down towards the Cisco UCS FI,
each one will act as a separate uplink and thus should not be configured as a port
channel.

Port
E1/9
VLAN(s)
110,120,130,10,100
SW3
E1/9
110,120,130,10,100
SW2
E1/10
210,220,230,10,100
SW3
E1/10
Switch
SW2
210,220,230,10,100
Ensure that all ports transition to the forwarding spanning-tree state as quickly as
possible as the Cisco UCS will not send any BDPUs
Ensure that SW2 and SW3 never allow their L3 VLAN 110 and VLAN 210 interfaces to go
into the down state in the event of a VPC peer link failure.
Ensure that if SW3 was to lose its peer link to SW2 and suspend its vPC member ports
that it would also in turn suspend its ports down to the FI so that the FI would know to
use fabric A.
Task 1.8: FEX Configuration (3 Points)
After careful consideration of the Pros and Cons of eVPC and standard vPC, you have
chosen not to implement eVPC
Configure the FEXs attached to SW2 and SW3 as per the table below
Switch
SW2
Port
Eth1/13
FEX
FEX 192
SW3
Eth1/14
FEX 193
Ensure each FEX has a description, ### FEX 1XX ### where X is the FEX number
Task 1.9: vPC Member Port (3 Points)
Configure a vPC port channel down to the Cisco C-Series Server from port 1/15 on SW2
and SW3. This port channel should use no negotation to bring up this port channel
This Server provides some NFS functionality, so thus should carry the NFS VLAN Only
ensuring this VLAN is untagged.
This port should be configured to bypass listening and learning for Spanning-tree as a
server port should be.
Task 1.10: Access Ports (3 Points)
Configure port E1/11 on SW2 and SW3 for VLAN 100.

Ensure the ports are set to bypass listening/learning
Ensure the ports are untagged for this VLAN
Ensure all traffic is tagged with a CoS value of 4
2.0 Storage Configuration
(25 points)
Task 2.1: Initial VSAN Configuration (2 Points)
Configure the following VSAN/VLANs on the respective switches
Switch
MDS1
VSAN
310
VLAN
N/A
MDS1
320
N/A
MDS2
410
N/A
MDS2
420
N/A

Task 2.2: Trunking Port Channel (3 Points)
Configure a E SAN-Port Channel Trunk between MDS 1 and SW2 using the table below
MDS1
Fc1/13
SW2
Fc1/31
SAN-Port-Channel-Number
113
Fc1/14
Fc1/32
114
Ensure this is a trunking E port

Verify this port channel is up and trunking correctly.
Hint: (highlight this section to show hint (Can we do that?) You are allowed to make any
changes to the default configuration necessary to bring this port channel up.
Task 2.3: JBOD Configuration (3 Points)
The JBOD Ports have been preconfigured for you
You will be implementing boot from iSCSI for the ACME blade servers, ensure that JBOD
1 is in VSAN 310 for MDS 1 and 410 for MDS2, and JBOD 2 is in VSAN 320 for MDS1 and
420 for MDS2.
Task 2.4: E-Port traffic engineering (4 Points)
Configure two E Ports between MDS 2 and SW3

MDS2
SW3
Fc1/13
Fc1/31
Fc1/14
Fc1/32
Configure the above so that port 13 and 31 carry VSAN 410 traffic primarily (with VSAN
420 as backup) and ports 14 and 32 carry VSAN 420 primarily (with VSAN 410 as backup)
Task 2.5: iSCSI implementation (6 Points)

Configure iSCSI on GI1/1 on MDS1 and MDS2 respectively
Configure static targets as per the table below
VSAN
310
Target PWWN
22:00:00:11:c6:a6:24:4c
IQN
iqn.2013-10.com.ipexpert:vsan310
410
21:00:00:11:c6:a6:24:4c

Use the following IP addressing information on Gi1/1 on each switch.

Switch
IP Address

10.0.100.10/24
MDS1

10.0.100.20/24
MDS2

Configure the following iSCSI initiators with system-assigned pWWNs
Switch
MDS1
IQN
iqn.2013-10.com.ipexpert:init1a:3
MDS2
Task 2.6: FCoE Server Port (4 Points)
Configure an FCoE Connection from N5k1 and N5k2 down to the C Series server
connected on port 1/15 on each switch. Keeping in mind the separation of fabrics.
The vFC should be configured in such a way that it does not rely on the port-channel
being UP in order for the server to correctly login to the fabric.
This should carry vsan 310 on SW2 and 410 on SW3 respectively.
Task 2.7: Zoning (3 Points)
Based on the IQNs created above, create the following zones on MDS1 and MDS2 using
basic zoning, be sure to use the iQN symbolic node names in your zoning.
Configure a zone called VSAN310_Zoneset in VSAN 310 with the following Zones and
Members
Zone Name
VSAN310_Zone_Blade1

Members
WWPN 22:00:00:11:c6:a6:24:4c
IQN iqn.2013-10.com.ipexpert:init1a:3

Members
Zone Name
VSAN410_Zone_Blade1

10
Members
WWPN 21:00:00:11:c6:a6:24:4c
11
3.0 UCS Configuration
(43 points)

As a cloud services provider, your UCS infrastructure is a common resource between multiple
companies, the UCS configuration below is based on the idea that the infrastructure is shared.
Keep this in mind with all questions and solutions.

Task 3.1: Uplink/Server port configuration (3 Points)
Configure the following ports as Uplink ports

Switch
FI-A
Port
9
FI-A
10
FI-B
FI-B
10
Configure the following ports as Server ports.

Switch
FI-A
Ports
1,3,5,7
FI-B
1,3,5,7
Task 3.2: VLAN Configuration (2 Points)
Configure the following VLANs on UCS

VLAN
110
Name
AcmeCorp-Data
12
120
AcmeCorp-Voice
130
AcmeCorp-DMZ
210
MegaCorp-Data
220
MegaCorp-Voice
230
MegaCorp-DMZ
10
NFS
100
iSCSI-Network
Task 3.3: Disjoint L2 (5 Points)
In order to keep the network traffic separated for MegaCorp and AcmeCorp, configure a
disjoint L2 domain, VLANs 110-130 should travel over the Port 9 uplink on FI-A and FI-B.
VLANs 210 230 should travel over Port 10. The NFS and iSCSI networks are a shared
resource and thus can travel across both uplinks.
Your junior engineer does not understand the concept of designated receiver and its
impact on network traffic, login to the Cisco CLI and run the command to show the
designated receiver for VLAN 110. Save this command and its output as a notepad file
on your desktop.

Task 3.4: SAN Connectivity (6 Points)
Although SAN Connectivity is not required for initial deployment, MegaCorp have
requested you provision the network in preparation for SAN Connectivity in the near
future. The ports on the FI are Ports 2/1 and 2/2 and the ports on the MDSs are FC1/9
and FC1/10
Configure the following VSANs and VLANs on Cisco UCS, Where VSAN 310 and 410 are
used by the AcmeCorp, and VSANs 410 and 420 are used by MegaCorp.
VSAN
310
Mapped VLAN
310
Fabric
FI-A
320
320
FI-A
13
410
410
FI-B
420
420
FI-B
The storage uplinks between the FIs should be able to handle multiple VSANs, they
should also be configured as a SAN-Port-Channel in order to provide the highest possible
bandwidth.
Your junior engineer often has difficulty setting up a SAN Port channel from UCS to
other storage devices, this is often because he does not know what configuration Cisco
UCS will place onto the SAN Port channel when configured from the GUI, show him the
commands required on the UCS CLI to see the configuration applied to your SAN port
channels and paste the output into notepad, then save on your desktop.
Task 3.5: Pool Configuration (3 Points)

Two organizations must be created within Cisco UCS, AcmeCorp and MegaCorp, create
these two organizations and then assign the following UUID, MAC address, WWPN and
WWNN Pools
Organization
AcmeCorp
Pool Type
Mac
Pool Name
MAC_POOL
Value
00:25:B5:00:00:00
Size
32
AcmeCorp
UUID
UUID_POOL
Derived (Prefix)
32
Suffix (000A-000000000001)
AcmeCorp
IQN
IQN_POOL
Prefix: iqn.2013-10.com.ipexpert
Block: init1A
Start with: 0
AcmeCorp
Iscsi Initiator
N/A
10.0.100.100-10.0.100.131/24 (GW:
10.0.100.1) (DNS: N/A)
32
14
Task 3.6: Jumbo MTU Support (6 Points)
Both iSCSI and NFS, like FC traffic are crucial bits of storage traffic that should be
assigned a class that implements Pause frames and their MTU should be able to reach
the maximum allowed on the nexus platform. Assign to Class 4 CoS 4.
The north Nexus 5k Switches from the FI should support this configuration.
Continue up the storage network and implement this configuration all the way to MDS1
and MDS2. Our final goal will be to ensure that our iSCSI and NFS vNICs on our server
blades are able to connect to the 10.0.100.10 and 10.0.100.20 iSCSI Target Portal IP
addresses with an MTU of 9216 with no fragmentation (dont forget about IP overheads,
so exact value may not be 9216). You are allowed to make all necessary changes to L3
and L2 MTU configuration.
Task 3.7: vNIC Template (4 Points)
Create a vNIC template for iSCSI and NFS for AcmeCorp Only
These templates should not be configured for a method of failover that is transparent to
the operating system: storage traffic should utilize a separate Fabric A/Fabric B
configuration.
Name these templates iSCSI-vNIC-A and NFS-vNIC-A for Fabric A, iSCSI-vNIC-B and NFS-
vNIC-B for Fabric B.
VLAN 100 should be native VLAN for iSCSI and VLAN 10 is native for NFS
These vNICs should support Jumbo MTUs.
The Template should be configured in such a way that changes to the template at a later
date are not reflected on vNICs that were created based off the template.

Task 3.8: Description Support (2 Points)
The Physical Server Blade 1 was purchased by AcmeCorp. In order to easily show this fact,
ensure the GUI reflects this as per the screenshot below:
15
Task 3.9: Service Profile Configuration (4 Points)
Create a service profile called iSCSIBlade under the AcmeCorp organization using the
pools assigned previously
The vNIC templates should be utilized in the creation of the iSCSI NIC as per the table
below
vNIC
iscsi-A
Template
iSCSI-vNIC-A
nfs-A
NFS-vNIC-A
iscsi-B
iSCSI-vNIC-B
nfs-B
NFS-vNIC-B
Do not create any vHBAs

Ensure your server uses a local disk configuration policy that can only be applied to
servers with enough disks to support RAID 0.
Read the below section which relates to boot from SAN and configure the iSCSI overlays
as part of the service profile

Task 3.10: Boot from SAN (5 Points)
Configure two iSCSI overlays to be used for boot from SAN

Name the overlays iscsioverlay-A and iscsioverlay-B respectively
Create a boot from SAN iSCSI Policy called iSCSI-Boot
16
Assign this boot policy to your service profile and make the necessary iSCSI parameter
changes, the destination static target should be iqn.2013-10.com.ipexpert:vsan310 for
Fabric A and iqn.2013-10.com.ipexpert:vsan410 for Fabric B
Please note the server will not boot a copy of ESX, you do not have to successfully boot the
server into an operating system, just prepare the server so that it will install to a SAN disk
and boot from SAN in the future.

Task 3.11: Locale Implementation (3 Points)
Create a Locale called AcmeLocale for AcmeCorp and a Locale Called MegaLocale for
MegaCorp
Create an admin user for AcmeCorp called AcmeAdmin and a user for Megacorp called
MegaAdmin
Ensure these users only have access to the appropriate locales.
17

Challenge 2

Chapter 21: Mock Lab Challenge 2 is the second of 3 mock lab challenges that will test you on
all aspects of the CCIE Data Center Blueprint. This first lab will have an equal difficulty level as
the actual lab to get you familiar with the set-up and all aspects involved.
We highly recommend creating your own diagram at the beginning of each lab so you are able
to draw on your own diagram, making it much easier when you step into the real lab.
Multiple topology drawings are available for this chapter.
General Rules
Try to diagram out the task. Draw your own connections the way you like it
Create a checklist to aid as you work thru the lab
Take a very close read of the tasks to ensure you dont miss any points during grading!
Monitor your time. This is a Mock Lab. Verify how many points you earn in a given time
frame
Partial credit is not given. Any task should be completed 100% to receive credit
You require a score of 80 out of 100 points to have a passing score
Estimated Time to Complete: < 8 hours
Copyright 2013 by IPexpert. All rights reserved.
Solutions
In this chapter we are working on the second of 3 Mock Lab Challenges that this workbook
contains. This mock lab challenge will simulate a full CCIE Data Center Lab experience. This first
lab has a difficulty level which is similar or a little lower than that of the CCIE Data Center lab.
Still this is a tough lab and you will need to work on a lot of different tasks and keep an
absolute close eye on the wording of the tasks. Be sure to read the whole task before starting
with the configuration or you will be needing to go back and change topics.
The devices have a little configuration loaded on them already to make the initial configuration
easier. Be very careful as the configuration might have errors in it that you will be forced to
correct. This might cost precious time while doing this mock lab.
When you are progressing through the tasks you will see that there will be small drawings in
the text to help you. Pay close attention to the task itself as the text is always leading. The
diagram is only there to help you.
Try to measure the time it takes you to finish sections and the whole lab so you get a good
understanding on which part you need to study more. You should be able to finish a full scale
lab like this in like 6 hours to have enough time to go back and re-read the tasks and your
questions. When you are rushing through the tasks you will not always be sharp to answer the
question 100% correct. Know that you will not get any partial credit for any task so you need to
be absolutely sure that your answer is correct otherwise the points are not given to you.
Section 1
Data Center Networking
Task 1: VDC allocations

The first task is about verifying the pre-configuration on the Nexus 7000. It is pre-configured
with 4 VDCs which have a port allocation configured. The lab states a certain port allocation
which should be properly checked when starting with the lab.
Its a very good practice in checking all the configurations for all the devices you are faced with
in the lab. Some might have pre-configuration and some will not.
SW1-1
vdc combined-hostname
vdc SW1-1 id 1
limit-resource module-type m1 f1 m1xl
allow feature-set fabricpath
allocate interface
Ethernet3/1,Ethernet3/3,Ethernet3/5,Ethernet3/7,Ethernet3/9
,Ethernet3/11,Ethernet3/13,Ethernet3/15,Ethernet3/25-32
allocate interface Ethernet4/1-2,Ethernet4/9-12,Ethernet4/25-32
limit-resource vlan minimum 1 maximum 2
limit-resource monitor-session minimum 0 maximum 2

limit-resource monitor-session-erspan-dst minimum 0 maximum 23
limit-resource vrf minimum 2 maximum 4096
limit-resource port-channel minimum 0 maximum 768
limit-resource u4route-mem minimum 96 maximum 96
limit-resource m4route-mem minimum 58 maximum 58
vdc SW1-2 id 2
allocate interface
Ethernet3/2,Ethernet3/4,Ethernet3/6,Ethernet3/8,Ethernet3/1
0,Ethernet3/12,Ethernet3/14,Ethernet3/16
allocate interface Ethernet4/3-4,Ethernet4/13-16
boot-order 1
3

vdc SW1-3 id 3
allocate interface Ethernet3/17,Ethernet3/19,Ethernet3/21,Ethernet3/23
boot-order 1
vdc SW1-4 id 4
allocate interface Ethernet3/18,Ethernet3/20,Ethernet3/22,Ethernet3/24

boot-order 1
We see that some things are not properly configured where we could run into serious problems
if not fixed. Besides the fact that we loose points.
SW1-1
no vdc combined-hostname
vdc SW1-1 id 1

vdc SW1-2 id 2
vdc SW1-3 id 3
vdc SW1-4 id 4
We see a combined-hostname command, which means that our switch names will not match
our drawings and our tasks. By configuring the NO command, only the VDC name will be used
as the hostname for that particular VDC.
Next we also see a very limited resource being allocated to the amount of VLANs that can be
created per VDC. This should be changed as we will be configuring much more than 2 VLANs
probably. If this would not be changed, we would receive an error message when trying to
configure more than 2 VLANs in any given VDC.
We also verify the other (if existing) pre-configurations and we do not see any errors there. Pay
close attention to the port allocations of the VDCs.

Task 2: DC 1 VLAN
In this next task we start configuring our VLANs on the locations that we need them to be.
This is not a difficult task, but pay close attention to the numbers, names and which switches
you configure them on as you could not only loose this single point. This might result in loosing
all the points for this section if its related to a certain VLAN!
SW1-1
SW1-1(config)# vlan 123
SW1-1(config-vlan)# name USERS
SW1-1(config-vlan)# vlan 124
SW1-1(config-vlan)# name USERS2
SW1-1(config-vlan)# name SERVERS
SW1-1(config-vlan)# name SERVERS2
SW1-2
5

SW1-2(config-vlan)# name SERVERS
SW1-2(config-vlan)# name SERVERS2
SW2
SW2(config)# vlan 123
SW2(config-vlan)# name USERS
SW2(config-vlan)# vlan 124
SW2(config-vlan)# name USERS2
SW2(config-vlan)# name SERVERS
SW2(config-vlan)# name SERVERS2
SW2(config-vlan)# name EIGRP
SW2(config-vlan)# name VRRP
SW3
SW2(config-vlan)# name FCIP

SW3(config-vlan)# name USERS
SW3(config-vlan)# name USERS2
SW3(config-vlan)# name SERVERS
SW3(config-vlan)# name SERVERS2
SW3(config-vlan)# name EIGRP
SW3(config-vlan)# name VRRP
SW3(config-vlan)# name FCIP
Verify that the VLANs are properly created and that they are in the VLAN database.

SW1-1(config)# show vlan
VLAN Name
Status
Ports
---- -------------------------------- --------- ------------------------------1
default
active
Eth4/1, Eth4/2, Eth4/9, Eth4/10

Eth4/11, Eth4/12, Eth4/25
Eth4/26, Eth4/27, Eth4/28
Eth4/29, Eth4/30, Eth4/31
Eth4/32
123
USERS
active
124
USERS2
active
125
SERVERS
active
126
SERVERS2
active
VLAN Type
Vlan-mode
---- ----- ---------enet
CE
123
enet
CE
124
enet
CE
125
enet
CE
126
enet
CE
Task 3: Trunk interfaces

Next we configure our trunk link between the 2 Nexus 5000 switches in the topology. Pay
attention to only allow the VLANs that are required according to the mock lab VLAN list.
Another thing to think about is to enable Spanning-Tree Bridge Assurance.
Lets first configure the trunk connection between the 2 switches.
SW2
SW2(config-if)# int e1/5-6
SW2(config-if-range)# sw mode trunk
SW2(config-if-range)# spanning-tree port type network
SW2(config-if-range)# sw trunk allowed vlan 123,124,125,126,1011,1012,1111
SW3
SW3(config-if)# int e1/5-6
SW3(config-if-range)# sw mode trunk
SW3(config-if-range)# spanning-tree port type network

SW3(config-if-range)# sw trunk allowed vlan 123,124,125,126,1011,1012,1111
!Command: show running-config interface Ethernet1/5-6
!Time: Sun Oct 13 12:35:47 2013
version 5.1(3)N1(1)
switchport mode trunk
switchport trunk allowed vlan 123-126,1011-1012,1111
spanning-tree port type network
switchport trunk allowed vlan 123-126,1011-1012,1111
spanning-tree port type network
Next we should make sure that Ethernet1/5 will go into blocking state when this is required by
Spanning-Tree calculations. Normally we would fix Spanning-Tree traffic engineering using the
cost metric, but in this case we are using multiple connections between the same physical
switches. Therefore we need to be using port priority instead of cost.
SW3(config)# show span vlan 123

VLAN0123
Spanning tree enabled protocol rstp
Root ID
Bridge ID
Interface
Priority
32891
Address
002a.6a1a.7c41
Cost
2000
Port
133 (Ethernet1/5)
Hello Time
sec
Max Age 20 sec
Priority
32891
Address
002a.6a1f.de81
Hello Time
sec
Role Sts Cost
Forward Delay 15 sec
(priority 32768 sys-id-ext 123)

Max Age 20 sec
Prio.Nbr Type
---------------- ---- --- --------- -------- -------------------------------
Eth1/5
Root FWD 2000
128.133
Network P2p
Eth1/6
Altn BLK 2000
128.134
Network P2p
By default the lower priority is better, therefore in this case Ethernet1/5 would always win the
election and Ethernet1/6 will be blocking traffic.
SW3
SW3(config)#
SW3(config)# int e1/5
SW3(config-if)# spanning-tree vlan 1-4094 port-priority ?
<0-224>
Port priority in increments of 32
SW3(config-if)# spanning-tree vlan 1-4094 port-priority 200

ERROR: % Port Priority in increments of 32 is required
Allowed values are:
0
32
64
96
128
160
192
224
SW2
Just like with the Spanning-Tree priority the port-priority needs to be configured in certain
increments. In this case its increments of 32. Therefore we configure our port priority to be
higher, meaning to make Ethernet1/6 more interesting in the election.
SW3(config-if)# show span vlan 123
VLAN0123
Root ID
Bridge ID
Interface
Priority
32891
Address
002a.6a1a.7c41
Cost
2000
Port
134 (Ethernet1/6)
Hello Time
Priority
32891
Address
002a.6a1f.de81
Hello Time
sec
sec
Role Sts Cost
Max Age 20 sec

Max Age 20 sec
Prio.Nbr Type

---------------- ---- --- --------- -------- ------------------------------Eth1/5
Altn BLK 2000
192.133
Network P2p
Eth1/6
Root FWD 2000
128.134
Network P2p
After a link flap, the new port is now elected to be the new forwarding port and Ethernet1/5 is
now blocking traffic.
Finally we should enable Jumbo frames in our configuration. This is enabled using the global
QoS policy configuration.
By default the normal QoS policy is applied to the Nexus 5000 switches. When we enable the
FCoE features it will activate the FCoE QoS configuration. So because we will be using the FCoE
features later on, we will already enable it here in our configuration.
First take a look at the default policy-map for network-qos. We can then copy and paste that
configuration to ensure we are using a consistent configuration.
SW3# show policy-map type network-qos
Type network-qos policy-maps

===============================
policy-map type network-qos default-nq-policy

class type network-qos class-default
mtu 1500
multicast-optimize
policy-map type network-qos fcoe-default-nq-policy
class type network-qos class-fcoe
pause no-drop
mtu 2158
mtu 1500
multicast-optimize
10
We will be using the FCoE policy to create our own policy as we are not able to change the
default policies.
SW2
SW2(config)# policy-map type network-qos FCOE-JUMBO
SW2(config-pmap-nq)# class type network-qos class-fcoe
SW2(config-pmap-nq-c)# mtu 2158
SW2(config-pmap-nq-c)# pause no-drop
SW2(config-pmap-nq-c)# class type network-qos class-default
SW2(config-pmap-nq-c)# multicast-optimize
SW2(config-pmap-nq-c)# mtu ?
<1500-9216>
MTU value

SW2(config-pmap-nq-c)# exit
SW2(config-pmap-nq)# exit
SW2(config)# system qos
SW2(config-sys-qos)# service-policy type network-qos FCOE-JUMBO
SW2(config-sys-qos)# exit
SW3
SW3(config)# policy-map type network-qos FCOE-JUMBO

SW3(config-pmap-nq)# class type network-qos class-fcoe
SW3(config-pmap-nq-c)# pause no-drop
SW3(config-pmap-nq-c)# class type network-qos class-default
SW3(config-pmap-nq-c)# multicast-optimize
SW3(config-pmap-nq-c)# exit
SW3(config-pmap-nq)# exit
SW3(config)# system qos
SW3(config-sys-qos)# service-policy type network-qos FCOE-JUMBO
SW3(config-sys-qos)# exit
On the Nexus 5000 platform its not possible to configure MTU directly under the interfaces.
After applying the new policy to the system qos section we see the MTU is now set.
SW3(config)# show policy-map system type network-qos
11

===============================
policy-map type network-qos FCOE-JUMBO
match qos-group 1
pause no-drop
mtu 2158
match qos-group 0
mtu 9216
multicast-optimize
SW3(config)#
Task 4: Routing
Next a bit more complicated task is our next configuration. The routing features will be used a
lot and we will see different subjects being tested in this single task.
We will first focus on the IP addressing in this lab. It is very easy to miss a different subnetmask
for example. In your lab everything works, but the task is wrong, because you did not comply to
the rules of the task.
Next the OSPF protocol will be configured and finally the EIGRP protocol where we will need to
configured some redistribution as well.
First the IP addressing. Its a lot of typing and again pay attention to the subnet masks!
SW1-1
SW1-1(config)# int e3/9
SW1-1(config-if)# no sw
SW1-1(config-if)# ip add 198.18.12.1/25
SW1-1(config-if)# no shut
SW1-1(config-if)# int e3/11
12

SW1-2
SW1-2(config-if)#
SW2
SW2(config-if)# no sw

SW2(config-if)# ip add 198.19.12.2/26
SW2(config-if)# no shut
SW2(config-if)# int e1/2
SW2(config-if)# feature interface-vlan
SW2(config)# int vlan 1011
SW2(config-if)#
SW3
13

SW3(config-if)# feature interface-vlan
SW3(config-if)# int vlan 1011
SW3(config-if)# no shutdown
After configuring the IP addresses on all of the interfaces we need to configure the OSPF
network. In this case we need to assign a correct network type to the correct network
interfaces. This means that we need to assign a point-to-point network type on the /30 links
and a broadcast network type on the other links. There we need to make sure the highest
numbered switch will receive the higher OSPF DR priority setting.
SW1-1
SW1-1(config)# feature ospf
SW1-1(config)# router ospf 1
SW1-1(config-router)# int e3/9
SW1-1(config-if)# ip router ospf 1 area 0

SW1-1(config-if)# ip ospf network point-to-point
SW1-1(config-if)#
SW1-2
SW1-2(config)# feature ospf
SW1-2(config)# router ospf 1
SW1-2(config-router)# exit
SW1-2(config-if)# ip ospf priority 200
SW1-2(config-if)# ip ospf priority 200
14

SW1-2(config-if)#
SW2
SW2(config-if)# feature ospf
SW2(config)# router ospf 1
SW2(config-router)#
SW2(config-router)#
SW2(config-router)# exit
SW2(config-if)# ip ospf prio 200
SW2(config-if)# ip router ospf 1 area 0
SW2(config-if)# ip ospf network point-to-point
SW2(config-if)#
SW3
SW3(config-if)# feature ospf
SW3(config)#
SW3(config)#
SW3(config)# router ospf 1
SW3(config-router)# int e1/1
SW3(config-if)#
Next we check if all OSPF adjacencies are up and if the right devices became the DR routers.
SW1-1(config-if)# sh ip ospf nei
OSPF Process ID 1 VRF default
Total number of neighbors: 4
15
Neighbor ID
Pri State
Up Time
Address
198.19.12.2
200 FULL/BDR
00:10:35 198.19.12.2
Interface
Eth3/1

198.19.13.2
1 FULL/ -
00:01:17 198.19.13.2
Eth3/5
198.18.12.2
200 FULL/DR
00:12:17 198.18.12.2
Eth3/9
198.18.12.2
200 FULL/DR
00:12:17 198.18.21.2
Eth3/11
Up Time
Interface
SW1-2(config-if-range)# sh ip ospf nei

Neighbor ID
Pri State
Address
198.19.12.2
1 FULL/ -
00:08:36 198.19.22.2
Eth3/2
198.19.13.2
1 FULL/ -
00:07:44 198.19.23.2
Eth3/6
198.18.12.1
1 FULL/BDR
00:10:26 198.18.12.1
Eth3/10
198.18.12.1
1 FULL/BDR
00:10:26 198.18.21.1
Eth3/12
Up Time
Interface
SW2(config-if)# sh ip ospf nei

Neighbor ID
Pri State
Address
198.18.12.1
1 FULL/DR
00:08:39 198.19.12.1
Eth1/1
198.18.12.2
1 FULL/ -
00:08:32 198.19.22.1
Eth1/2
Up Time
Interface

Neighbor ID
Pri State
Address
198.18.12.1
1 FULL/ -
00:01:43 198.19.13.1
Eth1/1
198.18.12.2
1 FULL/ -
00:10:01 198.19.23.1
Eth1/2
Next we configure our Loopback interfaces and advertise them into OSPF.
SW1-1
SW1-1(config-if)# int lo0
SW1-1(config-if)#
SW1-2
SW1-2(config)# int lo0
SW1-2(config-if)#
16
SW2
SW2(config-if)# int lo0
SW3
SW3(config-if)# int lo0
We verify that the new Loopback addresses are injected into OSPF.
SW3(config-if)# sh ip route ospf
IP Route Table for VRF "default"
'*' denotes best ucast next-hop
'**' denotes best mcast next-hop
'[x/y]' denotes [preference/metric]
198.18.0.11/32, ubest/mbest: 1/0
*via 198.19.13.1, Eth1/1, [110/5], 00:18:14, ospf-1, intra
198.18.0.2/32, ubest/mbest: 2/0

198.18.0.12/32, ubest/mbest: 1/0
198.18.12.0/25, ubest/mbest: 2/0
198.18.21.0/24, ubest/mbest: 2/0
198.19.12.0/26, ubest/mbest: 1/0
198.19.22.0/30, ubest/mbest: 1/0
SW3(config-if)#
17
The final question of our routing task is to configure another routing protocol between SW2
and SW3.
SW3
SW3(config-if)# feature eigrp
SW3(config)# router eigrp 1
SW3(config-router)# int vlan 1011
SW3(config-if)# ip router eigrp 1
SW2
SW2(config)# feature eigrp
SW2(config-router)# int vlan 1011
SW2(config-if)# ip router eigrp 1
SW2(config-if)#
After establishing the adjacency between the 2 switches we will configure our redistribution
which is what we need to finalize the routing task. To ensure we offer redundancy we need to
make sure that all routes of both protocols are available at any time. Now fortunately due to
the nature of the EIGRP protocol we do not need to worry about routing loops.
SW2
SW2(config-if)# route-map PERMIT permit 10
SW2(config-route-map)# exit
SW2(config-router)# redistribute ospf 1 route-map PERMIT
SW2(config-router)# router ospf 1
SW2(config-router)# redistribute eigrp 1 route-map PERMIT
SW2(config-route-map)# router eigrp 1
SW2(config-router)# redistribute direct route-map PERMIT
SW2(config-router)#
SW3
SW3(config-if)# route-map PERMIT permit 10
SW3(config-route-map)# exit
SW3(config-router)# redistribute ospf 1 route-map PERMIT
SW3(config-router)# router ospf 1
SW3(config-router)# redistribute eigrp 1 route-map PERMIT
18

SW2(config-route-map)# router eigrp 1
SW2(config-router)# redistribute direct route-map PERMIT
SW2(config-router)#
What we do need to take care for is that the direct routes (in this case the Loopback address)
needs to be advertised as well in case of a failure. Therefore besides advertising possible OSPF
links, we need to advertise the direct links and we receive EIGRP routes for all OSPF
destinations. Therefore we have a correct working network again in case of a double failure.

SW2(config-if-range)# sh ip route eigrp
198.18.0.1/32, ubest/mbest: 1/0
*via 198.19.223.2, Vlan1011, [170/51456], 00:00:11, eigrp-1, external
198.18.0.12/32, ubest/mbest: 1/0
198.18.12.0/25, ubest/mbest: 1/0

198.18.21.0/24, ubest/mbest: 1/0
SW3(config-router)# sh ip route eigrp

198.18.0.2/32, ubest/mbest: 1/0
SW3(config-router)#
19
Task 5: vPC
Next we will start configuring the Virtual Port-Channel feature. This feature is always
complicated to configure where we really need to focus on the order of operation in how we
enable the feature on the Nexus switches.
SW2
SW2(config-router)# feature vpc
SW2(config)#
SW2(config)#
SW2(config)# vpc domain 5
SW2(config-vpc-domain)# peer-keepalive destination 198.18.0.3 source 198.18.0.2
vrf default
SW2(config-vpc-domain)# role priority 255
Warning:
!!:: vPCs will be flapped on current primary vPC switch while attempting role
change ::!!
Note:
--------:: Change will take effect after user has re-initd the vPC peer-link
::-------system-mac
SW2(config-vpc-domain)# sys
system-priority
SW2(config-vpc-domain)# system-priority 100

SW2(config-vpc-domain)# system-mac 12:34:56:78:ab:cd
SW2(config-vpc-domain)# auto-recovery ?
<CR>
reload-delay
Duration to wait after reload to recovery vPCs
SW2(config-vpc-domain)# auto-recovery reload-delay ?

<240-3600>
Time-out for restoring vPC links (in seconds)
SW2(config-vpc-domain)# auto-recovery reload-delay 300

Warning:
Enables restoring of vPCs in a peer-detached state after reload, will wait for
240 seconds (by default) to determine if peer is un-reachable
SW2(config-vpc-domain)#
SW3
SW3(config)# feature vpc
20

SW3(config)# vpc domain 5
SW3(config-vpc-domain)# peer-keepalive destination 198.18.0.2 source 198.18.0.3
vrf default
SW3(config-vpc-domain)# system-priority 100
SW3(config-vpc-domain)# system-mac 12:34:56:78:ab:cd
SW3(config-vpc-domain)# auto-recovery reload-delay 300
SW3(config-vpc-domain)#
Now we configured the basic parameters for the VPC feature and we verify that the peer-
keepalive which we configured is operational.
SW2(config-vpc-domain)# sh vpc
Legend:
(*) - local vPC is down, forwarding via vPC peer-link
vPC domain id
: 5
Peer status
: peer link not configured
vPC keep-alive status
: peer is alive
Configuration consistency status: failed

Per-vlan consistency status
: failed
Configuration consistency reason: vPC peer-link does not exist

: failed
Type-2 consistency reason
: vPC peer-link does not exist
vPC role
: none established
Number of vPCs configured
: 0
Peer Gateway
: Disabled
Dual-active excluded VLANs
: -
Graceful Consistency Check
: Disabled (due to peer configuration)
Type-2 consistency status
Next we configure the VPC peer-link.

SW3
SW3(config-vpc-domain)# int e1/6
SW3(config-if)# channel-gr 6 mode on
SW3(config-if)# int po6
SW3(config-if)# sw mode trunk
SW3(config-if)# vpc peer-link
Please note that spanning tree port type is changed to "network" port type
on vPC peer-link.
This will enable spanning tree Bridge Assurance on vPC peer-link provided
the STP Bridge Assurance
(which is enabled by default) is not disabled.
21

SW3(config-if)#
SW2
SW2(config-vpc-domain)# int e1/6
SW2(config-if)# vpc peer-link
We verify the vPC peer-link

SW2(config-if)# show vpc
Legend:
vPC domain id
: 5
Peer status
: peer adjacency formed ok
: peer is alive
Configuration consistency status: success

: success
: success
vPC role
: primary
: 0
Peer Gateway
: Disabled
: -
: Enabled
vPC Peer-link status

--------------------------------------------------------------------id
Port
Status Active vlans
--
----
------ --------------------------------------------------
Po6
up
1,1011
SW2(config-if)#
Everything seems operational!

Next we can start configuring the VPCs themselves.
SW2
SW2(config-if)# feature lacp
22

SW2(config-if)# channel-gr 15 mode active
SW2(config-if)# span port type edge trunk
Warning: Edge port type (portfast) should only be enabled on ports
connected to a single
host. Connecting hubs, concentrators, switches, bridges, etc... to this
interface when edge port type (portfast) is enabled, can cause temporary
bridging loops.
Use with CAUTION
SW2(config-if)# vpc 15
SW3
SW3(config-if)# feature lacp
SW3(config-if)# channel-gr 15 mode activ

bridging loops.
Use with CAUTION
After configuring the vPC interfaces the vPC should come online when the access port is
correctly configured.
Finally we verify that our other vPC settings are correct, like the role assignments and the LACP
MAC addresses, etc.
SW2(config-if)# show vpc role
vPC Role status
----------------------------------------------------
23
vPC role
: primary
Dual Active Detection Status
: 0
vPC system-mac
: 12:34:56:78:ab:cd

vPC system-priority
: 100
vPC local system-mac
: 54:7f:ee:c2:7d:01
vPC local role-priority
: 255
SW2(config-if)#
Task 6: FEX
Next we will start configuring our Fabric Extender task. This task is about enabling the Nexus
2200 switches that we have connected to our Nexus 5000 switches.
We need to make sure that they are connected to both of the Nexus 5000 switches, which
means we are going to connect them using a VPC configuration.
Pay attention to the numbering of the FEX as this is crucial during your lab.
SW2
SW2(config-if)# feature fex
SW2(config-if)# sw mode fex
SW2(config-if)# fex asso 105

SW2(config-if)#
SW3
SW3(config-if)# feature fex
SW3(config-if)#
24

After configuring our VPC based FEX set-up we see that our FEXes are coming online and we
configured them using a vPC where they also got the correct number.
SW2(config-if)# show fex
FEX
Number
FEX
FEX
Description
State
FEX
Model
Serial
-----------------------------------------------------------------------105
FEX0105
Online
N2K-C2248TP-1GE
SSI14310218
106
FEX0106
Online
N2K-C2248TP-1GE
SSI142916SP
SW2(config-if)# show int fex-fabric

Fabric
Fex
Fabric
Port
Port State
Fex
Uplink
FEX
Model
Serial
--------------------------------------------------------------105
Eth1/13
Active
N2K-C2248TP-1GE
SSI14310218
106
Eth1/14
Active
N2K-C2248TP-1GE
SSI142916SP
SW2(config-if)# show vpc
Legend:

vPC domain id
: 5
Peer status
: peer is alive

: success
: success
vPC role
: primary
: 99
Peer Gateway
: Disabled
: -
: Enabled

---------------------------------------------------------------------
25
id
Port
Status Active vlans
--
----
------ --------------------------------------------------

1
Po6
up
1,1011
vPC status
--------------------------------------------------------------------------id
vlans
Port
Status Consistency Reason
Active
------ ----------- ------ ----------- -------------------------- ---------13
Po13
up
success
success
14
Po14
up
success
success
15
Po15
up
success
success
up
success
success
106496 Eth105/1/1
This whole process might take a while, so please be patient when configuring vPC with FEXes. It
might take up to 10 minutes for the whole switch to be discovered, not incorporating a possible
software upfes
Next we will make use of the FEXes by configuring another port-channel down to a connecting
router. This means we will be configuring an EvPC.
SW2
SW2(config-if)# int e105/1/1

bridging loops.
Use with CAUTION
SW2(config-if)# sw trunk allowed vlan 125-126
SW3
26

bridging loops.
Use with CAUTION
Finally we verify the EvPC configuration by checking if the vPC is reported up on both of the vPC
peers.
SW3(config-if)# sh vpc
Legend:

vPC domain id
: 5
Peer status
: peer is alive

: success
: success
vPC role
: secondary
: 100
Peer Gateway
: Disabled
: -
: Enabled

--------------------------------------------------------------------id
Port
Status Active vlans
--
----
------ --------------------------------------------------
Po6
up
1,1011
vPC status
27

--------------------------------------------------------------------------id
vlans
Port
Active
------ ----------- ------ ----------- -------------------------- ---------13
Po13
up
success
success
14
Po14
up
success
success
up
success
success
262243 Po100
Task 8: First Hop Redundancy

Next task is to configure some FHRP protocols. Which means we need to ensure Layer 3
redundancy for layer 2 hosts.
This means that we will be configuring protocols like HSRP and VRRP. The first task states that
we should configure a First Hop Redundancy mechanism for VLAN 1012. IP addressing for this
VLAN is not configured yet and will first need to be done.
We are using a standards based FHRP which means we are going to use VRRP.
SW2
SW2(config)# feature vrrp


SW2(config-if)# vrrp 1
SW2(config-if-vrrp)# address 172.22.12.254
SW2(config-if-vrrp)# no shut
SW2(config-if-vrrp)# exit
SW2(config-if)#
SW3
SW3(config)# feature vrrp
SW3(config-if-vrrp)# address 172.22.12.254
SW3(config-if-vrrp)# no shut
SW3(config-if-vrrp)# exit
28

SW3(config-if)#
we configured the VRRP protocol using the defaults. Before configuring the tweaks that we
need to do, we first configure the HSRP protocol for VLAN 125.
SW2
SW2(config-if)# feature hsrp
SW2(config-if)# hsrp 1
SW2(config-if-hsrp)# ip 172.22.125.1
SW2(config-if-hsrp)# no shut
SW2(config-if)#
SW3
SW3(config-if)# feature hsrp
SW3(config-if-hsrp)# ip 172.22.125.1
SW3(config-if-hsrp)# no shut
SW3(config-if)#
Then we verify that the switches have reachability to each other across these 2 VLANs to
ensure the FHRP protocols are working.
SW2(config-if)# sh vrrp
Interface
VR IpVersion Pri
Time Pre State
VR IP addr
--------------------------------------------------------------Vlan1012
IPV4
100
1 s
Master 172.22.12.254
SW2(config-if)# sh hsrp brie

P indicates configured to preempt.
|
Interface
Grp Prio P State
Active addr
Standby addr
Group addr
Vlan125
172.22.125.3
local
172.22.125.1
100
Standby
(conf)
SW2(config-if)#
SW3(config-if)# show vrrp
29

Interface
VR IpVersion Pri
Time Pre State
VR IP addr
--------------------------------------------------------------Vlan1012
IPV4
100
1 s
Backup 172.22.12.254
SW3(config-if)# show hsrp brie

|
Interface
Grp Prio P State
Active addr
Standby addr
Group addr
Vlan125
local
172.22.125.2
172.22.125.1
100
Active
(conf)
SW3(config-if)#
We see that a master and a standby router is elected for both protocols, meaning our
configurations work!
Next we need to make sure that SW2 is the primary gateway for VLAN 1012 and SW3 is the
gateway for VLAN 125. Funny enough this is already the case by default, but of course we need
to make sure of this by configuring priority values.
SW2
SW2(config-if-vrrp)# priority 120
SW2(config-if-vrrp)#
SW3
SW3(config-if-hsrp)# prio 120
SW3(config-if-hsrp)#
After applying the configuration we see that the priority values are correctly applied to both
the FHRP protocols.
SW2(config-if-vrrp)# show vrrp
Interface
VR IpVersion Pri
Time Pre State
VR IP addr
--------------------------------------------------------------Vlan1012
IPV4
110
1 s
Master 172.22.12.254
SW2(config-if-vrrp)# show hsrp brie

|
Interface
addr
30
Grp Prio P State
Active addr
Standby addr
Group

Vlan125
1
172.22.125.1
100
Standby
172.22.125.3
local
(conf)
SW2(config-if-vrrp)#
SW3(config-if-hsrp)# show vrrp
Interface
VR IpVersion Pri
Time Pre State
VR IP addr
--------------------------------------------------------------Vlan1012
IPV4
100
1 s
Backup 172.22.12.254
SW3(config-if-hsrp)# show hsrp brie

|
Interface
addr
Grp Prio P State
Vlan125
1
172.22.125.1
125
Active
Active addr
Standby addr
local
172.22.125.2
Group
(conf)
Next we need to tweak the reasons why the HSRP configuration should fail-over. Well before
we configure our tracking groups to monitor the OSPF uplinks, we need to make sure that the
other switch will take over the primary role when a switch is still online. This means using the
HSRP pre-empt feature.
SW2
SW2(config-if-hsrp)# preempt
SW3
SW3(config-if-hsrp)# int vlan 125
SW3(config-if-hsrp)# preempt
31
Next we start configuring our tracking objects. Iniitially we need to make sure that SW2 will not
forward traffic related to vPC interfaces. This is done by using a special priority value, called the
forwarding threshold.
SW3
SW3(config-if-hsrp)# int vlan 125
SW3(config-if-hsrp)# prio 120 forwarding-threshold lower 106 ?
upper
Set upper threshold value
SW3(config-if-hsrp)# prio 120 forwarding-threshold lower 106 upper ?

<0-255>
Upper threshold value
SW3(config-if-hsrp)# prio 120 forwarding-threshold lower 106 upper 110

SW3(config-if-hsrp)# exit
SW3(config-if)# track 1 interface ethernet1/1 ?
*** No matching command found in current mode, matching in (config) mode
***
Sub interface separator
ip
IPv4 parameters
line-protocol
Track interface line-protocol
SW3(config-if)# track 1 interface ethernet1/1 line-protocol ?

*** No matching command found in current mode, matching in (config) mode
***
<CR>
SW3(config-if)# track 1 interface ethernet1/1 line-protocol
SW3(config-track)# exit
SW3(config)# track 2 interface ethernet1/2 line-protocol
SW3(config-track)# exit
SW3(config-if-hsrp)# track 1 ?
<CR>
decrement
Decrements when tracked object goes down
SW3(config-if-hsrp)# track 1 decrement 15

SW3(config-if-hsrp)# track 2 decrement 15
32
What happens with this configuration is that when one of the OSPF uplinks fails, the priority
value will be lowered with 15. This means that our priority value will be lowered from 120 to
105. This is lower than the forwarding-threshold describes and therefore the switch will no
longer respond to Layer 3 requests on the HSRP virtual MAC.
When the second uplink fails, the priority falls to 90, which is lower than the next best router in
the network. This router will take over the primary role.
The final question of this task is to take down a HSRP adjacency when no hello packets are
received for 750ms. This means we need to configure subsecond hello intervals. On the Nexus
7000 we would have the option for BFD, but this in unsupported on the Nexus 5000.
SW3
SW3(config-if-hsrp)# timers ?
<1-254>
Hello interval in seconds
msec
Specify hello interval in milliseconds
SW3(config-if-hsrp)# timers msec ?

<250-999>
Hello interval in milliseconds
SW3(config-if-hsrp)# timers msec 250 ?

Hold time in seconds
msec
Specify hold interval in milliseconds
<3-255>
SW3(config-if-hsrp)# timers msec 250 msec ?

<750-3000>
Hold interval in milliseconds
SW3(config-if-hsrp)# timers msec 250 msec 750

SW2
SW2(config-if-hsrp)# timers msec 250 msec 750
33
And we finished our FHRP task.

Task 9: FabricPath
The next task consists of 2 parts. The first part is configured now, where the second part is
configured in the DC2 tasks. Pay attention that this could potentially cost a lot of points when
something doesnt work.
We will start by configuring the switches of DC1 for FabricPath.
SW1-1
SW1-1(config)# conf t
SW1-1(config)# feature-set fabricpath
SW1-1(config)# fabric switch-id 11
SW1-1(config-vlan)# mode fabricpath
SW1-1(config-vlan)# int e4/11-12
SW1-1(config-if-range)# sw mode fabricpath
SW1-1(config-if-range)# fabric isis hello-interval ?
Hello interval value
<1-65535>
*Default value is 10
SW1-1(config-if-range)# fabric isis hello-interval 3
SW1-1(config-if-range)# fabric isis hello-multiplier ?
<3-1000>
Hello multiplier value

*Default value is 3
SW1-1(config-if-range)# fabric isis hello-multiplier 4

SW1-1(config-if-range)# exit
SW1-1(config)# key chain FP_KEY
SW1-1(config-keychain)# key 1
SW1-1(config-keychain-key)# key-string FPauth
SW1-1(config-keychain-key)# exit
SW1-1(config-keychain)# exit
SW1-1(config)# fabricpath domain default
SW1-1(config-fabricpath-isis)# authentication-type cleartext
SW1-1(config-fabricpath-isis)# authentication key-chain FP_KEY
SW1-1(config-fabricpath-isis)# authentication-check
34

SW1-1(config-fabricpath-isis)# exit
SW1-1(config)#
We configured the proper Switch ID for FabricPath on SW1-1 and enabled the interfaces and
VLANs. Next we ensured that the correct hello interval and multipliers are configured to
support the 12 second failover.
Finally we need to make sure that all current and future links will support authentication. This
means that we need to use the domain authentication instead of the link authentication.
Next is configuring SW1-2, the other switches will be configured in the second section of this
task.
SW1-2

SW1-2(config)#
35
Task 10: OTV

Next is the configuration of the OTV feature. Pay close attention to the order of operation for
configuring the OTV feature, because this matters a lot. We need to make sure we are
configuring the right parameters.
There are a number of thing important in the OTV configuration.
Plain Layer 2 network for VLANs to be transported

Layer 3 connectivity to other OTV device
Empty Site-VLAN for AED election
Multicast IP addressing for Control group
SSM multicast group for Data group
All information is stated in the questioning, so we need to extract that and start configuring our
devices.
We start by configuring the Layer 2 access interfaces for the VLANs that need to be transported
between the Data Centers.
SW3
SW3(config)# vlan 201,202,203
SW3(config-vlan)# exit

SW3(config-if)# span port type normal
SW3(config-if)# exit
SW3(config)# feature interface-vlan
SW3(config-if)#
SW1-3
SW1-3(config)# vlan 201,202,203
SW1-3(config-vlan)# int e3/19
SW1-3(config-if)# sw
SW1-3(config-if)# sw mode trunk
36

SW1-3(config-if)# sw trunk allowed vlan 201-203
SW1-3(config-if)# span port type normal
SW1-3(config-if)# feature interface-vlan
SW1-3(config)# int vlan 201
SW1-3(config-if)# int vlan 202
SW1-3(config-if)#
Next we configure our layer 2 and layer 3 interfaces on the OTV devices.
SW1-2
SW1-2(config)# vlan 201,202,203
SW1-2(config-vlan)# exit

SW1-2(config-if)#
SW1-2(config-if)#
SW1-4
SW1-4(config)# vlan 201-203
37

SW1-4(config-if)#
Make sure the 2 OTV devices can reach each other across the Layer 3 cloud infrastructure.
SW1-4(config-if)# ping 198.1.24.1
PING 198.1.24.1 (198.1.24.1): 56 data bytes
Request 0 timed out
64 bytes from 198.1.24.1: icmp_seq=1 ttl=254 time=1.141 ms
--- 198.1.24.1 ping statistics --5 packets transmitted, 4 packets received, 20.00% packet loss
round-trip min/avg/max = 0.674/0.813/1.141 ms
SW1-4(config-if)#
Verify that the Client devices cannot ping each other!
SW3(config-if)# ping 198.0.201.13

PING 198.0.201.13 (198.0.201.13): 56 data bytes
Request 0 timed out
Request 1 timed out
Request 2 timed out
Request 3 timed out
Request 4 timed out
SW3(config-if)#
Now we can start building our OTV solution, using the questions requirements. This means we
are going to use Multicast where we are free to use multicast IP addressing. Pay attention to
the Site Identifiers. These are given to you as decimal numbers, but the configuration in NX-OS
is hexadecimal.
SW1-4
SW1-4(config-if)# feature otv
SW1-4(config)# int overlay0
38

SW1-4(config-if-overlay)# shut
SW1-4(config-if-overlay)# otv join-interface ethernet3/22
OTV needs join interfaces to be configured for IGMP version 3
SW1-4(config-if-overlay)# otv control-group 239.8.8.8
SW1-4(config-if-overlay)# otv data-group 232.8.8.0/24
SW1-4(config-if-overlay)# otv extend-vlan 201,202
SW1-4(config-if-overlay)# exit
SW1-4(config)# otv site-identifier 0xD
% Site Identifier mismatch will prevent
overlays from forwarding traffic.
SW1-4(config)# otv site-vlan 203

SW1-4(config-site-vlan)# exit
SW1-4(config-if-overlay)# no shut
SW1-4(config-if-overlay)# exit
SW1-4(config-if)# ip igmp v 3
SW1-4(config-if)#
SW1-2
SW1-2(config)# feature otv
SW1-2(config)# otv site-identifier 0xC

% Site Identifier mismatch will prevent
overlays from forwarding traffic.
SW1-2(config)# otv site-vlan 203

SW1-2(config-site-vlan)# exit
SW1-2(config-if-overlay)# otv join-interface ethernet3/14
OTV needs join interfaces to be configured for IGMP version 3
SW1-2(config-if-overlay)# otv control-group 239.8.8.8
SW1-2(config-if-overlay)# otv data-group 232.8.8.0/24
SW1-2(config-if-overlay)# otv extend-vlan 201,202
SW1-2(config-if-overlay)# no shut
SW1-2(config-if-overlay)# int e3/14
SW1-2(config-if)# ip igmp v 3
SW1-2(config-if)#
After our OTV configuration the Overlay should come online and after waiting for AED election,
we will see that the OTV connection is coming online.
SW1-4(config)# sh otv
39

OTV Overlay Information
Site Identifier 0000.0000.000d
Overlay interface Overlay0
VPN name
: Overlay0
VPN state
: UP
Extended vlans
: 201-202 (Total:2)
Control group
: 239.8.8.8
Data group range(s) : 232.8.8.0/24

Join interface(s)
: Eth3/22 (198.1.24.2)
Site vlan
: 203 (up)
AED-Capable
: Yes
Capability
: Multicast-Reachable
SW1-4(config)#
SW1-2# sh otv
OTV Overlay Information
Site Identifier 0000.0000.000c
Overlay interface Overlay0

VPN name
: Overlay0
VPN state
: UP
Extended vlans
: 201-202 (Total:2)
Control group
: 239.8.8.8
Data group range(s) : 232.8.8.0/24

Join interface(s)
: Eth3/14 (198.1.24.1)
Site vlan
: 203 (up)
AED-Capable
: Yes
Capability
: Multicast-Reachable
SW1-2#
Finally we try to ping the hosts in the OTV configuration and test the connectivity of the
Extended VLANs. It might take a few pings before the connectivity is established due to the
nature of the OTV protocol, where it translates ARPs to ensure a more controlled data center
interconnect.
40

PING 198.0.201.13 (198.0.201.13): 56 data bytes
Request 0 timed out
Request 1 timed out
PING 198.0.202.13 (198.0.202.13): 56 data bytes
Request 0 timed out
Request 1 timed out
Request 2 timed out
Request 3 timed out

PING 198.0.202.13 (198.0.202.13): 56 data bytes
SW3(config-if)#
41
Task 11: DC2 VLANs

Next we start with the second part of the networking tasks, which is configuring Data Center 2.
We start again by creating some VLANs on all the switches.
SW1-3
SW1-3# conf t
Enter configuration commands, one per line.
End with CNTL/Z.

SW1-3(config-vlan)# name UCS1
SW1-3(config)#
SW1-4
SW1-4# conf t
Enter configuration commands, one per line.
End with CNTL/Z.

SW1-4(config)#
42
After configuring the VLANs we have our Layer 2 network in place so we can start configuring
the rest of DC2.

Task 12: FabricPath #2

In this next task we will need to finish what we started in the initial FabricPath task. The
configuration is very similar, just on other devices and other links.
SW1-3
SW1-3(config)#
SW1-4
43

SW1-4(config)#
Now our adjacencies will be established and we will have a FabricPath network.
Task 13: QoS
Next is our QoS configuration. On the Nexus 7000 this can only be configured from the default
VDC, therefore we only require a single policy for all of the VDCs. We do need to configure
Jumbo frames on all our DC2 interfaces.

SW1-3
SW1-3(config)# system jumbomtu 9216
SW1-3(config)# interface ethernet3/19
SW1-3(config-if)# mtu 9216
SW1-3(config)# interface ethernet4/17-20
SW1-4
44

SW1-4(config)# system jumbomtu 9216
SW1-3(config)# interface ethernet4/21-24
SW1-1
class-map type qos match-any GOLD
match cos 1
match cos 2
class-map type qos match-any SILVER
match cos 4
match cos 5
class-map type queuing GOLD
match qos-group 1
class-map type queuing SILVER
match qos-group 4
class GOLD
policy-map type qos TASK14_QOS_IN

set qos-group 1
class SILVER
set qos-group 4
class class-default
set qos-group 0
policy-map type queuing TASK14_QUEUEING
class type queuing GOLD
bandwidth percent 20
class type queuing SILVER
class type queuing class-default
class-map type network-qos GOLD
match qos-group 1
class-map type network-qos SILVER
match qos-group 4
45
policy-map type network-qos TASK14_NQ

class type network-qos GOLD
mtu 9216
class type network-qos SILVER
mtu 9216
mtu 9216
system qos
service-policy type qos input TASK14_QOS_IN
service-policy type queuing input TASK14_QUEUEING
service-policy type queuing output TASK14_QUEUEING
service-policy type network-qos TASK14_NQ
Task 14: Port-Channel
SW1-3
The final task of the Networking part of this Mock Lab is to prepare the configurations for the
UCS systems which will be configured in the last section of the mock lab. The port-channels will
be single link and will be normal VLANs. The more interesting part of this configuration will be
in the UCS section where we are expected to match this configuration.
SW1-3(config-if)# feature lacp

SW1-3(config-if)# channel-gr 45 mode act
SW1-3(config-if)# int po45
SW1-3(config-if)# sw trunk allowed vlan 123,124,301,303
SW1-3(config-if)# span port type edge trunk
bridging loops.
Use with CAUTION
46

SW1-3(config-if)#
SW1-4
SW1-4(config-if)# feature lacp
SW1-4(config-if)# channel-gr 46 mode act
SW1-4(config-if)# int po46
SW1-4(config-if)# sw trunk allowed vlan 123,124,301,303
SW1-4(config-if)# span port type edge trunk
bridging loops.
Use with CAUTION
SW1-4(config-if)#
Now we finished the networking part of our Mock Lab and we continue with the Storage part.

47
Section 2
Storage Networking
Task 1: FCoE
The first task of the Storage Networking part is regarding FCoE configuration, both Multi-Hop
and an Access based connection.
We need to make sure that the C-series server connected to Ethernet1/15 through a vPC
configuration is reachable from FCoE.
First we enable the storage features on SW2 and SW3 to enable FCoE technologies.
SW2
SW2(config-if)# feature fcoe
FC license checked out successfully
fc_plugin extracted successfully
FC plugin loaded successfully
FCoE manager enabled successfully
FC enabled on all modules successfully
Enabled FCoE QoS policies successfully
SW2(config)#
SW3
SW3(config-if)# feature fcoe
FC license checked out successfully
fc_plugin extracted successfully
FC plugin loaded successfully
FCoE manager enabled successfully
FC enabled on all modules successfully
Enabled FCoE QoS policies successfully
SW3(config)#
Next we configure the VLANs and VSANs that we need for this task. Note that we are using
different VLAN and VSANs on both switches because we want full separation on the FC level
between 2 fabrics.
SW2
SW2(config-vlan)# fcoe vsan 2000
SW2(config)# vsan data
SW2(config-vsan-db)# vsan 2000
SW2(config-vsan-db)# exit
48

SW2(config)# sh vlan fcoe
Original VLAN ID
Translated VSAN ID
Association State
----------------
------------------
-----------------
2000
2000
Operational
SW2(config)#
SW3
SW3(config)# vsan data
SW3(config)# show vlan fcoe
Original VLAN ID
Translated VSAN ID
Association State
----------------
------------------
-----------------
2001
Operational
2001
SW3(config)#
The trick in the NX-OS implementation of FCoE is that it automatically filters out the FCoE
VLANs on the vPC peer-link and put the VLAN as err-disabled on the trunk. Of course a better
solution would be to filter out the VLAN in the allowed list on the peer-link, but this is not
necessary as this is automatic behavior. Spanning-Tree Bridge Assurance is the feature which
takes care of this.
SW3(config)# show vpc
Legend:
vPC domain id
: 5
Peer status
: peer is alive
49
: success
: success
vPC role
: secondary
: 1

Peer Gateway
: Disabled
: -
: Enabled

--------------------------------------------------------------------id
Port
Status Active vlans
--
----
------ --------------------------------------------------
Po1
up
vPC status
---------------------------------------------------------------------------id
Port
Active vlans
------ ----------- ------ ----------- -------------------------- ----------15
Po15
up
success
success
1,2001
SW3(config)# sh int trunk

-------------------------------------------------------------------------------Port
Native
Status
Vlan
Port
Channel
-------------------------------------------------------------------------------Eth1/5
trnk-bndl
Po1
Eth1/15
trnk-bndl
Po15
Po1
trunking
--
Po15
trunking
--
-------------------------------------------------------------------------------Port
Vlans Allowed on Trunk
-------------------------------------------------------------------------------Eth1/5
1-3967,4048-4093
Eth1/15
1-3967,4048-4093
Po1
1-3967,4048-4093
Po15
1-3967,4048-4093
-------------------------------------------------------------------------------Port
Vlans Err-disabled on Trunk
--------------------------------------------------------------------------------
50
Eth1/5
2001
Eth1/15
none

Po1
2001
Po15
none
-------------------------------------------------------------------------------Port
STP Forwarding
-------------------------------------------------------------------------------Eth1/5
none
Eth1/15
none
Po1
Po15
1,2001
-------------------------------------------------------------------------------Port
Vlans in spanning tree forwarding state and not pruned
-------------------------------------------------------------------------------Eth1/5
--
Eth1/15
--
Po1
--
Po15
--
-------------------------------------------------------------------------------Port
Vlans Forwarding on FabricPath
-------------------------------------------------------------------------------SW3(config)# sh span vlan 2001

VLAN2001
Root ID
Priority
34769
Address
547f.eec2.7f01
This bridge is the root
Bridge ID
Interface
Hello Time
sec
Priority
34769
Max Age 20 sec
Address
547f.eec2.7f01
Hello Time
sec
Role Sts Cost
Max Age 20 sec
Prio.Nbr Type
---------------- ---- --- --------- -------- -------------------------------Po15
Desg FWD 1
128.4110 (vPC) Edge P2p
SW3(config)#
51
Next we can start creating the VFC configurations.

SW2
SW2(config)# int vfc1
SW2(config-if)# bind interface po15
SW2(config-if)# sw mode f
SW2(config-if)# sw trunk allowed vsan 2000
SW2(config-if)# vsan data
SW2(config-vsan-db)# vsan 2000 interface vfc1
SW2(config-vsan-db)# int vfc1
SW2(config-if)#
SW3
SW3(config-if)# bind interface po15
SW3(config-if)# sw mode f
SW3(config-vsan-db)# vsan 2001 interface vfc2
SW3(config-vsan-db)# int vfc2

SW3(config-if)#
We can bind the interfaces to the port-channel in this case, because we only have one local
connection going down to the FCoE device. Otherwise we would need to bind the VFC to the
physical port on either the Nexus 5000 or the FEX.
SW3(config-if)# show int vfc2
vfc2 is trunking
Bound interface is port-channel15
Hardware is Ethernet
Port WWN is 20:01:54:7f:ee:c2:7e:ff
Admin port mode is F, trunk mode is on
snmp link state traps are enabled
Port mode is TF
Port vsan is 2001
Trunk vsans (admin allowed and active) (2001)
52
Trunk vsans (up)
()
Trunk vsans (isolated)
()

Trunk vsans (initializing)
(2001)
1 minute input rate 0 bits/sec, 0 bytes/sec, 0 frames/sec

1 minute output rate 0 bits/sec, 0 bytes/sec, 0 frames/sec
0 frames input, 0 bytes
0 discards, 0 errors
0 frames output, 0 bytes
last clearing of "show interface" counters never
Interface last changed at Wed Oct 16 19:34:13 2013
SW3(config-if)#

Now we have a successfully configured FCoE access connection. It could happen that the VSAN
does not come online. This is due to the implementation of FCoE on the C-series server ,which
might cause the connection to only come online when the server is rebooted.
Next we need to configure the Multi-Hop FCoE configuration between SW2 and SW3. We need
to trunk 2 new VSANs between the switches.
SW2
So we configure the new VSANs and their respective VLANs.

SW2(config-vlan)#
SW3
53

SW3(config-vlan)#
We verify the new VLAN to VSAN mapping configuration.

SW2(config)# show vlan fcoe
Original VLAN ID
Translated VSAN ID
Association State
----------------
------------------
-----------------
2299
299
Operational
2000
2000
Operational
2188
188
Operational
SW2(config)#
SW3(config)# sh vlan fcoe
Original VLAN ID
Translated VSAN ID
Association State
----------------
------------------
-----------------
299
Operational
2001
2001
Operational
188
Operational
2299
2188
SW3(config)#
Now in this case we do want our VLANs to cross the vPC peer-link between the switches
because we are setting up a FCoE connection between the 2 switches.
Now this becomes impossible, because we cannot allow the FCoE VLANs to go over a vPC peer-
link. Therefore we force the FCoE VLANs to go over the connection which is not a vPC peer-link
on this switch.
SW2
SW2(config-if)# sw trunk allowed vlan add 2188,2299
SW3
SW3(config-if)# sw trunk allowed vlan add 2188,2299
54
Now the VLANs are available on both of the switches and we have a forwarding path between
them.
SW2
SW2(config-if)# int vfc3
SW2(config-if)# shut
SW2(config-if)# bind interface ethernet1/5
SW2(config-if)# sw mode e
SW2(config-if)# sw trunk allowed vsan add 299
SW3
SW3(config-if)# int vfc3
SW3(config-if)# shut
SW3(config-if)# bind interface ethernet1/5
After enabling the VFC interfaces and we wait a while. We see that both VSANs are now up and
trunking.
vfc3 is trunking
Bound interface is Ethernet1/5
Port WWN is 20:02:54:7f:ee:c2:7c:ff
Admin port mode is E, trunk mode is on
Port mode is TE
Port vsan is 1
Trunk vsans (admin allowed and active) (188,299)
Trunk vsans (up)
(188,299)
()
()

55

And with this step we finished our FCoE configuration and we continue with the native FC
configuration.

Task 2: JBOD
Next we configure our VSANs on the MDS switches and prepare for connections.
MDS1
MDS1(config)# vsan data
MDS1(config-vsan-db)# vsan 188
MDS1(config-vsan-db)# vsan 188 name ML2_VSAN1
MDS1(config-vsan-db)# exit
MDS2
MDS2(config)# vsan data
MDS2(config-vsan-db)# exit
Verify that the VSANs are created in the database.

MDS1(config)# show vsan membership
vsan 1 interfaces:
fc1/1
fc1/2
fc1/3
fc1/4
fc1/5
fc1/6
fc1/7
fc1/8
fc1/9
fc1/10
fc1/11
fc1/12
fc1/13
fc1/14
fc1/15
fc1/16
fc1/17
fc1/18
vsan 188 interfaces:
56
vsan 299 interfaces:

vsan 4079(evfp_isolated_vsan) interfaces:
vsan 4094(isolated_vsan) interfaces:

Next we will configure our links on the MDS switches so the JBODs are seen in the FLOGI
database.
MDS1
MDS1(config)# int fc1/6
MDS1(config-if)# sw mode fl
MDS1(config-if)# no shut
MDS1(config-if)# vsan data
MDS1(config-vsan-db)# vsan 188 interface fc1/6
MDS2
MDS2(config)# int fc1/5
MDS2(config-if)# sw mode fl
MDS2(config-vsan-db)# vsan 299 interface fc1/5
Verify that the FLOGIs are seen on the MDS switches.

MDS1(config-vsan-db)# show flogi data
-------------------------------------------------------------------------------INTERFACE
VSAN
FCID
PORT NAME
NODE NAME
--------------------------------------------------------------------------------
57
fc1/6
188
0x260073
22:00:00:11:c6:a6:24:4c 20:00:00:11:c6:a6:24:4c
fc1/6
188
0x260074
22:00:00:14:c3:a0:68:59 20:00:00:14:c3:a0:68:59
fc1/6
188
0x260079
22:00:00:14:c3:a0:60:38 20:00:00:14:c3:a0:60:38
fc1/6
188
0x26007a
22:00:00:11:c6:a6:3c:6f 20:00:00:11:c6:a6:3c:6f
fc1/6
188
0x260081
22:00:00:14:c3:a0:60:05 20:00:00:14:c3:a0:60:05
fc1/6
188
0x260082
22:00:00:11:c6:a6:2c:65 20:00:00:11:c6:a6:2c:65
fc1/6
188
0x26008f
22:00:00:11:c6:a6:3a:36 20:00:00:11:c6:a6:3a:36

fc1/6
188
0x260090
22:00:00:11:c6:a6:3a:9c 20:00:00:11:c6:a6:3a:9c
Total number of flogi = 8.

MDS1(config-vsan-db)#
MDS2(config-if)# show flogi data
-------------------------------------------------------------------------------INTERFACE
VSAN
FCID
PORT NAME
NODE NAME
-------------------------------------------------------------------------------fc1/5
299
0x6e0059
21:00:00:11:c6:a6:2a:60 20:00:00:11:c6:a6:2a:60
fc1/5
299
0x6e0063
21:00:00:14:c3:a0:60:d5 20:00:00:14:c3:a0:60:d5
fc1/5
299
0x6e0065
21:00:00:11:c6:a6:24:ca 20:00:00:11:c6:a6:24:ca
fc1/5
299
0x6e0069
21:00:00:11:c6:a6:ee:8a 20:00:00:11:c6:a6:ee:8a
fc1/5
299
0x6e006a
21:00:00:14:c3:a0:60:1b 20:00:00:14:c3:a0:60:1b
fc1/5
299
0x6e006d
21:00:00:11:c6:87:00:92 20:00:00:11:c6:87:00:92
fc1/5
299
0x6e006e
21:00:00:11:c6:a6:25:de 20:00:00:11:c6:a6:25:de
Total number of flogi = 7.
MDS2(config-if)#
Task 4: ISL
Next we will configure the interlink between the Nexus 5500 switch and the MDS switches. We
will need to configure this to be in a port-channel with negotiating a protocol.
First we will need to convert the interfaces on the Nexus 5548UP model as we will be using
ports 31 and 32 on the chassis which requires a reboot to take effect.
SW3
SW3(config)# slot 1
SW3(config-slot)# port 31-32 type fc
SW3(config-slot)#
end
SW3# copy run start

[########################################] 100%
Copy complete, now saving to disk (please wait)...
SW3# reload
WARNING: This command will reboot the system
Do you want to continue? (y/n) [n] y
58
Next we can configure our ports on the MDS switches while waiting for the reboot of the Nexus
5548UP.
MDS2
MDS2(config-if)# int fc1/13-14
MDS2(config-if)# channel-gr 100
command failed: port not compatible [port mode]
** You can use force option to override the port's parameters
** (e.g. "channel-group X force")
MDS2(config-if)# sw mode e
fc1/14: (error) Auto/E mode is not allowed in shared rate-mode
fc1/13: (error) Auto/E mode is not allowed in shared rate-mode
MDS2(config-if)# sw rate-mode dedicated
fc1/13 fc1/14 added to port-channel 100 and disabled
please do the same operation on the switch at the other end of the portchannel,
then do "no shutdown" at both ends to bring it up
MDS2(config-if)# int po100

MDS2(config-if)# channel mode ?
active
Configure ACTIVE port-channel
MDS2(config-if)# channel mode active

MDS2(config-if)# sw trunk allowed vsan 188
MDS2(config-if)# sw trunk allowed vsan add 299
MDS2(config-if)#
SW3
SW3(config)# int fc1/31-32
SW3(config-if)# channel-gr 100
59

SW3(config-if)# int san-port-channel 100
SW3(config-if)# channel mode acive
^
% Invalid command at '^' marker.
SW3(config-if)# channel mode active
SW3(config-if)# int fc1/31-32
After configuring the ports will come online and the port-channel is established.
MDS2(config-if)# sh int po100
port-channel 100 is trunking
Hardware is Fibre Channel
Port WWN is 24:64:00:05:9b:7f:aa:40
Port vsan is 1
Port mode is TE
Speed is 8 Gbps
Trunk vsans (up)
(188,299)
()
()
5 minutes input rate 1520 bits/sec, 190 bytes/sec, 1 frames/sec

5 minutes output rate 1480 bits/sec, 185 bytes/sec, 1 frames/sec
0 CRC,
0 unknown class
0 too long, 0 too short

10 input OLS, 11 LRR, 12 NOS, 0 loop inits
8 output OLS, 4 LRR, 9 NOS, 0 loop inits
Member[1] : fc1/13
Member[2] : fc1/14
60
MDS2(config-if)#
SW3(config-if)# show int san100 trunk vsan
san-port-channel 100 is trunking
Vsan 188 is down (Initializing)
Vsan 299 is up (None)
SW3(config-if)# show int san 100
Port WWN is 24:64:54:7f:ee:c2:7e:c0
Port mode is TE
Port vsan is 1
Speed is 8 Gbps
Trunk vsans (up)
(188,299)
()
()

0 CRC,
0 unknown class

Member[1] : fc1/31
Member[2] : fc1/32
SW3(config-if)#
Now to finally verify if we indeed see name server entries on the Nexus switches.
SW3(config-if)# show fcns data
61
VSAN 299:
-------------------------------------------------------------------------FCID
TYPE
PWWN
(VENDOR)
FC4-TYPE:FEATURE
-------------------------------------------------------------------------0x6e0059
NL
21:00:00:11:c6:a6:2a:60
scsi-fcp:target
0x6e0063
NL
21:00:00:14:c3:a0:60:d5
scsi-fcp:target
0x6e0065
NL
21:00:00:11:c6:a6:24:ca
scsi-fcp:target
0x6e0069
NL
21:00:00:11:c6:a6:ee:8a
scsi-fcp:target
0x6e006a
NL
21:00:00:14:c3:a0:60:1b
scsi-fcp:target
0x6e006d
NL
21:00:00:11:c6:87:00:92
scsi-fcp:target
0x6e006e
NL
21:00:00:11:c6:a6:25:de
scsi-fcp:target
Total number of entries = 7

SW3(config-if)#
Which we do.

Task 5: FC security
Next we need to secure the inter-links between the switches. Which is what we will do using
FC-SP as we need to use authentication hashes before links will come online.
Pay attention that FC-SP should also be turned on the FCIP connections which we are
configuring later.
MDS2
MDS2(config-if)# feature fcsp
MDS2(config)# fcsp dhchap password MDS2securehash
MDS2(config)# fcsp dhchap hash ?
MD5
MD5 Hash Algorithm
SHA1
SHA-1 Hash Algorithm
MDS2(config)# fcsp dhchap hash sha1

MDS2(config)# fcsp dhchap devicename 20:00:00:05:9b:7f:6e:00 ?
password
Configure DHCHAP password of remote device
MDS2(config)# fcsp dhchap devicename 20:00:00:05:9b:7f:6e:00 password

MDS1securehash
MDS2(config)# fcsp dhchap devicename 20:00:54:7f:ee:c2:7e:c0 password
SW3securehash
MDS2(config)#
62

MDS2(config)# int port-channel 100
MDS2(config-if)# fcsp on
MDS1
MDS1(config-vsan-db)# feature fcsp
MDS1(config)# fcsp dhchap password MDS1securehash
MDS1(config)# fcsp dhchap devicename 20:00:00:05:9b:7f:aa:40 password
MDS2securehash
MDS1(config)#
SW3
SW3(config)# feature fcsp
SW3(config)# fcsp dhchap password SW3securehash
SW3(config)# fcsp dhchap devicename 20:00:00:05:9b:7f:aa:40 password
MDS2securehash
SW3(config)# fcsp dhchap devicename 20:00:54:7f:ee:c2:7c:c0 password
SW2securehash
SW3(config)#
SW3(config)# int san-port-channel 100
SW3(config-if)# fcsp on
SW2
SW2(config)# feature fcsp
SW2(config)# fcsp dhchap password SW2securehash
SW2(config)# fcsp dhchap devicename 20:00:54:7f:ee:c2:7e:c0 password
SW3securehash
SW2(config)#
63
We enabled the current active interfaces and the FC-SP feature is already pre-configured to
support the configuration of MDS1 and MDS2 across the FCIP tunnels.

Task 6: FCIP
Now its time to configure the FCIP configuration between MDS1 and MDS2. Pay attention to
the questioning as we will need to perform changes to SW2 and SW3 as well, because the
Ethernet connections of the MDS switches are connected to the Nexus 5000 switches, so it
requires some additional configuration.
We first prepare our Nexus 5000 switches to support the FCIP configuration.
SW2
SW2(config)# int e1/11-12
SW2(config-if-range)# speed 1000
SW2(config-if-range)# int e1/11
SW2(config-if)# sw trunk allowed vlan 1111

bridging loops.
Use with CAUTION
SW2(config-if)# sw mode acc
SW2(config-if)# sw acc vlan 1012
SW2(config-if)# span port type edge
bridging loops.
Use with CAUTION
Edge Port Type (Portfast) has been configured on Ethernet1/12 but will
only
have effect when the interface is in a non-trunking mode.
64

SW2(config-if)#
SW3
SW3(config)# int e1/11-12
SW3(config-if-range)# speed 1000
SW3(config-if-range)# int e1/11
SW3(config-if)# sw trunk allowed vlan 1111
bridging loops.
Use with CAUTION
SW3(config-if)# sw mode acc
sw acc vlan 1012
SW3(config-if)#
SW3(config-if)# span port type edge

bridging loops.
Use with CAUTION
Edge Port Type (Portfast) has been configured on Ethernet1/12 but will
only
have effect when the interface is in a non-trunking mode.
SW3(config-if)#
65
Now we configure the IP addressing on the MDS switches. Sometimes its required to already
enable the FCIP feature when doing this, because the code on the MDS switches might show
strange error messages otherwise.
MDS1
MDS1(config)# int gi1/1
MDS1(config-if)# int gi1/1.1111
MDS1(config-if)# ip add 198.18.111.1 255.255.255.128
Failed to configure IP address: the interface does not exist
MDS1(config-if)# feature fcip
MDS1(config)# int gi1/1.1111
MDS1(config-if)# int gi1/2
MDS1(config-if)#
MDS1
MDS2(config)#
MDS2(config)# feature fci

MDS2(config)# int gi1/1
MDS2(config-if)# int gi1/1.1111
MDS2(config-if)# int gi1/2
MDS2(config-if)#
Now we try to ping the MDS switches from each other, meaning we are able to successfully set-
up an FCIP connection to them.
MDS1(config-if)# ping 198.18.111.2
PING 198.18.111.2 (198.18.111.2) 56(84) bytes of data.
66
--- 198.18.111.2 ping statistics --4 packets transmitted, 4 received, 0% packet loss, time 3000ms
rtt min/avg/max/mdev = 0.368/0.446/0.536/0.063 ms
MDS1(config-if)# ping 172.22.12.102
--- 172.22.12.102 ping statistics --4 packets transmitted, 4 received, 0% packet loss, time 2997ms
rtt min/avg/max/mdev = 0.334/0.376/0.440/0.042 ms
MDS1(config-if)#
Next we can start configuring the FCIP tunnels.

We start by seeing that we need to configure a SAN port-channel from 2 different FCIP tunnels,
this is the only we ensure that a single failure of a GigE connection or an FCIP tunnel will not
cause an FSPF re-calculation.
Again do not forget to add the FC-SP security configuration as this was a requirement in the
previous task.
MDS1
MDS1(config)# fcip prof 1
MDS1(config-profile)# ip address 198.18.111.1
MDS1(config-profile)# fcip prof 2
MDS1(config-profile)# ip add 172.22.12.101
MDS1(config-profile)# int fcip1
MDS1(config-if)# use-profile 1
MDS1(config-if)# peer ipaddr 198.18.111.2
MDS1(config-if)# write-accelerator
fcip1 added to port-channel 101 and disabled
MDS1(config-if)# int fcip2
67

MDS1(config-if)# peer ip 172.22.12.102
MDS1(config-if)# int fcip1-2
MDS2
MDS2(config)# fcip prof 1
MDS2(config-profile)# ip address 198.18.111.2
MDS2(config-profile)# fcip prof 2

MDS2(config-profile)# ip add 172.22.12.102
MDS2(config-profile)# int fcip1
MDS2(config-if)# int fcip2
MDS2(config-if)# use 2
68

MDS2(config-if)# int fcip1-2
MDS2(config-if)#
After this configuration our FCIP connections should come online in a port-channel and we
should successfully authenticate the FC-SP configuration.
MDS2(config-if)# show fcip summary
------------------------------------------------------------------------------Tun prof
Eth-if
peer-ip
Status T W T Enc Comp

E A A
Bandwidth
max/min
rtt
(us)
------------------------------------------------------------------------------1
GE1/1.1111 198.18.111.1
TRNK
Y Y N
1000M/500M
1000
GE1/2
TRNK
Y Y N
1000M/500M
1000
172.22.12.101
MDS2(config-if)# sh int fcip1-2

fcip1 is trunking
Hardware is GigabitEthernet
Port WWN is 20:14:00:05:9b:7f:aa:40
Peer port WWN is 20:14:00:05:9b:7f:6e:00
Port mode is TE
Port vsan is 1
Speed is 1 Gbps
Belongs to port-channel 101
FCSP Status: Successfully authenticated
69
Trunk vsans (up)
(188,299)
()

Using Profile id 1
()
(interface GigabitEthernet1/1.1111)
Peer Information
Peer Internet address is 198.18.111.1 and port is 3225
Write acceleration mode is configured on; operationally on
Tape acceleration mode is configured off
Tape Accelerator flow control buffer size is automatic
FICON XRC Accelerator is configured off
Ficon Tape acceleration configured off for all vsans
IP Compression is disabled
Maximum number of TCP connections is 2
QOS control code point is 0
QOS data code point is 0
TCP Connection Information
2 Active TCP connections
Control connection: Local 198.18.111.2:3225, Remote 198.18.111.1:65531
Data connection: Local 198.18.111.2:3225, Remote 198.18.111.1:65533
6 Attempts for active connections, 0 close of connections
TCP Parameters
Path MTU 1500 bytes
Current retransmission timeout is 200 ms
Round trip time: Smoothed 1 ms, Variance: 1 Jitter: 150 us

Advertized window: Current: 2045 KB, Maximum: 24580 KB, Scale: 5
Peer receive window: Current: 37 KB, Maximum: 37 KB, Scale: 5
Congestion window: Current: 36 KB, Slow start threshold: 112 KB
Current Send Buffer Size: 26 KB, Requested Send Buffer Size: 0 KB
CWM Burst Size: 50 KB
Measured RTT : 500000 us Min RTT: 49 us Max RTT: 0 us
30 Class F frames input, 4940 bytes
16 Class 2/3 frames input, 1680 bytes
0 Reass frames
0 Error frames timestamp error 0
31 Class F frames output, 4500 bytes
16 Class 2/3 frames output, 1584 bytes
0 Error frames
70

fcip2 is trunking
Port WWN is 20:18:00:05:9b:7f:aa:40
Peer port WWN is 20:18:00:05:9b:7f:6e:00
Port mode is TE
Port vsan is 1
Speed is 1 Gbps
Belongs to port-channel 101
FCSP Status: Successfully authenticated
Trunk vsans (up)
(188,299)
()
()
Using Profile id 2
(interface GigabitEthernet1/2)
Peer Information
Peer Internet address is 172.22.12.101 and port is 3225
Write acceleration mode is configured on; operationally on
Tape acceleration mode is configured off
Tape Accelerator flow control buffer size is automatic
FICON XRC Accelerator is configured off

Ficon Tape acceleration configured off for all vsans
IP Compression is disabled
Maximum number of TCP connections is 2
QOS control code point is 0
QOS data code point is 0
TCP Connection Information
2 Active TCP connections
Control connection: Local 172.22.12.102:65523, Remote 172.22.12.101:3225
Data connection: Local 172.22.12.102:65525, Remote 172.22.12.101:3225
6 Attempts for active connections, 0 close of connections
TCP Parameters
Path MTU 1500 bytes
Current retransmission timeout is 200 ms
Round trip time: Smoothed 1 ms, Variance: 1 Jitter: 152 us
Advertized window: Current: 29 KB, Maximum: 24580 KB, Scale: 5
Peer receive window: Current: 2042 KB, Maximum: 2042 KB, Scale: 5
Congestion window: Current: 17 KB, Slow start threshold: 518 KB
Current Send Buffer Size: 26 KB, Requested Send Buffer Size: 0 KB
71

CWM Burst Size: 50 KB
Measured RTT : 500000 us Min RTT: 47 us Max RTT: 0 us
2 Reass frames
0 Error frames
MDS2(config-if)# show int po101
Port WWN is 24:65:00:05:9b:7f:aa:40
Port vsan is 1
Port mode is TE
Speed is 2 Gbps
Trunk vsans (up)
(188,299)
()
()

2 Reass frames
0 Error frames
Member[1] : fcip1
Member[2] : fcip2
72
MDS2(config-if)#
Do not forget to enable the write-accelerator as well, as this is the improvement on the sending
of R_RDY frames as the question is asking for.
Here we finished the configuration of our FCIP task.
73
CCIE Data Center Lab Preparation Detailed Solution Guide
Task 7: Zoning
Next we will configure our zoning. We will already configure the zoning based on the initiators
from the UCS section where we will use the WWPN pools to create device-aliases and then
enable the initiator-target-zone model as the question is asking for.
First we need to enable enhanced device-alias mode, because we want to keep the device-alias
name in our zoning configuration.
MDS1
MDS1(config)# device-alias mode enhanced
MDS1(config)# device-alias commit
MDS1(config)#
MDS1(config)#
MDS1(config)# device-alias data
MDS1(config-device-alias-db)# show fcns data
VSAN 188:
-------------------------------------------------------------------------FCID
TYPE
PWWN
(VENDOR)
FC4-TYPE:FEATURE
-------------------------------------------------------------------------NL
22:00:00:11:c6:a6:24:4c
scsi-fcp:target
0x260074
NL
22:00:00:14:c3:a0:68:59
scsi-fcp:target
0x260079
NL
22:00:00:14:c3:a0:60:38
scsi-fcp:target
0x26007a
NL
22:00:00:11:c6:a6:3c:6f
scsi-fcp:target
0x260081
NL
22:00:00:14:c3:a0:60:05
scsi-fcp:target
0x260073
0x260082
NL
22:00:00:11:c6:a6:2c:65
scsi-fcp:target
0x26008f
NL
22:00:00:11:c6:a6:3a:36
scsi-fcp:target
0x260090
NL
22:00:00:11:c6:a6:3a:9c
scsi-fcp:target

VSAN 299:
-------------------------------------------------------------------------FCID
TYPE
PWWN
(VENDOR)
FC4-TYPE:FEATURE
-------------------------------------------------------------------------0x6e0059
NL
21:00:00:11:c6:a6:2a:60
scsi-fcp:target
0x6e0063
NL
21:00:00:14:c3:a0:60:d5
scsi-fcp:target
0x6e0065
NL
21:00:00:11:c6:a6:24:ca
scsi-fcp:target
0x6e0069
NL
21:00:00:11:c6:a6:ee:8a
scsi-fcp:target
0x6e006a
NL
21:00:00:14:c3:a0:60:1b
scsi-fcp:target
74
0x6e006d
NL
21:00:00:11:c6:87:00:92
scsi-fcp:target
0x6e006e
NL
21:00:00:11:c6:a6:25:de
scsi-fcp:target

MDS1(config-device-alias-db)# device-alias name V188_BOOT_DISK pwwn
22:00:00:11:c6:a6:24:4c
MDS1(config-device-alias-db)# device-alias name V188_DISK2 pwwn
22:00:00:14:c3:a0:68:59
MDS1(config-device-alias-db)# device-alias name V299_BOOT_DISK pwwn
21:00:00:11:c6:a6:27:4c
MDS1(config-device-alias-db)# device-alias name V299_DISK2 pwwn
21:00:00:11:c6:a6:2a:60
MDS1(config-device-alias-db)# device-alias name UCS_A_1 pwwn
20:00:00:25:B5:A0:00:00
20:00:00:25:B5:A0:00:01
20:00:00:25:B5:A0:00:02
20:00:00:25:B5:A0:00:03
MDS1(config-device-alias-db)# device-alias name UCS_B_1 pwwn
20:00:00:25:B5:B0:00:00

20:00:00:25:B5:B0:00:01
20:00:00:25:B5:B0:00:02
20:00:00:25:B5:B0:00:03
MDS1(config-device-alias-db)# device-alias commit
MDS1(config)#
We verify with the FC Name Server to ensure we have the right disks that we configure a
device-alias for. Then we use the UCS PWWN pools to allocate device-aliases for the UCS
blades.
MDS1
MDS1(config)# zoneset name V188_ZS1 v 188
MDS1(config-zoneset)# zone name ML2_V188_Z1
MDS1(config-zoneset-zone)# member device-alias V188_BOOT_DISK
MDS1(config-zoneset-zone)# member device-alias UCS_A_1
MDS1(config-zoneset-zone)# member device V188_BOOT_DISK
75
MDS1(config-zoneset-zone)# zone name ML2_V188_Z3

MDS1(config-zoneset-zone)# zoneset activ name V188_ZS1 v 188
Zoneset activation initiated. check zone status
MDS1(config)# zoneset name V299_ZS1 v 299
MDS1(config-zoneset-zone)# member device-alias UCS_B_1
MDS1(config-zoneset-zone)# zoneset activ name V299_ZS1 v 299

Zoneset activation initiated. check zone status
MDS1(config)#
Now we configured all zoning for our UCS blades and based on the boot disks that we have
ready.
MDS1(config)# show zoneset active
zoneset name V188_ZS1 vsan 188
zone name ML2_V188_Z1 vsan 188
* fcid 0x260073 [device-alias V188_BOOT_DISK]
device-alias UCS_A_1
zone name ML2_V188_z2 vsan 188
76
zoneset name V299_ZS1 vsan 299
device-alias V299_BOOT_DISK
device-alias UCS_B_1
MDS1(config)#
We verify that the zoning has been activated and is now working on our Fibre Channel Fabrics!


As for the final task of the storage section we will be configuring the uplinks towards the UCS
system. Again we are making port-channels for these connections.
Pay attention that this is not an E port connection, but this is an end host connecting, or an NPV
switch, therefore this should be an F port and we should enable the bundling of F ports.
MDS1
MDS1(config)# feature fport-channel-trunk
Admin trunk mode has been set to off for
1- Interfaces with admin switchport mode F,FL,FX,SD,ST in admin down state
2- Interfaces with operational switchport mode F,FL,SD,ST.
77
MDS1(config)#
MDS1(config)# int fc1/9-10
MDS1(config-if)# sw mode f
MDS1(config-if)# sw trunk mode on
MDS1(config-vsan-db)# vsan 188 interface po102
MDS2
MDS2(config)# feature fport-channel-trunk

Admin trunk mode has been set to off for
1- Interfaces with admin switchport mode F,FL,FX,SD,ST in admin down state
2- Interfaces with operational switchport mode F,FL,SD,ST.
MDS2(config)#
MDS2(config)# int fc1/9-10
MDS2(config-if)# sw trunk mode on
78
MDS2(config-vsan-db)# vsan 188 interface po103
And with the preparation of the UCS uplink port-channels we finished the storage section of
this mock lab. We will continue with the final section, the UCS section.
79
Section 3
Unified Computing
Task 1: Chassis initialization

The first task in setting up the UCS is discovering the blade chassis which are connected to it.
This means that we first need to assign server ports on the Fabric Interconnect. Before we
configure the server ports its good to configure the Chassis Discovery Policy, because after we
configure the server ports the chassis immediately starts initializing. Therefore we can better
use the correct settings when initializing the chassis.
We need to set any number of links, which means we set the amount of links to 1 and we allow
port-channeling when we have 2200 series IO modules.
Next we can configure the Server links, which are ports 1, 3, 5 and 7 on our Fabric
Interconnects.
80

Do not forget to configure this on both Fabric Interconnects.
After the chassis is initialized we need to make sure that all links are used and not just the
single link it is discovered with now.

81
By re-acknowledging the chassis we will utilize all the links that we have and will start using all
links in a port-channel when we have a 2200 extender. The re-acknowledgement is really
necessary as otherwise only a single link is used.

Task 2: VLANs and Uplinks
Next we will focus on the networking uplinks. First we create the VLANs that we also allow on
the uplinks going down to the UCS Fabric Interconnects.
82
When the 4 VLANs are created we can configure the uplink port-channels. Keep in mind that we
need to use the same port-channel numbering as in the networking section.
83
Port-channels always need to be enabled as they are in shutdown state by default.

Now we configured the first Fabric Interconnect. Next is Fabric Interconnect B.
84
85
After a successful configuration of the network uplinks the only thing that remains is making
sure that the QoS configuration is complimenting the network QoS configuration.
Task 3: VSANs and Uplinks

Next part is configuring the Storage uplinks. The configuration will be a little different from the
other Mock Labs as we are now configuring both VSANs to cross at both Fabrics. Normally you
would configure only a single VSAN to go to a single Fabric Interconnect. In this case we want to
test where the port VSAN is configured and that it is possible to trunk multiple VSANs to a
Fabric Interconnect.
We first create the VSANs globally so they are automatically created on both Fabric
Interconnects.

86

Now the VSANs are created and now we can create the port-channels that we need to create
according to our previously configured Storage port-channels on the MDS switches.

87
Pay attention that you select VSAN 188 in the Dropdown box for VSAN to make this the Port or
Native VSAN for this port-channel!
88

Storage uplinks are now created and configured as they should.
Task 4: Pools
We continue the lab with pools of addresses that we need to allocate. Pay close attention to
the naming, the prefixes and the size of the pools as everything can cost you a lot of points.
Also note that your Storage zoning is dependent on this configuration.
89

90

91
Task 5: Server pools

Next we are creating Server Pools, which are configured in a way so we automatically add our
blades to a server pool as soon as they are seen in the UCS Manager.

We first determine the amount of memory in our blades.
92
In this case we see only a very limited amount of memory in the blades. We will configure this
server pool and policies related to it.

We create an empty pool.
93
Next is the qualification policy where we set the minimum amount of memory.
Finally we combine the pool and qualification in a policy and from that moment the servers
which are already discovered are added automatically to the server pool.

94
Next we repeat the same process for blades with a Cisco VIC card or Cisco M81KR mezzanine
card.
95
96
Task 6: vNIC template

The next task is to configure templates for our network connectivity on the blades. This means
that we are configuring the networking part of the blade connectivity towards the rest of the
network.
The task starts that we should use templates that will be updated when we apply changed to
the template after its been initially applied.
We first create a vNIC template to support the management traffic, which we should map to
VLAN 301 and it is sent untagged, therefore we mark it as the default VLAN on this particular
vNIC template. Besides that we first create a QoS policy to support the marking of the traffic to
CoS 1.
97
We already complimented the QoS settings of the network in the network uplink settings,
therefore we can skip it in this section.
Second we create vNIC templates for the 2 different fabrics where we are able to transport al
the VLANs between the network and the UCS and we need to ensure that the QoS marking is
kept when traffic is received.
To support the marking of the CoS settings, we need to trust the blade in a separate QoS policy.
98
Ensure you select the correct type of template (Updating) and select the MAC pools as we
created them for fabric A and fabric B.
Now we have created our vNIC templates and we can start using them in our soon to be
created service profile templates (and service profiles).
99
Task 7: vHBA template

Next are the vHBA templates. vHBA templates are very similar to vNIC templates, except that
they define the template for the FCoE interface (or vHBA) from the NIC in the blade to the
Fabric Interconnect.
Be sure to select the correct VSAN, type of template and WWN pool.

Task 8: Policies
Next we create policies that are going to be used in the service profile template in the next
task.
100
First we have a question about the local disks used in the blades. We need to create a policy so
the disks are not used in the configuration. This means that we create a disk policy set to Any
Configuration, because when we would set the configuration to be No Local Disks, it will fail
to associate to the blade as the blades will contain disks.
Next we create a firmware management and a host firmware package so we are sure that our
blades run the correct version of software at all times!

101
Next we create a new policy to support the user acknowledgement when maintenance is
necessary on the blade.
102
Finally we need to make sure that disk contents and bios settings are maintained when a
service profile is disassociated from a blade.
Task 9: Service Profiles
Now we can start working on the real configuration of the blades. Meaning we will start
configuring the Service Profile Template that we are applying our settings to the UCS Blades
that we configured in previous tasks.
103
Ensure you are using all the pools and policies which we previously configured.
104
We assign the ANY disk configuration policy and start configuring our vHBAs using the Expert
configuration.
After creating the Storage configuration we configure the Ethernet networking configuration,
also using the Expert configuration mode to ensure we can use the vNIC templates that we
previously configured.
105
Next we leave the NIC placement as the system suggests. This only becomes interesting when
we are configuring a system using the VIC-1240 or VIC-1280 cards, where we can have different
placements in the same blade. Or in a full width blade you can determine which vNIC is hosted
on which physical mezzanine card.
106
Next we create a custom boot policy to ensure that our system boots up from the correct
settings as is described in the question.

We create an initial SAN boot target and secondly we create another SAN boot target based on
the different fabrics.
107
Next we assign our previously configured User Acknowledgement policy.
108
Finally we assign our firmware management policy and we will assign our servers later to this
service profile template.
Last step is to assign the scrub policy that we configured to ensure that nothing is erased as
soon as we disassociate the service profile from the blade.
109
Task 10: Server boot
Now we can create a service profile out of the template. We need to assign them to 3 servers.
Therefore we create 3 Service Profiles with the correct naming prefix.
110
Task 11: Cloning
Next we need to create a copy of the Service profile and need to adapt it so that it will support
a server, which does not have the Cisco VIC card. This means a number of things.
First of all only the VIC card supports fabric fail-over. Therefore we need to make sure we
disable the fail over on the management vNIC. Next the non-VIC cards only support 2 vNICs per
mezzanine card. Therefore we will change the dataA vNIC to make sure that the management
vNIC is deleted fully.
111
Next we change the boot policy so that the server is no longer booting from Fibre Channel, but
now booting from its local disks.
112
Now the final step is to associate the server to the fourth blade in the chassis and we are
finished!
Task 12: Management

The final task of our mock lab is to configure Active Directory authentication of UCS Manager.
The UCS Manager supports LDAP authentication and the task gives you a lot of details that you
should use to configure this LDAP authentication.
Now in the lab there will not be an Active Directory server to test it out, but we can assume
there is one and configure the UCS authentication based on all information given to us.

113
114

Now we configured the Active Directory authentication. The final task is to configure the
authentication of groups in Active Directory. Pay close attention to how you configure this as
the full LDAP path needs to be configured there and not just the group name!
115
116
Now you finished Mock Lab 2! Count your points and if you scored 80 points or above you
received a PASS!
117
CCIE Data Center Mock Lab Challenge Chapter 22 Detailed Solution Guide
Chapter 22: Mock Lab Solution

Guide
Chapter 22 of the Mock Lab solution guide was a lab designed to test you heavily on L2 and UCS.
Several issues existed in the intial configuration that you would have needed to resolve
successfully in order to complete the lab.
A copy of the entire configurations for the solution are available as text downloads in your
Ebooks/Downloads section of the IPExpert workbook so you can verify your solution against the
configuration. Note that the CCIE Exams are outcome based: you can use whatever method you
want to configure a solution as long as it meets the requirements and does not violate any
restrictions set out in the general rules or in the explicit instructions of the question.
Use the verification show commands in this guide as your primary method of marking a question
as correct or incorrect, not by comparing your configuration with the sample solution
configuration provided. The solutions may vary!
This workbook was written with pride by IPExpert staff. We love feedback! If you have any bugs
you encounter, or you just want to chat about the workbook maybe you found it too easy or it
was too hard send us an email at feedback@ipexpert.com so we can continuously improve the
product.
General Rules

You will need to pre-configure the network with the base configuration files
NOTE: Static/default routes are NOT allowed unless otherwise stated in the task
NOTE: Unless otherwise noted in the task you can add user cisco pw cisco to the local
database to test management access to the device
Estimated Time to Complete:
8-10 Hours
Solutions
1.0 Data Center Configuration
(32 points)
Task 1.1: Initial Setup (2 Points)
Configure the switches with the following VLANs and be sure to name them as per the table
below
VLAN
110
Switch
SW1-1,SW1-2,SW2,SW3
Name
AcmeCorp-Data
120
SW1-1,SW1-2,SW2,SW3
AcmeCorp-Voice
130
SW1-1,SW1-2,SW2,SW3
AcmeCorp-DMZ
210
MegaCorp-Data
220
MegaCorp-Voice
230
MegaCorp-DMZ
500
SW1-1,SW1-2,SW1-3,SW1-4
Spine1
600
SW1-1,SW1-2,SW1-3,SW1-4
Spine2
10
SW1-1,SW1-2,SW1-3, SW1-4,SW2,SW3
NFS
100
SW1-1,SW1-2,SW1-3, SW1-4,SW2,SW3
iSCSI-Network
310
SW2,SW3
AcmeCorp-VSAN310
320
SW2,SW3
AcmeCorp-VSAN320
410
SW2,SW3
MegaCorp-VSAN410
420
SW2,SW3
MegaCorp-VSAN420
Detailed Solution
SW1-1
vlan 10
mode fabricpath
name NFS
vlan 100
mode fabricpath
name iSCSI-Network
vlan 110
name AcmeCorp-Data
vlan 120
name AcmeCorp-Voice
vlan 130
name AcmeCorp-DMZ
vlan 210
name MegaCorp-Data
vlan 220
name MegaCorp-Voice
vlan 230
name MegaCorp-DMZ
vlan 500
mode fabricpath
name Spine1
vlan 600
mode fabricpath
name Spine2
SW1-2
vlan 10
mode fabricpath
name NFS
vlan 100
mode fabricpath
name iSCSI-Network
vlan 110
name AcmeCorp-Data
vlan 120
name AcmeCorp-Voice
vlan 130
name AcmeCorp-DMZ
vlan 210
name MegaCorp-Data
vlan 220
name MegaCorp-Voice
vlan 230
name MegaCorp-DMZ
vlan 500
mode fabricpath
name Spine1
vlan 600
mode fabricpath
name Spine2

SW1-3
vlan 1
vlan 10
mode fabricpath
name NFS
vlan 100
mode fabricpath
name iSCSI-Network
vlan 500
mode fabricpath
name Spine1
vlan 600
mode fabricpath
name Spine2

SW1-4
vlan 1
vlan 10
mode fabricpath
name NFS
vlan 100
mode fabricpath
name iSCSI-Network
vlan 500
mode fabricpath
name Spine1
vlan 600
mode fabricpath
name Spine2
SW2
vlan 1
vlan 10
name NFS
vlan 100
name iSCSI-Network
vlan 110
name AcmeCorp-Data
vlan 120
name AcmeCorp-Voice
vlan 130
name AcmeCorp-DMZ
vlan 210
name MegaCorp-Data
vlan 220
name MegaCorp-Voice
vlan 230
name MegaCorp-DMZ

SW3
vlan 1
vlan 10
name NFS
vlan 100
name iSCSI-Network
vlan 110
name AcmeCorp-Data
vlan 120
name AcmeCorp-Voice
vlan 130
name AcmeCorp-DMZ
vlan 210
name MegaCorp-Data
vlan 220
name MegaCorp-Voice
vlan 230
name MegaCorp-DMZ

I am sure if you are going for the CCIE Data Centre you know how to correctly configure VLANs
with appropriate names, just be careful with Case Sensitivity.

Verification
SW1-1# show vlan
VLAN Name
Status
Ports
---- -------------------------------- --------- -----------------------------1
default
active
Eth4/1, Eth4/2, Eth4/5, Eth4/6

Eth4/9, Eth4/10, Eth4/11
Eth4/12, Eth4/25, Eth4/26
Eth4/27, Eth4/28, Eth4/29
Eth4/30, Eth4/31, Eth4/32
10
NFS
active
100
iSCSI-Network
active
110
AcmeCorp-Data
active
120
AcmeCorp-Voice
active
130
AcmeCorp-DMZ
active
210
MegaCorp-Data
active
220
MegaCorp-Voice
active
230
MegaCorp-DMZ
active
500
Spine1
active
600
Spine2
active
Task 1.2: L3 Initial configuration (2 Points)
Configure the following L3 Interfaces
VLAN
100
Switch
SW1-3
IP Address
10.0.100.1/24
10
SW1-4
10.0.10.1/24
110
SW2
10.100.10.1/24
210
SW3
10.200.10.1/24
Detailed Solution
SW1-3
feature interface-vlan
interface vlan 100
ip add 10.0.100.1/24
no shut
!
SW1-4
interface vlan 10
ip add 10.0.10.1/24
no shut
!
SW2
interface vlan 110
ip add 10.100.10.1/24
no shut
!
SW3
interface vlan 210
ip add 10.200.10.1/24
no shut
!

The most important thing to remember is the feature interface-vlan command, this allows you to
create the Layer 3 VLAN interfaces. It is also important when you are dealing with a real world
network to ensure that the Nexus 5000 you are configuring has the Layer 3 Daughtercard.
SW3# show module
Mod Ports
Module-Type
Model
Status
--- -----
-------------------------------- ---------------------- ------------
32
O2 32X10GE/Modular Universal Pla N5K-C5548UP-SUP
active *
O2 Daughter Card with L3 ASIC
ok
N55-D160L3

And the LAN base license:
SW3# show license usage
Feature
Ins
Lic
Status Expiry Date Comments
Count
-------------------------------------------------------------------------------FCOE_NPV_PKG
Yes
Unused Never
FM_SERVER_PKG
No
Unused
ENTERPRISE_PKG
Yes
Unused Never
FC_FEATURES_PKG
Yes
Unused Never
VMFEX_FEATURE_PKG
No
Unused
ENHANCED_LAYER2_PKG
Yes
Unused Never
LAN_BASE_SERVICES_PKG
Yes
In use Never
LAN_ENTERPRISE_SERVICES_PKG
Yes
Unused Never
--------------------------------------------------------------------------------
Verification
Simply execute a show ip int brief on each device and ensure it has a L3 interface as defined in
the table.
SW3(config)# show ip int brief
IP Interface Status for VRF "default"(1)
Interface
IP Address
Interface Status
Vlan210
10.200.10.1
protocol-up/link-up/admin-up
Configure vPC between SW1-1 and SW1-2 using only the following interfaces for the vPC
peer link

Switch
SW1-1
Interface
Eth3/9
SW1-2
Eth3/10
You may use any vPC domain ID you choose.

Configure a keepalive mechanism between SW1-1 and SW1-2 using a dedicated L3
interface on each switch as per the table below

Switch
SW1-1
Interface
Eth3/11
SW1-2
Eth3/12
Use any IP addressing information you desire for this keepalive link, but ensure it is
located within its own dedicated VRF. Name the VRF IPExpertVRF
Ensure that in the event of both switches failing, but only one rebooting successfully and
turning on successfully that after 240 seconds the switch will restore vPC functionality.

Detailed Solution
Now we have some nice meaty questions! Lets take a look. When it comes to vPC its always best
to perform your work in a very particular order:

Get everything sorted out for the keepalive
Configure the keepalive
Configure any common settings
Configure the peerlink
If you follow this order your vPC will come up for you straight away every time.

SW1-1
feature vpc
vrf context IPExpertVRF
exit
int eth3/11
vrf member IPExpertVRF
% Deleted all L3 config on interface Ethernet3/11
ip add 169.254.1.1/24
vpc domain 100
peer-keepalive destination 169.254.1.2 source 169.254.1.1 vrf IPExpertVRF

SW1-2
feature vpc
vrf context IPExpertVRF
exit
int eth3/11
vrf member IPExpertVRF
% Deleted all L3 config on interface Ethernet3/11
ip add 169.254.1.1/24
vpc domain 100
peer-keepalive destination 169.254.1.1 source 169.254.1.2 vrf IPExpertVRF

So the first part of the configuration involves setting up the vPC keepalive mechanism, because
we are using a Single Nexus 7000 with multiple VDCs and trying to vPC them together we cannot
use the mgmt0 port: the Mgmt0 port blocks communication between VDCs on the same switch
i.e. Switch 1-1 to Switch 1-2 cannot communicate on the mgmt0 interface.
10
Therefore in order to support the vPC peer keepalive we configure a dedicated Layer 3 interface
between the switches. In order to ensure that this peer keepalive functions correctly, which is
very important to maintaining vPC functionality, we create a dedicated VRF so that routing
protocols etc within the network cannot affect this vPC keepalive. Finally we are told we can
choose any IP addressing scheme we desire, the best choice is the 169.254 range which is set
aside as a dedicated non-routable subnet.
We choose the vPC domain ID 100, we can use whatever domain ID we want but this MUST be
different for each vPC pair in your network as the Domain ID is used in forming the LACP system
identifier.
The next step is to configure the vPC peer-link, but before we do this, one of the questions asks
us to enable auto-recovery. Auto-recovery meets the requirements set out in this queston. Auto
recovery is a vPC feature that solves the following problem: Lets say you have two vPC Switches,
SW1 and SW2, and that both of these switches turn off, possibly because of a power failure, then
the power is restored but SW2 is dead: a power supply has blown or something else has
happened. When SW1 starts, without auto-recovery set the vPC will never come up on SW1.
Auto-recovery resolves this situation by telling SW1 that after 240 seconds (the default timeout
value) if it has not had a vPC peer come up, to assume that SW2 is dead and become the vPC
primary.

SW1-1
vpc domain 100
auto-recovery
Warning:
Enables restoring of vPCs in a peer-detached state after reload, will wait
for 240 seconds to determine if peer is un-reachable
int eth3/9
channel-group 169 mode on
no shut
!
int po169
switchport
vpc peer-link
!
11
SW1-2
vpc domain 100
auto-recovery
Warning:
Enables restoring of vPCs in a peer-detached state after reload, will wait
for 240 seconds to determine if peer is un-reachable
int eth3/10
channel-group 169 mode on
no shut
!
int po169
switchport
vpc peer-link
!
Verification
Lets show the commands I executed as I configured, the first is to verify that the keepalive will
work with the new VRF:
SW1-2# ping 169.254.1.1 vrf IPExpertVRF
PING 169.254.1.1 (169.254.1.1): 56 data bytes
Request 0 timed out
SW1-2# show vpc

Legend:
vPC domain id
: 100
Peer status
: peer link not configured
12
: peer is alive
Configuration consistency status
: failed
: failed
Configuration inconsistency reason: vPC peer-link does not exist

: failed
Type-2 inconsistency reason
: vPC peer-link does not exist
vPC role
: none established
: 0
Peer Gateway
: Disabled
: -
: Disabled (due to peer configuration)
Auto-recovery status
: Disabled
Once the peer-link was configured and auto-recovery enabled, the show vpc output should show
as below:
SW1-1# show vpc
Legend:
vPC domain id
: 100
Peer status
: peer is alive
Configuration consistency status
: success
: success
Type-2 inconsistency reason
: Consistency Check Not Performed
vPC role
: primary
: 0
Peer Gateway
: Disabled
: -
: Enabled
Auto-recovery status
: Enabled (timeout = 240 seconds)

--------------------------------------------------------------------id
Port
Status Active vlans
--
----
------ --------------------------------------------------
Po169
up
1,10,100,110,120,130,210,220,230,500,600
13

Configure vPC between SW2 and SW3 using a domain ID of your choosing.
Use mgmt0 for keepalive mechanism
Ensure SW2 is the vPC Primary
Use all available links between SW2 and SW3 for the vPC Peer link.
Configure a back to back vPC from Sw2 and SW3 to SW1-1 and SW1-2
Ensure that this back to back vPC forms port channels using a negotiation protocol
Detailed Solution
In this queston we are going to configure a vPC domain between SW2 and SW3, then back-to-
back it with our SW1-1 and SW1-2 peers, this is where having the same domain-ID for your vPC
would trip you up: if you have the same vPC domain ID configured on each of these pairs then the
vPC will never come up. We also have to ensure that SW2 is our primary. Finally, we will be using
a negotiation protocol to bring this port-channel up, and the only negotiation protocol available
for port-channels on NX-OS is LACP (PAGP has been depreciated).
It is best to configure the role priority and any other vPC options you might use BEFORE you bring
up the peer-link. That way you dont have to flap the vPC peer-link up and down in order for the
change to take effect.
SW2
Feature lacp
Feature vpc
vpc domain 200
role priority 254
peer-keepalive destination 10.10.210.51
int eth1/5 - 8
channel-group 170 mode active
int po170
vpc peer-link
!

SW3
Feature lacp
14
Feature vpc
vpc domain 200
peer-keepalive destination 10.10.210.52
int eth1/5 - 8
int po170
vpc peer-link
!
Next we configure our back-to-back vPC with SW1-1 and SW1-2

SW1-1
Feature lacp
int eth3/1,eth3/3,eth3/5,eth3/7
no shut
int po180
switchport
vpc
!

SW1-2
Feature lacp
int eth3/2, eth3/4, eth3/6, eth3/8
no shut
int po180
switchport
vpc
!

SW2
int eth1/1 4
15

no shut
int po180
switchport
vpc
!

SW3
int eth1/1 - 4
no shut
int po180
switchport
vpc
!

Verification
Verification of this task is pretty straight forward, is our vPC up? Is SW2 the primary? Is our Port-
channel (back-to-back vPC) configured between our vPC peers?
We can safely ignore the Type-2 inconsistency Errors (SVI Type-2 Configuration incompatible) as
we know we have diffirent routing interfaces on each switch. We will be looking at resolving this
later.
SW2# show vpc
Legend:
vPC domain id
: 200
Peer status
: peer is alive

: success
: failed
16
: SVI type-2 configuration incompatible
vPC role
: primary
: 0
Peer Gateway
: Disabled
: -
: Enabled

--------------------------------------------------------------------id
Port
Status Active vlans
--
----
------ --------------------------------------------------
Po170
up
1,10,100,110,120,130,210,220,230,310,320,410,420
SW2# show vpc role

vPC Role status
---------------------------------------------------vPC role
: primary
Dual Active Detection Status
: 0
vPC system-mac
: 00:23:04:ee:be:c8
vPC system-priority
: 32667
vPC local system-mac
: 54:7f:ee:c2:7d:01
vPC local role-priority
: 254
SW1-1# show vpc 180

vPC status
---------------------------------------------------------------------id
Port
Active vlans
--
----
------ ----------- ------
------------
180
Po180
up
1,10,100,11
success
success
0,120,130,2
10,220,230,
500,600
Task 1.5: FabricPath Configuration (6 Points)
Configure SW1-3 and SW1-4 for fabric path and enable Fabric Path on the interfaces
connecting these two switches
17
Configure Fabric Path on SW1-2 and SW1-1, ensuring all F-Line-card ports facing towards
SW1-3 and SW1-4 are enabled for fabric path
To make identification of these switches easier, ensure the switches are assigned the
following Switch IDs:
Switch
SW1-3
Switch-ID
130
SW1-4
140
SW1-2
120
SW1-1
110
The following VLANs should be set to FabricPath VLANs
Switch
500
Switch-ID
FabricPath
600
FabricPath
100
FabricPath
10
FabricPath
SW1-1 and SW1-2 are the leaf switches in this configuration, configure spanning-tree as
appropriate in such a design bearing in mind that SW1-1 and SW1-2 are vPC Peers and
that we want to avoid any STP convergence issues should the vPC primary switch fail (I.E.
Both switches should be sending BPDUs)
All areas of FabricPath should be authenticated including Adjacencies and updates using
the key CCIEDC-IPEXPERT
Detailed Solution
18
Quite a bit of work to do with this question! We follow a process just like with vPC: get everything
ready to activate FabricPath with all the settings we want then no shut the links when everything
is finished. There is also a hidden little vPC question in this workbook so watch out! The question
is the one that refers to SW1-2 and SW1-1 being the leaf switches: this means you want to set the
spanning-tree root to these switches, but they are in a vPC configuration so we need to take that
into account, and we have to make sure they are both sending BPDUs. We need to use peer-
switch on the vPC.
A few things to watch out for: Dont forget to set the Switch-id, when youre doing a key-chain
your key number (in our example below, key 0) must match between the peers and finally there
are TWO methods of authentication used by Fabricpath, the first is an adjacency authentication,
the second is an ISIS update authentication mechanism, we pretty much go over every possible
fabricpath configuration in this Lab so if you score well on this, you can be comfortable with your
fabric-path expertise!
SW1-1
Install feature-set fabricpath
Feature-set fabricpath
fabricpath switch-id 110
key chain IPEXPERT
key 0
key-string CCIEDC-IPEXPERT
vlan 500,600,10,100
mode fabricpath
spanning-tree vlan 1-4094 priority 8192
vpc domain 100
peer-switch
fabricpath domain default
authentication-type md5
authentication key-chain IPEXPERT
!
interface Ethernet4/11 - 12
switchport mode fabricpath
fabricpath isis authentication-type md5
fabricpath isis authentication key-chain IPEXPERT
no shutdown

19
SW1-2
key chain IPEXPERT
key 0
vlan 500,600,10,100
mode fabricpath
!
spanning-tree vlan 1-4094 priority 8192
vpc domain 100
peer-switch

no shutdown

SW1-3
key chain IPEXPERT
key 0
vlan 500,600,10,100
mode fabricpath
!
20

no shutdown

SW1-4
key chain IPEXPERT
key 0
vlan 500,600,10,100
mode fabricpath
!
no shutdown

Verification
We have lots of good verification commands for Fabricpath that we can use to verify we
configured it correctly. One of the questions asks us to make sure we are using every possible link
between the switches to provide the fabricpath functionality, a great way to verify the ability of a
particular port to support Fabricpath (or indeed any other technology) is the show interface
capabilities command:
SW1-4(config)# show int eth4/21 capabilities
Ethernet4/21
Model:
N7K-F132XP-15
Type (SFP capable):
10Gbase-(unknown)
Speed:
1000,10000
Duplex:
full
Trunk encap. type:
802.1Q
Channel:
yes
Broadcast suppression: percentage(0-100)

Flowcontrol:
rx-(off/on/desired),tx-(off/on/desired)
21
Rate mode:
dedicated
QOS scheduling:
rx-(8q4t),tx-(3p5q1t)
CoS rewrite:
yes
ToS rewrite:
yes
SPAN:
yes
UDLD:
yes
Link Debounce:
yes
Link Debounce Time:
yes
MDIX:
no
Pvlan Trunk capable:
yes
Port Group Members:
21-22
TDR capable:
no
FabricPath capable:
yes
Port mode:
Switched
FEX Fabric:
no
dot1Q-tunnel mode:
no

Next lets verify our fabricpath interfaces have adjacancies, multiple adjacancies to the same
switch will show up multiple times when you have multiple links between the switches
SW1-4# show fabricpath isis adj
Fabricpath IS-IS domain: default Fabricpath IS-IS adjacency database:
System ID
SNPA
Level
State
Hold Time
Interface
SW1-3
N/A
UP
00:00:31
Ethernet4/21
SW1-3
N/A
UP
00:00:28
Ethernet4/22

We can use another command to verify that this adjacency is being authenticated:
SW1-4# show fabricpath isis interface eth4/21
Fabricpath IS-IS domain: default
Interface: Ethernet4/21
Status: protocol-up/link-up/admin-up
Index: 0x0002, Local Circuit ID: 0x01, Circuit Type: L1
Authentication type MD5
Authentication keychain is IPEXPERT
Authentication check specified
Extended Local Circuit ID: 0x1A194000, P2P Circuit ID: 0000.0000.0000.00
Retx interval: 5, Retx throttle interval: 66 ms
22
LSP interval: 33 ms, MTU: 1500

P2P Adjs: 1, AdjsUp: 1, Priority 64
Hello Interval: 10, Multi: 3, Next IIH: 00:00:05
Level
1
Adjs
AdjsUp
Metric
CSNP
40
60
Next CSNP
Last LSP ID
00:00:54
ffff.ffff.ffff.ff-ff
Topologies enabled:
Topology Metric
MetricConfig Forwarding
no
40
UP
Finally we check if our switch-IDs have been applied:

SW1-1# show fabricpath switch-id
FABRICPATH SWITCH-ID TABLE
Legend: '*' - this system
=========================================================================
SWITCH-ID
SYSTEM-ID
FLAGS
STATE
STATIC
EMULATED
----------+----------------+------------+-----------+-------------------*110
64a0.e73f.b4c1
Primary
Confirmed
Yes
No
120
64a0.e73f.b4c2
Primary
Confirmed
Yes
No
130
64a0.e73f.b4c3
Primary
Confirmed
Yes
No
140
64a0.e73f.b4c4
Primary
Confirmed
Yes
No
Total Switch-ids: 4

Task 1.6: FabricPath Traffic Engineering (4 Points)
The E4/19 and E4/11 interface on SW1-3 and SW1-1 respectively is a high-cost link that
should not be used if the E4/20 and E4/12 link is available, use traffic engineering to meet
this requirement
Ensure that the broadcast traffic tree used by Fabric Path is rooted at SW1-4 switch.

Detailed Solution
Now we are getting into some fun FabricPath traffic engineering! The first question requires us to
modify the metric of the E4/19 and E4/11 interfaces to make them less desirable, but in order to
be able to do that first of all we check to see what the default metric is.
SW1-1
23
SW1-1# show fabricpath isis interface eth4/11

Fabricpath IS-IS domain: default
Interface: Ethernet4/11
Status: protocol-up/link-up/admin-up
Index: 0x0002, Local Circuit ID: 0x01, Circuit Type: L1
Authentication type MD5
Authentication keychain is IPEXPERT
Authentication check specified
Extended Local Circuit ID: 0x1A18A000, P2P Circuit ID: 0000.0000.0000.00
Retx interval: 5, Retx throttle interval: 66 ms
LSP interval: 33 ms, MTU: 1500
P2P Adjs: 1, AdjsUp: 1, Priority 64
Hello Interval: 10, Multi: 3, Next IIH: 00:00:07
Level
1
Adjs
AdjsUp
Metric
CSNP
40
60
Next CSNP
Last LSP ID
00:00:50
ffff.ffff.ffff.ff-ff
Topologies enabled:
Topology Metric
MetricConfig Forwarding
no
40
UP
Now (that) we know what the metric is, we can modify the metric of the E4/19 and E4/11
interfaces to make them less desirable. This is done as per the below:

SW1-1
fabricpath isis metric 55555
SW1-3
fabricpath isis metric 55555

Next we need to change the root of tree #1. Quick overview: Fabricpath is a replacement for
spanning-tree designed to utilize all links and ensure all links are forwarding, but spanning-tree
was invented to solve the problem that Ethernet does not deal with Loops. FabricPath deals with
loops by treating L2 traffic almost like it is routing the traffic, but there is always traffic in
Ethernet that needs to reach every node (like broadcast, unknown unicast and multicast traffic.).
To deal with this issue fabricpath implements 2 trees that are used to set a path for these types
24
of traffic to flow. Why two trees? So that users with a lot of multicast traffic can ensure that
multicast traffic is spread out a little between available links. The two trees in Fabricpath are tree
1 and tree 2 with Tree 1 delivering all unknown unicast, broadcasts and some multicast traffic
and Tree 2 delivering multicast traffic. The multicast traffic is evenly distributed between the two
trees (although you can change this in the configuration.) Ok now we have that out of the way,
lets see how you change the root of tree 1 and tree 2. In the verification section we will look at
how you verify this.

SW1-4
root-priority 254

Verification
Here is how we can verify the route that fabricpath is now taking:
SW1-3# show fabricpath route
FabricPath Unicast Route Table
'a/b/c' denotes ftag/switch-id/subswitch-id
'[x/y]' denotes [admin distance/metric]
ftag 0 is local ftag
subswitch-id 0 is default subswitch-id
FabricPath Unicast Route Table for Topology-Default

0/130/0, number of next-hops: 0
via ---- , [60/0], 0 day/s 00:17:35, local
1/110/0, number of next-hops: 1
via Eth4/20, [115/40], 0 day/s 00:05:19, isis_fabricpath-default

As you can see from the above our only path that is currently valid to Switch ID 110 is via Eth4/20,
if you were to change the metric back to its default then Eth4/19 would also appear here.
Next lets take a look at how you verify the fabricpath root trees
SW1-2# show fabricpath isis topology summary
25
Fabricpath IS-IS domain: default FabricPath IS-IS Topology Summary

MT-0
Configured interfaces:
Ethernet4/15
Ethernet4/16
Number of trees: 2
Tree id: 1, ftag: 1, root system: 64a0.e73f.b4c4, 140
Tree id: 2, ftag: 2, root system: 64a0.e73f.b4c3, 130

This shows that the root of tree 1 is Switch ID 140, which is switch 1-4 as per our workbook
question.

Task 1.7: vPC enhancement configuration (4 Points)
Configure the following ports On SW2 and SW3 to face down towards the Cisco UCS FI,
each one will act as a separate uplink and thus should not be configured as a port channel.

Switch
SW2
Port
E1/9
VLAN(s)
110,120,130,10,100
SW3
E1/9
110,120,130,10,100
SW2
E1/10
210,220,230,10,100
SW3
E1/10
210,220,230,10,100
Ensure that all ports transition to the forwarding spanning-tree state as quickly as possible
as the Cisco UCS will not send any BDPUs
Ensure that SW2 and SW3 never allow their L3 VLAN 110 and VLAN 210 interfaces to go
into the down state in the event of a VPC peer link failure.
Ensure that if SW3 was to lose its peer link to SW2 and suspend its vPC member ports that
it would also in turn suspend its ports down to the FI so that the FI would know to use
fabric A.

Detailed Solution
26
I hope youve started to have some fun and nice brain teasers by the time you get to this point in
the workbook J. We are now looking at some advanced vPC configuration. The first thing you
might notice is that we are configuring ports on the vPC switch that are NOT part of a port-
channel yet will be in VLANs that have been enabled for vPC (Any VLAN that is allowed to flow
over the peer-link in vPC is considered a vPC VLAN). This means these ports will be vPC orphan
ports which has interesting implications.
By default, a vPC orphan port will NOT be suspended if the peer-link goes down. If the orphan
ports are going down to a device that does not support port-channels but you want it to bring
down the port in the event of a peer-link failure then you need to add the command vpc orphan-
port suspend to the port. This can be useful to a device such as an ASA firewall which in previous
releases of ASA software did not support Etherchannel. In our case the device that is not using
port-channels (but could support it) is the FI.
Finally, we add some commands to the vPC configuration to make sure that if our vPC peer-link
goes down that the L3 interface assigned to a vPC VLAN stays up. By default all L3 interfaces are
suspended in the event that the peer-link goes down to prevent a dual-active situation. But we
can stop this behavior (which in our case is quite useful considering our switches have separate
L3 VLAN interfaces) with the dual-active exclude command.

SW2
switchport trunk allowed vlan 10,100,110,120,130
spanning-tree port type edge trunk
vpc orphan-port suspend
vpc domain 200
dual-active exclude interface-vlan 110
SW3
27


vpc domain 200
dual-active exclude interface-vlan 210

Verification
First lets check how to verify this VLAN is excluded from the dual-active check.
SW3(config)# show vpc
Legend:
vPC domain id
: 200
Peer status
: peer is alive

: success
: failed
: SVI type-2 configuration incompatible
vPC role
: secondary, operational primary
: 2
Peer Gateway
: Disabled
: 210
: Enabled

The orphan port is a bit more complicated to verify, first of all lets use a command to verify that
the port is indeed seen as an orphan-port
SW3# show vpc orphan-ports
28
Note:
--------::Going through port database. Please be patient.::-------VLAN
Orphan Ports
-------
-------------------------
Eth193/1/1
10
Eth1/9, Eth1/10
100
Eth1/9, Eth1/10, Eth1/11
110
Eth1/9
120
Eth1/9
130
Eth1/9
210
Eth1/10
220
Eth1/10
230
Eth1/10

If we now shut the peer-link down, the vPC Port-channels will suspend, but so will our orphan
port:
SW2# show int eth1/10
Ethernet1/10 is down (vpc peerlink is down)
Hardware: 1000/10000 Ethernet, address: 547f.eec2.7cd1 (bia 547f.eec2.7cd1)
MTU 1500 bytes, BW 10000000 Kbit, DLY 10 usec

Task 1.8: FEX Configuration (3 Points)
After careful consideration of the Pros and Cons of eVPC and standard vPC, you have
chosen not to implement eVPC
Configure the FEXs attached to SW2 and SW3 as per the table below

Switch
SW2
Port
Eth1/13
FEX
FEX 192
Ensure each FEX has a description, ### FEX 1XX ### where X is the FEX number

Detailed Solution
29
It may seem counter-intuitive but eVPC is not always the best solution! eVPC is great if you only
have a single FEX, because you can then dual home that single FEX to two 5ks and therefore you
have introduced a certain level of redundancy. However if you have two FEXs it is often better to
just single-home each of those FEXs to a single 5k, your servers can still port-channel across the
two FEXs. The disadvantage of dual-homing the FEX if you have two FEXs is you have no way of
knowing which switch is controlling the FEX, this means you can no longer perform ISSU and you
need to keep the configurations constantly in sync. For more information please see:
http://rednectar.net/2012/08/30/why-i-wouldnt-bother-with-enhanced-vpc/
For those of you who just want to know things in relation to the exam there is no garuantee
either way they could do straight FEX or eVPC, it is entirely up to them! So you need to make sure
youre comfortable doing either method. eVPC will be configured in a later workbook.
As for configuring normal FEX, the only possible tricky part of this configuration is you should
know that you can set a description under the fex using the description keyword.

SW2
Feature fex
fex 192
pinning max-links 1
description "### FEX 193 ###"
!
switchport mode fex-fabric
fex associate 192
channel-group 192
!
interface port-channel192
fex associate 192
!
SW3
Feature fex
fex 193
pinning max-links 1
30
description "### FEX 193 ###"

!
fex associate 193
channel-group 193
!
fex associate 193
!

Verification
A great command to work out what ports are connected to FEXs is the show int fex command.
SW3(config)# show int fex
Fabric
Fex
Port
Fabric
Port State
Fex
Uplink
FEX
Model
Serial
--------------------------------------------------------------192
Eth1/13
Active
N2K-C2248TP-1GE
SSI14310218
---
Eth1/14
Discovered
N2K-C2248TP-1GE
SSI142916SP

This can be exceptionally useful in determining which ports have FEXs attached and with the
serial number you can be sure youre configuring the correct port, this is handy when doing eVPC
configuration.
The show fex command helps verify the FEX is online and ready to go, it also shows the
description we have assigned.

SW2(config-if)# show fex
FEX
Number
FEX
FEX
Description
State
FEX
Model
Serial
-----------------------------------------------------------------------192
---
### FEX 192 ###

--------
Online
Discovered
N2K-C2248TP-1GE
N2K-C2248TP-1GE
SSI14310218
SSI142916SP
31
Task 1.9: vPC Member Port (3 Points)
Configure a vPC port channel down to the Cisco C-Series Server from port 1/15 on SW2
and SW3. This port channel should use no negotation to bring up this port channel.
This Server provides some NFS functionality, so thus should carry the NFS VLAN Only
ensuring this VLAN is untagged.
This port should be configured to bypass listening and learning for Spanning-tree as a
server port should be.
Detailed Solution
This looks like a fairly straightforward question but there is a trick, if you configure this as an
access port you will need to change your configuration later because later on I get you to
configure this port for FCoE, which requires a trunk so you can carry the FCoE VLAN.
We will be adding more VLANs to the allowed list in a later question so dont worry if your
allowed VLAN includes some extra ones right now J.
SW2
switchport trunk native vlan 10
switchport trunk allowed vlan 10
channel-group 129
!
speed 10000
vpc 129
SW3
32

speed 10000
vpc 129

Verification
Verifying this port config is pretty straightforward.
SW3(config-if)# show vpc 129
vPC status
---------------------------------------------------------------------------id
Port
Active vlans
------ ----------- ------ ----------- -------------------------- ----------129
Po129
up
success
success
10

Task 1.10: Access Ports (3 Points)
Configure port E1/11 on SW2 and SW3 for VLAN 100.

Ensure the ports are set to bypass listening/learning
Ensure the ports are untagged for this VLAN
Ensure all traffic is tagged with a CoS value of 4

Detailed Solution
Our final task in the DC Section and a nice easy one with only one special bit of configuration,
there is a command you may not have been aware of that will tag all untagged traffic with a CoS
Value as it enters our switch, which is going to be very useful as we then pass this traffic up to the
Cisco UCS as you will see later. Dont forget to set the Speed to 1000! You need to do this for the
SFP to validate on a N5k, on a 7k it will auto-detect this but not a 5k unfortunately. Note that the
Cisco MDS Gigabit Interface will (not exactly helpfully) always show as Up even if you have not set
the speed on the N5k end so be careful, verify the port is up using the N5k end not the MDS end.

33
SW2
untagged cos 4
switchport access vlan 100
spanning-tree port type edge
speed 1000

SW3
untagged cos 4
switchport access vlan 100
spanning-tree port type edge
speed 1000

Verification
Verify the port is up on the Switch not the MDS as per the detailed solution and dont forget to
set the speed or you will see this:
SW2# show int eth1/11
Ethernet1/11 is down (SFP validation failed)
Hardware: 1000/10000 Ethernet, address: 547f.eec2.7cd2 (bia 547f.eec2.7cd2)
MTU 1500 bytes, BW 10000000 Kbit, DLY 10 usec
reliability 255/255, txload 1/255, rxload 1/255
34
2.0 Storage Configuration
(25 points)
Task 2.1: Initial VSAN Configuration (2 Points)
Configure the following VSAN/VLANs on the respective switches

Switch
MDS1
VSAN
310
VLAN
N/A
MDS1
320
N/A
MDS2
410
N/A
MDS2
420
N/A

Detailed Solution
Very straightforward just need to enable FCoE on the appropriate switches, my advice however is
to scroll down and read the rest of the Storage questions and make sure you dont have one of
your switches needing to be in NPV mode for a later question, otherwise if you enable FCoE now
to enable FCoE-NPV you will need to restart the switch wasting valuable time! So always check
the later questions, this is a simple question but if I had an NPV question for you later on I could
make this very tricky.

SW2
Feature fcoe
vsan database
vsan 310
vsan 320
vlan 310
fcoe vsan 310
vlan 320
fcoe vsan 320
35
SW3
Feature fcoe
vsan database
vsan 410
vsan 420
vlan 410
fcoe vsan 410
vlan 420
fcoe vsan 420

MDS1
vsan database
vsan 310
vsan 320

MDS2
vsan database
vsan 410
vsan 420
Verification
Use show vsan database and show vlan fcoe to verify this configuration.
MDS2(config)# show vsan
vsan 1 information
name:VSAN0001
state:active
interoperability mode:default
loadbalancing:src-id/dst-id/oxid
operational state:down
vsan 410 information
name:VSAN0410
state:active
vsan 420 information
36
name:VSAN0420
state:active
Task 2.2: Trunking Port Channel (3 Points)
Configure a E SAN-Port Channel Trunk between MDS 1 and SW2 using the table below

MDS1
Fc1/13
SW2
Fc1/31
SAN-Port-Channel-Number
113
Fc1/14
Fc1/32
114
Ensure this is a trunking E port

Verify this port channel is up and trunking correctly.
Hint: (highlight this section to show hint (Can we do that?) You are allowed to make any
changes to the default configuration necessary to bring this port channel up.
Detailed Solution
Did this section trip you up? It may have, this had a troubleshooting aspect to it, the initial config
had an fcdomain ID set for both the 5k and the MDS that was exactly the same and set to static,
the FCDOMAIN id is negotiated but if you specify static it means the fabric will ONLY accept this
particular domain-id and will remain isolated if it cannot be given that particular domain-id (as
compared to the preferred keyword which will request a particular domain-id but accept another
if that domain-id is unavailable.
The key line of config is here:
MDS1
fcdomain domain 200 static vsan 310

SW2
37
Because this is the same on both switches when you bring up the port-channel you will get an
error, lets look at the port-channel config
MDS1
interface fc1/13
channel-group 50 force
no shutdown
!
interface fc1/14
no shutdown
!
interface port-channel 50
switchport mode E
switchport trunk allowed vsan 310
switchport trunk allowed vsan add 320
switchport rate-mode dedicated

SW2
interface fc1/31
no shutdown
interface fc1/32
no shutdown
interface san-port-channel 50
switchport mode E

When this port channel was first brought up, we would have received an error:
38
MDS1# show 2013 Oct 20 07:13:07 MDS1 %FCDOMAIN-2-EPORT_ISOLATED: %$VSAN 310%$

Isolation of interface port-channel 50 (reason: configured domain ID is
different from runtime domain ID)
int fc1/13
MDS1# show int po50 trunk vsan

Vsan 310 is down (Isolation due to domain configuration mismatch)
Vsan 320 is up (None)

Remember that command! Show int <interface> trunk vsan, it is an awesome command to see
why a particular VSAN is showing as isolated or even stuck in initializing.
The only way to resolve this issue is to remove the fcdomain command specifying a manual
switch-id and then either restart the switches (a long process and when youre in the exam youre
limited for time!) or run a special hidden command
MDS1(config)# no fcdomain domain 200 static vsan 310
MDS1(config)# fcdomain restart ?
vsan
Specify the vsan range
MDS1(config)# fcdomain restart disruptive ?

^
MDS1(config)# fcdomain restart disruptive vsan 310

Notice that the disruptive keyword is hidden! So you need to know exactly where to position that
keyword, just after the restart command. Once this command is executed our port-channel will
happily pass VSAN 310.

Verification
Use show the show commands to verify your san-port-channel
SW2# show int san-port-channel 50
Port WWN is 24:32:54:7f:ee:c2:7c:c0
39

Port mode is TE
Port vsan is 1
Speed is 4 Gbps
Trunk vsans (up)
(310,320)
()
()

0 CRC,
0 unknown class

Member[1] : fc1/31
Member[2] : fc1/32
Interface last changed at Sun Oct 20 04:42:02 2013

Another useful command is show port-channel database to check your invidiual member links are
down, in our case fc1/14 was down as this was the wrong kind of transceiver (2 Gig FC instead of
4 Gig FC), once we switched out the transceiver this was resolved. So keep an eye out, if your
port-channel comes up but not all member interfaces come up, check to make sure that those
member interfaces dont differ in some subtle way.
MDS1# show port-channel database

port-channel 50
Administrative channel mode is on
Operational channel mode is on
Last membership update succeeded
First operational port is fc1/13
2 ports in total, 1 port up
Ports:
fc1/13
[up] *
fc1/14
[down]
40
Task 2.3: JBOD Configuration (3 Points)

The JBOD Ports have been preconfigured for you
You will be implementing boot from iSCSI for the ACME blade servers, ensure that JBOD 1
is in VSAN 310 for MDS 1 and 410 for MDS2, and JBOD 2 is in VSAN 320 for MDS1 and 420
for MDS2.
Detailed Solution
Again we had a troubleshooting task here where one of the JBOD ports was set to F, when the
JBOD attached is an FL, this port would not have come up for you, the offending line of code is
here:
MDS2
int fc1/5
switchport mode f
!
This port would not have come up if you did not change this to FL mode, the rest of the relevant
config is shown below

MDS1
vsan database
vsan 310 interface fc1/5
interface fc1/5
switchport trunk mode off
no shutdown
!
interface fc1/6
no shutdown
!

MDS2
41
vsan database
interface fc1/5
no shutdown
!
interface fc1/6
no shutdown
!
Verification
If your JBOD configuration is correct you should see the devices in the flogi database.
MDS1# show flogi database
------------------------------------------------------------------------------INTERFACE
VSAN
FCID
PORT NAME
NODE NAME
------------------------------------------------------------------------------fc1/5
310
0xd50073
20:00:00:11:c6:a6:24:4c
22:00:00:11:c6:a6:24:4c
fc1/5
310
0xd50074
20:00:00:14:c3:a0:68:59
22:00:00:14:c3:a0:68:59
fc1/5
310
0xd50079
20:00:00:14:c3:a0:60:38
22:00:00:14:c3:a0:60:38
fc1/5
310
0xd5007a
20:00:00:11:c6:a6:3c:6f
22:00:00:11:c6:a6:3c:6f
fc1/5
310
0xd50081
20:00:00:14:c3:a0:60:05
22:00:00:14:c3:a0:60:05
fc1/5
310
0xd50082
20:00:00:11:c6:a6:2c:65
22:00:00:11:c6:a6:2c:65
fc1/5
310
0xd5008f
20:00:00:11:c6:a6:3a:36
22:00:00:11:c6:a6:3a:36
fc1/5
310
0xd50090
20:00:00:11:c6:a6:3a:9c
22:00:00:11:c6:a6:3a:9c
fc1/6
320
0xe40059
20:00:00:11:c6:a6:2a:60
22:00:00:11:c6:a6:2a:60
fc1/6
320
0xe40063
20:00:00:14:c3:a0:60:d5
22:00:00:14:c3:a0:60:d5
42
fc1/6
320
0xe40065
20:00:00:11:c6:a6:24:ca
22:00:00:11:c6:a6:24:ca
fc1/6
320
0xe40069
20:00:00:11:c6:a6:ee:8a
22:00:00:11:c6:a6:ee:8a
fc1/6
320
0xe4006a
20:00:00:14:c3:a0:60:1b
22:00:00:14:c3:a0:60:1b
fc1/6
320
0xe4006d
20:00:00:11:c6:87:00:92
22:00:00:11:c6:87:00:92
fc1/6
320
0xe4006e
20:00:00:11:c6:a6:25:de
22:00:00:11:c6:a6:25:de
Task 2.4: E-Port traffic engineering (4 Points)

Configure two E Ports between MDS 2 and SW3

MDS2
Fc1/13
SW3
Fc1/31
Fc1/14
Fc1/32

Configure the above so that port 13 and 31 carry VSAN 410 traffic primarily (with VSAN
420 as backup) and ports 14 and 32 carry VSAN 420 primarily (with VSAN 410 as backup)
Detailed Solution
We give you a rest from troubleshooting tasks here as there are no tricks with this one, just a
straight forward E-port channel, we then use the cost command of fspf (fibrechannel shortest
path first which is based on the same algorithim as OSPF and is basically the routing protocol for
fibre channel) to make one link preferred over the other, as you would expect we can do this on a
per-VSAN basis.

MDS2
interface fc1/13
fspf cost 100 vsan 410
43

no shutdown
interface fc1/14
no shutdown

SW3
interface fc1/31
no shutdown
interface fc1/32
no shutdown
!
Verification
Use the following command to verify what route traffic will take for FSPF.
SW3(config-if)# show fspf internal route vsan 410
FSPF Unicast Routes
--------------------------VSAN Number
Dest Domain
Route Cost
Next hops
----------------------------------------------410
0x8e(142)
100
fc1/31
SW3(config-if)# show fspf internal route vsan 420

FSPF Unicast Routes
--------------------------VSAN Number
Dest Domain
Route Cost
Next hops
44
----------------------------------------------420
0x04(4)
100
fc1/32
Task 2.5: iSCSI implementation (6 Points)
Configure iSCSI on GI1/1 on MDS1 and MDS2 respectively

Configure static targets as per the table below
VSAN
310
Target PWWN
22:00:00:11:c6:a6:24:4c
IQN
410
21:00:00:11:c6:a6:24:4c

Use the following IP addressing information on Gi1/1 on each switch.

Switch
MDS1
IP Address
10.0.100.10/24
MDS2
10.0.100.20/24

Configure the following iSCSI initiators with system-assigned pWWNs

Switch
MDS1
IQN
MDS2
Detailed Solution
This is a fairly big question but if you can pull this off you will be very comfortable with your iSCSI
skills and should be able to get a basic iSCSI up no problem at all (later labs will REALLY test your
iSCSI skills)
45
Lets take a look at the configuration as there is quite a lot involved, we will do the config in parts
and explain each part as we go along.

MDS1 and MDS2
feature iscsi
iscsi enable module 1
iscsi import target fc
interface iscsi1/1
no shutdown
!
interface GigabitEthernet1/1
ip address 10.0.100.10 255.255.255.0
no shutdown

The first few parts are just turning on iSCSI with the feature command, and enabling it for the
module, how do we know which module to enable it for? Whichever module has our Gigabit
interfaces on, so if your gigabit interfaces are 2/1, you would say iscsi enable module 2. In our
case our interfaces are Gi1/1, so if you cant execute the command interface iscsi1/1, the
chances are that you are missing this enable module 1 command. We have to no shut the iSCSI
interface for it to start working.
The iSCSI interface gets its IP Address and other info from the corresponding GI1/1 interface.
MDS1
iscsi virtual-target name iqn.2013-10.com.ipexpert:vsan310
pWWN 22:00:00:11:c6:a6:24:4c
advertise interface GigabitEthernet1/1
all-initiator-permit
!

MDS2
iscsi virtual-target name iqn.2013-10.com.ipexpert:vsan410
pWWN 21:00:00:11:c6:a6:24:4c
advertise interface GigabitEthernet1/1
all-initiator-permit
!
46
Here is where we setup our target. We give the target a name in IQN format, which is basically
iqn.<date>.<domain-name>:string, where <domain-name> is your domain-name you own and
<date> is a date in <year-month> format for a date when you owned that domain. It doesnt have
to be the date the domain was registered or anything like that, it is just a date that the domain
belonged to you. Now of course in the exam if they get you to setup iSCSI they will have an IQN
already specified for you like we do here. The final part :string is basically a free-flow field where
you can put anything meaningful to your organization.
The pWWN is the pWWN of the target you are going to advertise via iSCSI, the advertise interface
specifies which of your interfaces is going to allow connections to this iSCSI target.
With an iSCSI Gateway (which is essentially the functionality the MDS switch is providing here)
you have TWO levels of storage access control to worry about, iSCSI access control and Zoning for
the actual Fibre Channel just like you are used to. The all-initiator-permit command allows any
initiator access to this storage. You can be more granualar and only allow specific initiators.
MDS1
iscsi initiator name iqn.2013-10.com.ipexpert:init1a:3
static nWWN 21:01:00:05:9b:7f:6e:02
static pWWN 21:03:00:05:9b:7f:6e:02
static pWWN 21:04:00:05:9b:7f:6e:02
vsan 310
!
MDS2
iscsi initiator name iqn.2013-10.com.ipexpert:init1a:2
static nWWN 21:03:00:05:9b:7f:aa:42
static pWWN 21:04:00:05:9b:7f:aa:42
static pWWN 21:05:00:05:9b:7f:aa:42
vsan 410
!
The above configures our iSCSI initiators, in the output we have shown a pWWN and nWWN are
assigned but when you create the initiator and you specify Static pWWN one of the keyword
options available to you will be system-assign, its recommended to use this. Once you have
specified system-assign the macro will pick some suitable nWWNs and pWWNs for you.
47
Finally you specify what VSAN the initiator should be placed into when he comes into the fabric. If
all your initiators are on the same VSAN you can just put the actual iSCSI interface into a
particular VSAN under your VSAN Database.

Verification
There are plenty of things that can go wrong with iSCSI so we will show you some of the
verification commands you can use below.
The first and very useful command is show iscsi global
MDS1# show iscsi global
iSCSI/iSLB Global information (fabric-wide)
Authentication: CHAP, NONE
Initiator idle timeout: 300 seconds
Dynamic Initiator: iSCSI
iSLB Distribute: Disabled
iSLB CFS Session: Does not exist
Number of load balanced VRRP groups: 0
Number of load-balanced initiators: 0
iSCSI/iSLB Global information (local to this switch)
Import FC Target: Enabled
Initiator Plogi timeout: 2 seconds
Number of target node: 1
Number of portals: 6
Number of session: 0
Failed sessions: 3, Last failed initiator name: iqn.201310.com.ipexpert::init1a:3

As you can see this tells you how many targets you have setup and also the number of failed
sessions that have been recorded. This can be helpful in troubleshooting. Show iscsi virtual-target
is another good command to verify the config on your static target.
MDS1# show iscsi virtual-target
target: iqn.2013-10.com.ipexpert:vsan310
Port WWN 22:00:00:11:c6:a6:24:4c
Configured node (iSCSI)
No. of advertised interface: 1
GigabitEthernet 1/1
48
All initiator permit is enabled

Trespass support is
disabled
Revert to primary support is
disabled

A gotcha to watch out for comes in the next command, show iscsi initiator
MDS1# show iscsi initiator
iSCSI Node name is iqn.2013-10.com.ipexpert::init1a:3
Initiator ip addr (s): 10.0.100.129
iSCSI alias name: UCS1
Auto-created node (iSCSI)
Node WWN is 21:01:00:05:9b:7f:6e:02 (dynamic)
Member of vsans: 1
Number of Virtual n_ports: 1
Virtual Port WWN is 21:02:00:05:9b:7f:6e:02 (dynamic)
Interface iSCSI 1/1, Portal group tag: 0x3000
VSAN ID 1, FCID 0x010204

In the above output it shows us the initiator logged in and ready to go, but you must be careful!
The show iscsi initiator command will ONLY show iSCSI initiators that are currently logged in.
show iscsi initiator configured on the other hand will show all iscsi initiators that you have
configured
MDS2# show iscsi initiator configured
Member of vsans: 410
Node WWN is 21:03:00:05:9b:7f:aa:42
No. of PWWN: 2
Port WWN is 21:04:00:05:9b:7f:aa:42
Port WWN is 21:05:00:05:9b:7f:aa:42
Task 2.6: FCoE Server Port (4 Points)
Configure an FCoE Connection from N5k1 and N5k2 down to the C Series server
connected on port 1/15 on each switch. Keeping in mind the separation of fabrics.
The vFC should be configured in such a way that it does not rely on the port-channel being
UP in order for the server to correctly login to the fabric.
49
This should carry vsan 310 on SW2 and 410 on SW3 respectively.
Detailed Solution
With this question we attempt to trick you from an earlier question that had you configuring
eth1/15 for just one VLAN, since now you of course need to carry both the FCoE VLAN and the
Data VLAN you need your port to be a trunk port. Whenever it comes to ports like this as well
where you have a Port-channel down to a device, but then have a Fabric A and Fabric B FCoE
Configuration the question is always: what do I bind to?
The answer is easy, always bind to the physical interface and NOT the port-channel, that way if
the port-channel doesnt come up maybe because of LACP negotiations or some other problem
at least this way your storage will come up. This was actually introduced as a feature in an early
release of NXOS as it was an issue people were coming up against, so the ability to bind to a
physical port that is a member of a port-channel was introduced.
The next question you might wonder about is the vPC aspect to this: if my port-channel is in a
vPC, do I need to create the VSAN and VLAN for BOTH fabrics on each switch? Again the answer is
no, your vPC will let you have a switchport trunk allowed list that is diffirent on each switch and
this will not cause a type-1 inconsistency, so you can quite happily do this.
SW2
switchport trunk allowed vlan 10,410
speed 10000
vpc 129
interface vfc15
bind interface Ethernet1/15
no shutdown
vsan database
vsan 410 interface vfc15

SW3
50

switchport trunk allowed vlan 10,310
speed 10000
vpc 129
interface vfc15
bind interface Ethernet1/15
no shutdown
vsan database
vsan 310 interface vfc15
Verification
Lets verify our vFC Port.
vfc15 is trunking
Bound interface is Ethernet1/15
Port WWN is 20:0e:54:7f:ee:c2:7e:ff
Admin port mode is F, trunk mode is on
Port mode is TF
Port vsan is 410
Trunk vsans (admin allowed and active) (410)
Trunk vsans (up)
()
()
(410)

Interface last changed at Sun Oct 20 04:51:56 2013
The VSAN might be stuck in initializing; a great way to verify why is the following command
51
SW3# show int vfc15 trunk vsan

vfc15 is trunking
Vsan 410 is down (waiting for flogi)

We can see that the VSAN is simply waiting for the server to actually FLOGI, something in the
operating system (or on the HBA) must tell the server to FLOGI, so this output is normal in this
situation since we are not doing boot from SAN.

Task 2.7: Zoning (3 Points)
Based on the IQNs created above, create the following zones on MDS1 and MDS2 using
basic zoning, be sure to use the iQN symbolic node names in your zoning.
Members
Zone Name
VSAN310_Zone_Blade1

Members
WWPN 22:00:00:11:c6:a6:24:4c

Members
Zone Name
VSAN410_Zone_Blade1

Members
WWPN 21:00:00:11:c6:a6:24:4c
52

Detailed Solution
The final step in our storage section! The only tricky part of the configuration here is that we are
using symbolic node names for the zone members not the pWWN, this is purely for ease of
configuration.
MDS1
zone name VSAN310_Zone_Blade1 vsan 310
member pwwn 22:00:00:11:c6:a6:24:4c
member symbolic-nodename iqn.2013-10.com.ipexpert:init1a:3
zoneset name VSAN310_Zoneset vsan 310
member VSAN310_Zone_Blade1
zoneset activate name VSAN310_Zoneset vsan 310

MDS2
member pwwn 21:00:00:11:c6:a6:24:4c
member symbolic-nodename iqn.2013-10.com.ipexpert:init1a:2
member symbolic-nodename iqn.2013-10.com.ipexpert::init1a:2
member VSAN410_Zone_Blade1
zoneset activate name VSAN410_Zoneset vsan 410
Verification
As always the best command after we have activated a zoneset to verify is show zoneset active
MDS2# show zoneset active
* fcid 0x8e0073 [pwwn 21:00:00:11:c6:a6:24:4c]
symbolic-nodename iqn.2013-10.com.ipexpert:init1a:2
53

The * next to the fcid indicates that this device is currently logged into the fabric

54
3.0 UCS Configuration
(43 points)

As a cloud services provider, your UCS infrastructure is a common resource between multiple
companies, the UCS configuration below is based on the idea that the infrastructure is shared.
Keep this in mind with all questions and solutions.
Task 3.1: Uplink/Server port configuration (3 Points)
Configure the following ports as Uplink ports

Switch
FI-A
Port
9
FI-A
10
FI-B
FI-B
10

Configure the following ports as Server ports.
Switch
FI-A
Ports
1,3,5,7
FI-B
1,3,5,7
Detailed Solution
In this question we configure our uplinks to the UCS and the storage ports down to the chassis,
we have not done the Uplinks as etherchannel because I wanted to throw in a Disjoint L2
question. Our ports down to our chassis will not form a port-channel from the FI to the IOM
because the IOMs are 2104s which do not support the port-channel configuration, but in the
detailed solution guide below we can see the screenshots of where you would configure this.

55

In the screenshot above you can see we can highlight multiple ports at once and right-click and
designate as server-ports in order to save quite a bit of time! There are lots of instances in UCS
where you can do this so keep an eye out on ways to potentially save yourself a lot of time. The
screenshot below shows the same idea but for the uplink ports.

Verification
It is pretty easy to know if your server-ports have been configured correctly, the chassis will show
up and you will see the IOMs! Another great way is to login to the FI itself, this is something that
not enough people do. It is a great way to troubleshoot! Another tip is that you can specify a or
b at the end of the connect nxos command in order to specify which fabric NXOS you want to
login to!
UCS1-A # connect nxos a
UCS1-A(nxos)# show run int eth1/9 - 10
56
!Time: Sun Oct 20 05:51:19 2013

version 5.0(3)N2(2.05b)
description U: Uplink
pinning border
no shutdown
pinning border
no shutdown
Task 3.2: VLAN Configuration (2 Points)
Configure the following VLANs on UCS

VLAN
110
Name
AcmeCorp-Data
120
AcmeCorp-Voice
130
AcmeCorp-DMZ
210
MegaCorp-Data
220
MegaCorp-Voice
230
MegaCorp-DMZ
10
NFS
100
iSCSI-Network
57
Detailed Solution
Pretty straightforward to add VLANs to UCS, just go to the LAN tab and create the VLANs as per
the screenshot below

As you can see from the above screenshot you can save a lot by creating the VLANs in a single line
BUT you cant change the name later!

Verification
The VLANs will show under the LAN tab.
58
Task 3.3: Disjoint L2 (5 Points)
In order to keep the network traffic separated for MegaCorp and AcmeCorp, configure a
disjoint L2 domain, VLANs 110-130 should travel over the Port 9 uplink on FI-A and FI-B.
VLANs 210 230 should travel over Port 10. The NFS and iSCSI networks are a shared
resource and thus can travel across both uplinks.
Your junior engineer does not understand the concept of designated receiver and its
impact on network traffic, login to the Cisco CLI and run the command to show the
designated receiver for VLAN 110. Save this command and its output as a notepad file on
your desktop.
Detailed Solution
This topic is quite possibly one of the most misunderstood topics within Cisco UCS. A lot of
misinformation out there implies that all you need to do is pick your disjoint interface, add the
VLAN to it and away you go. But this can lead to a network-failure scenario due to something
called the designated receiver.
Lets look at how you do it properly.
Under the LAN Tab, Click the very top of the tree (LAN) as per the screenshot below

59

At the bottom of the screen, click Launch LAN Uplinks Manager

Once this is clicked, click on the VLAN tab then the VLAN Manager sub-tab
60

The important thing to do now, is as per our requirements, we only want the AcmeCorp VLANs to
flow up one link, and the MegaCorp VLANs to flow up another, lets do that as per the screenshot
below, Dont forget to add the uplinks for both Fabric A and Fabric B.

61
This is now the correct configuration. In the verification section we will show how the designated
receiver is involved.

Verification
Login to the Cisco UCS and lets take a look at what our GUI configuration has done to the NXOS
operating system running on the FI.
UCS1-A(nxos)# show run int eth1/9 - 10
!Time: Sun Oct 20 05:51:19 2013
version 5.0(3)N2(2.05b)
pinning border
switchport trunk allowed vlan 1,10,100,110,120,130
no shutdown
pinning border
switchport trunk allowed vlan 1,10,100,210,220,230
no shutdown

As you can see from above, by changing the VLANs in the VLAN manager we have changed the
switchport trunk allowed VLAN command. So each interface carries a particular VLAN. But the
important part comes next.
The designated receiver is a special interface used by Cisco UCS to receive broadcasts and
unknown unicasts, one is randomly chosen from the available uplinks that are CARRYING THAT
PARTICULAR VLAN. Before UCS 2.0 it used to be that the designated receiver was NOT chosen on
a per-VLAN basis, thus to implement disjoint L2 your only option was to switch the FI to switching
mode not end host mode. But with UCS 2.0 they have introduced a designated receiver per VLAN.
There is a command to determine the designated receiver
62
UCS1-A(nxos)# show platform software enm internal info vlandb id 210

vlan_id 210
------------Designated receiver: Eth1/10
Membership:
Eth1/10
UCS1-A(nxos)#
For VLAN 210 the designated receiver chosen is Eth1/10, the membership table shows us that the
only potential designated receiver is Eth1/10. This is expected and is as per our requirements and
everything will work correctly.
vlan_id 100
Membership:
Eth1/9
Eth1/10

vlan_id 10
Membership:
Eth1/9
Eth1/10

For VLAN 10 and VLAN 100 it has been specified that we should allow these VLANs to travel up
both uplinks, therefore both Eth1/9 and Eth1/10 are potential designated receivers. This is fine
because our upstream switches (both the N5ks) have VLAN 10 and VLAN 100 created, but lets
assume that VLAN 10 was not created on SW2, if this was the case, and Eth1/9 was chosen as the
designated receiver, broadcast traffic would not work for VLAN 10 down to the Cisco UCS, it
would never receive any broadcast or unknown unicast traffic on its Eth1/9 interface. You can
imagine this would not make for a functional network! Therefore its important to remember: if
youre creating a disjoint VLAN, you must make sure that you also specify the other VLANs should
63
only be allowed out the other Trunk. Lets take a simpler example, in the screenshot below, this is
how many people configure disjoint L2, however it is INCORRECT

In this example, Eth1/3 has been specified as the ONLY interface to carry traffic for VLAN 55
(Disjoint VLAN). But the problem is that the other VLANs will also attempt to travel over this link
as well (eth1/3). The danger is in if Eth1/3 is chosen as the designated receiver, because on the
interface northbound to Eth1/3 (on your Nexus 5k) youre likely to have a switchport trunk
allowed vlan list that only allows VLAN 55 (The disjoint VLAN). This leads to a variety of network
problems.
So the correct way in the example above to finish this would be to specify an uplink (or multiple
uplinks) for each VLAN so that they will only choose a designated receiver from those chosen
uplinks rather than the default behavior which is to choose a designated receiver from any uplink
port at all.

Task 3.4: SAN Connectivity (6 Points)
Although SAN Connectivity is not required for initial deployment, MegaCorp have
requested you provision the network in preparation for SAN Connectivity in the near
future. The ports on the FI are Ports 2/1 and 2/2 and the ports on the MDSs are FC1/9 and
FC1/10
Configure the following VSANs and VLANs on Cisco UCS, Where VSAN 310 and 410 are
used by the AcmeCorp, and VSANs 410 and 420 are used by MegaCorp.
64
VSAN
310
Mapped VLAN
310
Fabric
FI-A
320
320
FI-A
410
410
FI-B
420
420
FI-B
The storage uplinks between the FIs should be able to handle multiple VSANs, they
should also be configured as a SAN-Port-Channel in order to provide the highest possible
bandwidth.
Your junior engineer often has difficulty setting up a SAN Port channel from UCS to other
storage devices, this is often because he does not know what configuration Cisco UCS will
place onto the SAN Port channel when configured from the GUI, show him the commands
required on the UCS CLI to see the configuration applied to your SAN port channels and
paste the output into notepad, then save on your desktop.
Detailed Solution
This question requires us to support a port-channel up to the Cisco MDS, make sure the port-
channel is trunking and then copy some important output for our junior engineer, lets start with
each step
The first step is to enable the FI for trunking mode on its uplinks. This then applies switchport
trunk mode on for all FC trunking ports.
Go to the SAN tab, click on each individual fabric and then on the general tab you will see Enable
FC Uplink Trunking
65

Once this is checked, Create the VSANs ensuring to select only Fabric A or Fabric B for each VSAN
66
Once this is done, create the port-channel

Make sure you remember to enable the port-channel.

Next we need to configure MDS1 and MDS2. Cisco UCS uses the channel mode active command
under the Port-Channels and this cannot be removed, this is FCs way of negotiating port
channels so be sure to include this in your configuration. You also need to enable NPIV mode
since the Cisco UCS is operating in NPV Mode, and you also need to enable fport-channel-trunk
feature to support the use of the trunking F port. The other thing to keep in mind when doing F
port-channels, you do NOT need to specify rate-mode dedicated, rate-mode dedicated is ONLY
required for E Trunking port channels. The whole point of rate-mode dedicated is that an E port, a
trunking port is likely to be carrying a lot of traffic from one FC switch to another, but in the case
of an F port, obviously you are just carrying traffic down to a server so potentially the traffic
should be less than from one FC switch to another.
MDS1
feature npiv
feature fport-channel-trunk
67
channel mode active
switchport mode F
switchport rate-mode shared
interface fc1/9
no shutdown
interface fc1/10
no shutdown

MDS2
feature npiv
feature fport-channel-trunk
channel mode active
switchport mode F
switchport rate-mode shared
interface fc1/9
no shutdown
interface fc1/10
no shutdown
68
We will cover the final task of showing the Junior engineer how the port-channels are configured
on Cisco UCS in our verification section.

Verification
Login to the Cisco UCS NXOS and look to see how Cisco UCS has configured the SAN-Port-channel:
UCS1-A(nxos) # interface san-port-channel 10
channel mode active
switchport mode NP
switchport trunk mode on
!

You can show your junior engineer that this is how you tell how Cisco UCS has configured the
port-channel. This can be extremely useful when you are troubleshooting SAN connectivity issues
as you can tell what configuration you need to match on your MDS or 5k.
To verify your SAN connectivity from the UCS, use the following command:
UCS1-A(nxos)# show npv status
npiv is enabled
disruptive load balancing is disabled
External Interfaces:
====================
Interface:
fc2/1, State: Waiting For VSAN Up
Interface:
fc2/2, State: Waiting For VSAN Up
Interface:
fc2/3, State: Pre-Initialized
Interface:
Interface:
Interface:
Interface:
Interface:
Interface: san-port-channel 10, State: Trunking

VSAN:
320, State: Up
VSAN:
310, State: Up, FCID: 0xd50100
69
Number of External Interfaces: 9

Task 3.5: Pool Configuration (3 Points)
Two organizations must be created within Cisco UCS, AcmeCorp and MegaCorp, create
these two organizations and then assign the following UUID, MAC address, WWPN and
WWNN Pools
Organization
AcmeCorp
Pool Type
Mac
Pool Name
MAC_POOL
Value
00:25:B5:00:00:00
Size
32
AcmeCorp
UUID
UUID_POOL
Derived (Prefix)
32
Suffix (000A-000000000001)
AcmeCorp
IQN
IQN_POOL
Prefix: iqn.2013-10.com.ipexpert
Block: init1A
Start with: 0
AcmeCorp
Iscsi Initiator
N/A
10.0.100.100-10.0.100.131/24 (GW:
10.0.100.1) (DNS: N/A)
32

Detailed Solution
In this question we will be creating each of the Pools we require, the screenshots below show the
various stages of the pool creation, be sure to create the suborganizations before creating the
pools as per below:
70

We will not show screenshots of the creation of every single pool as this would take up needless
space, just be sure to create each Pool under the appropriate organization as shown in the
screenshot below
71
Verification
The only way to verify the Pools is to check they have been created as per the screenshot below
which shows verification for our IQN Pool
Task 3.6: Jumbo MTU Support (6 Points)
Both iSCSI and NFS, like FC traffic are crucial bits of storage traffic that should be assigned
a class that implements Pause frames and their MTU should be able to reach the
maximum allowed on the nexus platform. Assign to Class 4 CoS 4.
The north Nexus 5k Switches from the FI should support this configuration.
Continue up the storage network and implement this configuration all the way to MDS1
and MDS2. Our final goal will be to ensure that our iSCSI and NFS vNICs on our server
blades are able to connect to the 10.0.100.10 and 10.0.100.20 iSCSI Target Portal IP
addresses with an MTU of 9216 with no fragmentation (dont forget about IP overheads,
so exact value may not be 9216). You are allowed to make all necessary changes to L3 and
L2 MTU configuration.
Detailed Solution
This question is worth a lot of points so you can imagine it is not 100 percent straightforward,
quite a bit of configuration is required in your QoS to get this working. The first step is to enable
the jumbo frames within the QoS-Group under Cisco UCS as per the screenshot below

72

Other than the vNIC needing to have a tagged QoS class which we will deal with later this is the
only change we need to make to UCS, Enable the Class (Gold), set a CoS Value (4), disable packet-
drop (we want to disable packet drop for iSCSI and NFS traffic), and then configure the MTU to
the highest possible value supported on this hardware platform 9216.
The bulk of the configuration is on the Nexus Switches. The first thing we need to do is match all
traffic coming into the Switch that has a CoS 4 setting (which will be all traffic coming from the
Cisco UCS and also all traffic coming from the ports attached to the MDS switches (if you recall,
we tagged their traffic with CoS 4 in task number 1.10) and place it into qos-group 2 which we will
do something with shortly:

SW2 and SW3
class-map type qos match-all class-nfs-iscsi
match cos 4
policy-map type qos fcoe-storage-in-policy
class class-fcoe
set qos-group 1
class class-nfs-iscsi
set qos-group 2
class class-default
service-policy type qos input fcoe-storage-in-policy

Next we set the behavior for traffic (no drop, MTU etc) that matches our qos-groups, be careful
to include the FCoE Traffic (which is placed into qos-group 1)! Otherwise you will have major
problems with your FCoE interfaces.

73
SW2 and SW3

class-map type network-qos class-nfs-iscsi
match qos-group 2
policy-map type network-qos fcoe-storage-in-policy
pause no-drop
mtu 2158
class type network-qos class-nfs-iscsi
pause no-drop
mtu 9216
multicast-optimize
system qos
service-policy type network-qos fcoe-storage-in-policy

The final step is to set our Gi1/1 interfaces on the MDS to the appropriate MTU value:

MDS1 and MDS2
interface GigabitEthernet1/1
switchport mtu 9216

Verification
There are some very good verification commands for this on the nexus platform, these same
commands would work on the Cisco UCS if you wanted to add more verification
The first command, show policy-map system is a great way to verify what exact QoS policies are
currently applied on the switch. The output below shows the policy-map that would be applied
after issuing the feature fcoe command, it is useful to issue this command before you make any
changes to the QoS policies so you can check what classes are already included and make sure
you include them in your own policies.
SW3(config)# show policy-map system

===============================
74
policy-map type network-qos fcoe-default-nq-policy

match qos-group 1
pause no-drop
mtu 2158
match qos-group 0
mtu 1500
multicast-optimize
Service-policy (qos) input:
policy statistics status:
Class-map (qos):
fcoe-default-in-policy
disabled
class-fcoe (match-any)
Match: cos 3
set qos-group 1
Class-map (qos):
class-default (match-any)
Match: any
set qos-group 0
Service-policy (queuing) input:
Class-map (queuing):
fcoe-default-in-policy
disabled
Match: qos-group 1
Match: qos-group 0
Service-policy (queuing) output:
fcoe-default-out-policy
disabled
Match: qos-group 1
75
Match: qos-group 0
The show queuing command is a great way to verify your traffic is being classified correctly,
Observe the below, we start a ping on the MDS Switch attached to SW2
MDS1# ping 10.0.100.20 size 2000 timeout 2

Then we verify traffic is being classified correctly:
SW2# show queuing interface eth1/11
Ethernet1/11 queuing information:
TX Queuing
qos-group
sched-type
oper-bandwidth
WRR
50
WRR
50
WRR
RX Queuing
qos-group 0
q-size: 240960, HW MTU: 1500 (1500 configured)
drop-type: drop, xon: 0, xoff: 240960
Statistics:
Pkts received over the port
: 0
Ucast pkts sent to the cross-bar
: 0
Mcast pkts sent to the cross-bar
: 0
Ucast pkts received from the cross-bar
: 0
76
Pkts sent to the port
: 0
Pkts discarded on ingress
: 0
Per-priority-pause status
: Rx (Inactive), Tx (Inactive)
qos-group 1
drop-type: no-drop, xon: 20480, xoff: 40320
Statistics:
: 0
: 0
: 0
: 0
: 0
: 0
qos-group 2
drop-type: no-drop, xon: 17280, xoff: 37120
Statistics:
: 20
: 20
: 0
: 18
: 19
: 0
Total Multicast crossbar statistics:

Mcast pkts received from the cross-bar
: 1

As you can see from the output, traffic is being placed into Qos-group 2 as we ping from the MDS
switch.

Task 3.7: vNIC Template (4 Points)
Create a vNIC template for iSCSI and NFS for AcmeCorp Only
77
These templates should not be configured for a method of failover that is transparent to
the operating system: storage traffic should utilize a separate Fabric A/Fabric B
configuration.
Name these templates iSCSI-vNIC-A and NFS-vNIC-A for Fabric A, iSCSI-vNIC-B and NFS-
vNIC-B for Fabric B.
VLAN 100 should be native VLAN for iSCSI and VLAN 10 is native for NFS
These vNICs should support Jumbo MTUs.
The Template should be configured in such a way that changes to the template at a later
date are not reflected on vNICs that were created based off the template.
Detailed Solution
The order you perform vNIC related tasks could save you quite a bit of time in the lab, you should
be very careful to read all the questions regarding a vNIC template to ensure you use your time
wisely. In the example above we will need a QoS policy to ensure our vNICs support jumbo
frames because in our previous question where we enabled jumbo frames we did NOT change
the default MTU for the default class, therefore we need a QoS policy
78
The rest of our vNIC creation is fairly straightforward. As per the question we do not enable
transparent failover, we set the MTU to 9000 within the vNIC itself so that it tells the operating
system this vNIC supports an MTU of up to 9000 and finally we set the QoS Policy
79
Verification
There is no real verification method to verify a vNIC template other than double-checking you
have selected the right items. You can also login to the Cisco UCS FI and verify the veth
configuration but this is not necessary.

Task 3.8: Description Support (2 Points)

The Physical Server Blade 1 was purchased by AcmeCorp. In order to easily show this fact, ensure
the GUI reflects this as per the screenshot below:
Detailed Solution
For those of you who know about the labels in Cisco UCS this is a nice quick 2 points, for those
who dont this could take a little while to find. Most objects in UCS support a label field, and by
filling this in you can attach descriptions that will show in the GUI for those particular objects as
per the screenshot below
80

Fill in the user-label shown in the bottom right hand corner and look for user label in objects to
set it elsewhere.

Verification
N/A

Task 3.9: Service Profile Configuration (4 Points)
Create a service profile called iSCSIBlade under the AcmeCorp organization using the
pools assigned previously
The vNIC templates should be utilized in the creation of the iSCSI NIC as per the table
below

vNIC
iscsi-A
Template
iSCSI-vNIC-A
nfs-A
NFS-vNIC-A
iscsi-B
iSCSI-vNIC-B
nfs-B
NFS-vNIC-B
81
Do not create any vHBAs

Ensure your server uses a local disk configuration policy that can only be applied to
servers with enough disks to support RAID 0.
Detailed Solution
Once again it is important when configuring something like a service-profile to read the question
carefully to see if there are any dependency objects you are going to want to create before you
go and create the service profile: being half way through a service profile creation only to realize
there are some extra steps you need to go and do and therefore having to create the service-
profile from scratch again can really eat into your time. In task 3.10 we are also going to be
configuring boot from iSCSI which will require us to create some iSCSI vNICs. Again if you did not
read the whole section or the entire exam you would have to go back and create the iSCSI vNICs
later which could lose you valuable time. For the sake of this detailed solution guide we will show
the iSCSI vNIC creation for task 3.10 under task 3.10
In this question we are asked to create a disk policy that only allows for RAID 0, the screenshot
below shows this local disk policy being created

Next we create our service profile, selecting our previously created UUID pool. We will skip over
the vHBA section as we have been told not to worry about this since we are doing boot from ISCSI
and instead the screenshot below shows the creation of the vNICs based on our previously
created templates.
82
83
Verification
The best method of verification is to now assign this service profile to Blade 1/1 as per the
directions. When you assign to a blade that only has 1 disk or no disks at all, since your local disk
policy is set to RAID 0 which requires at least 2 disks, you would receive the following error
message

84
Task 3.10: Boot from SAN (5 Points)
Configure two iSCSI overlays to be used for boot from SAN

Name the overlays iscsioverlay-A and iscsioverlay-B respectively
Create a boot from SAN iSCSI Policy called iSCSI-Boot
Assign this boot policy to your service profile and make the necessary iSCSI parameter
changes, the destination static target should be iqn.2013-10.com.ipexpert:vsan310 for
Fabric A and iqn.2013-10.com.ipexpert:vsan410 for Fabric B
Please note the server will not boot a copy of ESX, you do not have to successfully boot the
server into an operating system, just prepare the server so that it will install to a SAN disk and
boot from SAN in the future.

Detailed Solution
Youre almost at the finish line! Our first step is to create two iSCSI overlay vNICs. Be sure to set
the MAC address assignment to None used by default, make sure you select the appropriate
VLAN and appropriate overlay NIC as per the screenshot below
85

Next you need to create the iSCSI boot policy, and add the iSCSI vNICs into the policy. Be VERY
CAREFUL when entering the name of the iSCSI vNIC as per the screenshot below to ensure that it
matches the name of the iSCSI vNICs you just created earlier. Remember: Case Sensitive
86
Once you have created the boot policy, you need to assign it to your service profile and then
modify the boot parameters as per the screenshot below
87
Click on the iSCSI vNIC and click Set iSCSI Boot Parameters then fill in the iSCSI Target Name and
IP address as per the screenshot below
88
Make sure you do this for both iSCSI vNICs and change the IP address as appropriate.
Verification
Boot from SAN either iSCSI or FC is enough to strike fear into many CCIE DC candidates, so many
things can go wrong! Fortunately there are some very good boot from SAN troubleshooting tools
bit into Cisco UCS, you just need to connect to the adapter: Thats correct, you can actually LOGIN
to the Cisco UCS VIC cards and issue commands on those cards to get detail on your boot from
iSCSI or boot from FC!
Then we verify traffic is being classified correctly:
UCS1-A# connect adapter 1/1/1
adapter 1/1/1 # connect
adapter 1/1/1 (top):1# attach-mcp
adapter 1/1/1 (mcp):1# help
Available commands:
adv_uifetscfg - Show advertised uif ets config
amp-dump - Dump AMP internals
amp-env - Dump AMP data
amp-stats - Dump AMP stats
89
bmc_chan - Show BMC Channels

bmc_env - Dump global BMC state
bmc_macs - Show BMC MAC usage
cfgblk - Show configuration block
dcbx-env - Dump environment
dcbx-port - Dump port information
dcem-env - Dump dcem environment
dcem-macstats - Display mac stats
dcem-port - Show dcem port
dcem-showlinkval - Show link startup vals
def_uifetscfg - Show default uif ets config
exit - Exit from subshell
fipd - Shows fipd status
fwdtab-show - Show forwarding table
fwdtab-tcamreg - Show tcam region table
help - List available commands
history - Show command history
iscsi_get_config - Get iSCSI config for all vnics
iscsi_ping - Test iSCSI connectivity
iscsi_show_ibft - Show iBFT posted
lif - List lif state/lif commands
lifstats - Show lif stats
lifstats_lifbase - Show lif stats baseline
lifstats_lifdelta - Show lif stats delta from baseline
lifstats_logical_uplink - Show logical uplink stats for uif
lifstats_uifbase - Show uif stats baseline
lldp-env - Show environment
lldp-port - Show port information
ptifs - Show palotool interfaces
uif - Show uif port
uifcfg - Show uif config
uifenv - Show uif env
uifetscfg - Show uif ets config
uiflldpcnt - Dump the lldp packet counter
uiflldppktinfo - Show history of lldp packets rcvd
uifportprofile - Show uif port profiles
uifportstatus - Show status for uif sub-port
uifqoscfg - Show uif qos config
vic-ccstats - Dump completion code stats
vic-dump - Dump VIC structures
90
vic-env - Show global env

vic-mstats - Dump VIC stats per msgtype
vic-stats - Dump VIC stats
vic_enum - Enumerate VIFs(1), VIF-LISTS UIFs(2), or UIFs(3)
vic_get_negotiation - Show VIC msg/req (none), FW ver (0), SW ver (1),
Nego ver (2), Nego req (3)
vic_stats_get - Get statistics of VIF(1) or UIF(2)
vicapp - List vic app state
vicappstats - List vic app stats
vicapptimer - vic app timer off/on
vif - List vif state
vif_long - List vif state in a long format
viflist - List viflists
viflist_stats - Show viflists stats
vnic - List vnic info/send vnic cmds
vnicfind - Find vnic by key (mac,type)
vnicl - List detailed vnic info
vnicpci - List vnic pci state
vnicpcibr - List bridge pci state
adapter 1/1/1 (mcp):2#
As you can see there are lots of options under the adapter that you can execute, I have
highlighted some of the more interesting ones that you might want to look at, but lets check out
some of the iSCSI ones that will be useful to verify that our boot from iSCSI has worked correctly.
adapter 1/1/1 (mcp):3# iscsi_get_config
vnic iSCSI Configuration:
----------------------------
vnic_id: 5
link_state: Up
Initiator Cfg:
initiator_state: ISCSI_INITIATOR_READY
initiator_error_code: ISCSI_BOOT_NIC_NO_ERROR
vlan: 0
dhcp status: false
91
IQN: iqn.2013-10.com.ipexpert::init1A:3
IP Addr: 10.0.100.129
Subnet Mask: 255.255.255.0
Gateway: 10.0.100.1
Target Cfg:
Target Idx: 0
State: ISCSI_TARGET_READY
Prev State: ISCSI_TARGET_DISABLED
Target Error: ISCSI_TARGET_NO_ERROR
IQN: iqn.2013-10.com.ipexpert:vsan310
IP Addr: 10.0.100.10
Port: 3260
Boot Lun: 0
Ping Stats: Success (9.877ms)
Session Info:
session_id: 0
host_number: 0
bus_number: 0
target_id: 0
adapter 1/1/1 (mcp):4# iscsi_ping

id
name
tgt
address
port tcp ping status
--- -------------- --- --------------- ----- --------------------------------5 vnic_1
10.0.100.10
3260 Success (9.877ms)
6 vnic_2
10.0.100.20
3260 Success (9.580ms)
The above output shows that we logged into the iSCSI target. If you want a more traditional
method of verifying, we can jump onto the MDS1 switch and issue show commands to prove that
the iSCSI login has been successful:
MDS1# show iscsi initiator
Initiator ip addr (s): 10.0.100.129
92
iSCSI alias name: UCS1

Node WWN is 21:01:00:05:9b:7f:6e:02 (configured)
Member of vsans: 310
Number of Virtual n_ports: 1
Virtual Port WWN is 21:03:00:05:9b:7f:6e:02 (configured)
Interface iSCSI 1/1, Portal group tag: 0x3000
VSAN ID 310, FCID 0xd50101
MDS1# show zoneset active
* fcid 0xd50073 [pwwn 22:00:00:11:c6:a6:24:4c]
* fcid 0xd50101 [symbolic-nodename iqn.2013-10.com.ipexpert::init1a:3]
Task 3.11: Locale Implementation (3 Points)
Create a Locale called AcmeLocale for AcmeCorp and a Locale Called MegaLocale for
MegaCorp
Create an admin user for AcmeCorp called AcmeAdmin and a user for Megacorp called
MegaAdmin
Ensure these users only have access to the appropriate locales.
Detailed Solution
The first step is to create our Locales as per the screenshot below (Admin User Management
User Services Locales)
93

Fill in the user-label shown in the bottom right hand corner and look for user label in objects to
set it elsewhere.
Assign the appropriate organization to AcmeLocal by Dragging it onto the AcmeLocal in the right-
hand pane. This is not exactly intuitive but simple enough to do

Next create a locally authenticated user under user management and ensure they have the
appropriate locale set as per the screenshot below, ensure their role is set to admin
94
Verification
Log in as your newly created user and ensure you only have access to shared resources and Acme
Resources.
95

IPE DC Lab

Hochgeladen von

Dokumentinformationen

Copyright

Verfügbare Formate

Dieses Dokument teilen

Dokument teilen oder einbetten

Freigabeoptionen

Stufen Sie dieses Dokument als nützlich ein?

Sind diese Inhalte unangemessen?

Copyright:

Verfügbare Formate

IPE DC Lab

Hochgeladen von

Copyright:

Verfügbare Formate

CCIE Data Center Lab Preparation Workbook

Chapter 21: Mock Lab

Estimated Time to Complete: < 8 hours

Connect to all devices within the topology

Copyright by IPexpert. All rights reserved.

CCIE Data Center Lab Preparation Workbook

Drawing 1: Physical Topology

Copyright by IPexpert. All rights reserved.

CCIE Data Center Lab Preparation Workbook

Copyright by IPexpert. All rights reserved.

CCIE Data Center Lab Preparation Workbook

Verify all pre-configuration as there is troubleshooting involved in this mocklab!

Copyright by IPexpert. All rights reserved.

CCIE Data Center Lab Preparation Workbook

Data Center Networking

Task 1: VDC allocations

Ethernet3/9, Ethernet3/11, Ethernet3/13, Ethernet3/15

Ethernet3/10, Ethernet3/12, Ethernet3/14, Ethernet3/16

Ethernet3/1, Ethernet3/3, Ethernet3/5, Ethernet3/7, Ethernet3/17, Ethernet3/19,

Ethernet3/2, Ethernet3/4, Ethernet3/6, Ethernet3/8, Ethernet3/18, Ethernet3/20,

Copyright by IPexpert. All rights reserved.

CCIE Data Center Lab Preparation Workbook

SW1-1, SW1-2, SW2, SW3

SW1-1, SW1-2, SW2, SW3

SW1-1, SW1-2, SW2, SW3

1. Assign the following VLANs with names.

Task 3: Trunk interfaces

Copyright by IPexpert. All rights reserved.

CCIE Data Center Lab Preparation Workbook

CCIE Data Center Lab Preparation Workbook

5. You should be certain that SW2 has the primary role

Copyright by IPexpert. All rights reserved.

CCIE Data Center Lab Preparation Workbook

Task 8: First Hop Redundancy

Copyright by IPexpert. All rights reserved.

CCIE Data Center Lab Preparation Workbook

Task 10: OTV

Copyright by IPexpert. All rights reserved.

CCIE Data Center Lab Preparation Workbook

Task 11: DC2 VLANs

Task 12: FabricPath #2

Copyright by IPexpert. All rights reserved.

CCIE Data Center Lab Preparation Workbook

7. FabricPath authentication using a text password of FPauth should automatically be

Task 13: QoS

Task 14: Port-Channel

Copyright by IPexpert. All rights reserved.

CCIE Data Center Lab Preparation Workbook

Copyright by IPexpert. All rights reserved.

CCIE Data Center Lab Preparation Workbook

Task 3: Access interfaces #1

Copyright by IPexpert. All rights reserved.

CCIE Data Center Lab Preparation Workbook

CCIE Data Center Lab Preparation Workbook

Copyright by IPexpert. All rights reserved.

CCIE Data Center Lab Preparation Workbook

Task 8: Access interfaces #2

Copyright by IPexpert. All rights reserved.

CCIE Data Center Lab Preparation Workbook

Task 1: Chassis initialization

Task 3: VSANs and Uplinks

Copyright by IPexpert. All rights reserved.

CCIE Data Center Lab Preparation Workbook

10.10.210.91/24 (GW: 10.10.210.254)