Understanding PVLANs in UCS

Introduction

This document explains the PVLAN support in UCS - a feature which is

introduced in the Balboa (1.4) release. This document explains the feature,
the caveats and configuration when using PVLANs with bare metal OS
and/or in conjunction with hypervisor switch like Nexus 1000v which
supports PVLANs too.

Background Theory
A private VLAN is a VLAN you configure to have Layer 2 isolation from
other ports within the same private VLAN. Ports belonging to a private
VLAN are associated with a common set of supporting VLANs that are used
to create the private VLAN structure.

There are three types of private VLAN ports: promiscuous, isolated, and
community.

•A promiscuous port communicates with all other private VLAN ports and is
the port you use to communicate with routers, backup servers, and
administrative workstations.

•An isolated port has complete Layer 2 separation, including broadcasts,

from other ports within the same private VLAN with the exception of the
promiscuous port.

•Community ports communicate among themselves and with their

promiscuous ports. These ports are isolated at Layer 2 from all other ports
in other communities or isolated ports within their private VLAN.
Broadcasts propagate only between associated community ports and the
promiscuous port.

Privacy is granted at the Layer 2 level because the switch blocks outgoing
traffic to all isolated ports. You assign all isolated ports to an isolated VLAN
where this hardware function occurs. Traffic received from an isolated port
is forwarded to all promiscuous ports only.

Within a private VLAN are three distinct classifications of VLANs: a single

primary VLAN, a single isolated VLAN, and a series of community VLANs.

RFC 5517 defines PVLANs theory and operations and is suggested for
reading to get a good understanding of the concepts behind PVLANs -
http://tools.ietf.org/html/rfc5517

PVLAN implementation in UCS

The important points are

a) Only Isolated ports are supported in UCS.

b) A server vNIC in UCS cannot carry both regular and isolated VLANs.
c) No support for Promiscuous ports/trunks, Community ports/trunks or
Isolated trunks.
d) Promiscuous ports need to be outside the UCS domain i.e upstream
switch/router.
Network Topology and Configuration
The configuration example in this document is for the topology described in
Figure 1.
The desired behavior is that Blade 1, VM1, VM2 and Blade 3 cannot
communicate with each other as they will part of the same isolated VLAN
while all of them should be able to communicate to the L3 port on the
Catalyst 6500 upstream which is configured as a promiscuous port.

Configuration
For this example, the following VLAN’s will be used.
Primary VLAN – 40

Secondary (Isolated) VLAN – 400

vNIC0 to the ESX host will carry the isolated VLAN.

UCS Configuration
Create the Primary VLAN (VLAN 40 in this example) in the VLAN tab

Similarly create the Secondary VLAN (VLAN 400 in this example) and
associate it with the Primary VLAN
Creating vNIC for a blade running bare metal OS (Linux/Windows) is
straight forward as the isolated VLAN needs to be chosen and set as the
Native VLAN. Fabric Failover can be enabled if required and supported by
the adapter in the blade to which the Service Profile will be assigned.

Creating vNICs for an ESX host is different and usually requires trunks
extended to the blade. As mentioned earlier, PVLANs and regular VLANs
cannot be extended on the same vNIC and also a VNIC can only have one
isolated VLAN.

This implies that a vNIC needs to be defined just to carry the isolated VLAN
to the ESX blade running Nexus 1000v.

In case of M81KR (Palo) adapter, it can be accomplished by creating vNICs

as per requirement.

In case of the M71KR E/Q (Menlo) adapters which are characterized by a

maximum of 2 vNICs, one vNIC can be defined as trunk to carry traffic for
Service Console, VMotion, Control, Packet etc (which can have Fabric
Failover enabled for redundancy) and one for carrying the isolated VLAN.
Note: In case of Menlo’s only 1 isolated VLAN is possible.

In case of the other adapters (82598KR, M61KR, M72KR E/Q and M51KR)
which do not support Fabric Failover, private VLANs are not feasible with
the Nexus 1000v if redundancy is required as the 2 available vNICs need
to be configured to back each other up.

Catalyst 6500 Configuration

Define the Primary and Secondary VLANs
vlan 40

private-vlan primary

private-vlan association 400

vlan 400

private-vlan isolated

Configuration of L3 interface which is configured as promiscuous

interface Vlan40

ip address 40.40.40.250 255.255.255.0

private-vlan mapping 400

Interface configuration of the trunk connecting to the FI
interface TenGigabitEthernet3/2

switchport

switchport trunk encapsulation dot1q

switchport trunk allowed vlan 10,20,40,180,400

switchport mode trunk

end

Nexus 1000v Configuration

Define the Primary and Secondary VLANs
vlan 40

private-vlan primary

private-vlan association 400

vlan 400

private-vlan isolated

Define the uplink port profile which will be assigned to the pNIC
port-profile type ethernet pv-lan

vmware port-group

switchport mode trunk

switchport trunk native vlan 40

switchport trunk allowed vlan 40,400

channel-group auto mode on mac-pinning

 no shutdown

state enabled

Define the veth port profile which the VM’s will consume
port-profile type vethernet vms

vmware port-group

switchport mode private-vlan host

switchport private-vlan host-association 40 400

no shutdown

state enabled

PVLANs with VMware DVS

As seen with the Nexus 1000v configuration, the uplink port-profile defined
on the VEM is a trunk with native set as the primary VLAN for the vNIC
which carries the isolated VLAN.

As all traffic on that vNIC as sent by the FI is untagged, it is processed by

the Nexus 1000v as coming in on the primary (native VLAN) and is
fowarded to the VM isolated ports.

VMware DVS does not give the option to configure native VLAN on uplinks
and hence PVLANs with DVS and UCS are currently not supported.