Issue

EMC Symmetrix disk array configuration in Common Serial Number Mode [C-bit setting]

Solution
Dynamic Multi-Pathing (DMP) works with the EMC Symmetrix disk array only if the disk array is
configured in the Common Serial Number Mode [C-bit set]. Without this mode, DMP cannot
provide multipathing functionality.
Until EMC Symmetrix started supporting devices above FFF, if digits 6 to 8 of the EMC
Symmetrix disk serial number are zeroed out then the C-bit is set and the array is configured in
the Common Serial Number Mode as required by DMP. But after EMC Symmetrix started
supporting devices above FFF, they used the last bit to set it to 8 to represent a device above
0xFFF. This "8" is added with a logical "OR" operation so that you device would end with 378
(370 or 008) on port 7dA and 3A8 (3A0 or 008) on port 10dA.
With the "C" bit, the serial number must be identical (common) on all the ports. The "8" remains
to represent a device above 0xFFF, the the rest is set to 0. 000 or 008 = 008.
The disk serial number can be obtained with various commands including the VERITAS
vxdmpinq, vxdmpdebug command or the vxexplorer output.
/etc/vx/diag.d/vxdmpinq /dev/rdsk/c3t12d15s2
Vendor id : EMC
Product id : SYMMETRIX
Revision : 5266
Serial Number : 1004E000a
or
/etc/vx/diag.d/vxdmpinq /dev/rdsk/c3t5006048C52A820D6d766s2
Inquiry for /dev/rdsk/c3t5006048C52A820D6d766s2, evpd 0x0, page code 0x0
Vendor id
: EMC
Product id
: SYMMETRIX
Revision
: 5772
Serial Number
: 273E2008a
The serial number above shows that digits 6-8 are zeroed out. The array is setup in Common
Serial Number Mode as required by DMP.
DMP needs to see identical device serial numbers from each path.
If the C-bit is not set then DMP sees multiple disks of the same device through different paths.
If the C-bit is set then DMP sees one unique disk device through multiple paths.
The layout for the EMC Symmetrix disk serial number is:

1004E000a (or) 273E2008a

SSVVVDDPx
SS = Last 2 digits of the Symmetrix serial number
VVV = Symmetrix volume number in hex. 000-FFF
DD = Director number [host adapter] 1-32
P = Port number on director card
x = lower case letter which does not belong to serial number according to EMC.
Note: If the C-bit is not set, data on an existing disk can easily be wiped out by disk operations
to the other path which appears like a different disk all together. Setting the C-bit nulls out digits
6 through 8 of the serial number.
Consult the Storage Foundation Hardware Notes for a complete list of DMP pre-requisites
regarding EMC Symmetrix arrays.

Legacy ID
325308

Article URL http://www.symantec.com/docs/TECH70715

Common Serial Number (C)

Host SCSI Compliance 2007 (OS2007)

SCSI-3 SPC-2 Compliance (SPC2)

SCSI-3 compliance (SC3)

Common serial number (C)

Auto negotiation (EAN) enabled

Fibrepath enabled on this port (ACLX)

SCSI 3 (SC3) (Optional)

Unique Worldwide Name (UWN)

SPC 2 (Decal) (SPC2) SPC 2 flag set

Common serial number (C)

Auto negotiation (EAN) enabled

1.

Fibrepath enabled on this port (VCM)

SCSI 3 (SC3) (Optional)

PTOP for FC-SW OR HDAD for FC-AL

VCM if using Volume Logix (optional)

UWN for Unique WWN

SC3 for heterogeneous port sharing (optional)

C or common serial number bit

Common Serial Number (C)

Host SCSI Compliance 2007 (OS2007)

SCSI-3 SPC-2 Compliance (SPC2)

SCSI-3 compliance (SC3)

For FC Switch Base Topology (FCSW), Enable Auto Negotiation (EAN), PointtoPoint (PP) , Unique

WWN (UWN).
For FC Loop Base Topology (FCAL), Enable Auto Negotiation (EAN), Unique World Wide Name (UWN)

1. What are the "port flags" settings for 5875 Enginuity and what do they mean?
Port flags in Enginuity are used to set behavior that enables a connected server/operating system
combination to work with the Symmetrix port (seen as a disk drive to the server) and to optimize
the operation of that server and disk combination. Symmetrix VMAX with 5875 is delivered
with the correct default port flag settings for most users so that few or no changes will be needed
during installation of the new system.
EMC strives to make setup and operation of our products as simple and easy as possible. We
believe that Enginuity 5875 for VMAX is delivered to you with correct default port flag settings
for most users and that few or no changes will be needed during installation of your new system.
Here is a summary of the default flag settings for the most popular operating systems. and
changes required by some of those operating systems.

2. What Servers can connect to 5875 Enginuity without changing Port Flags?
Operating Environment
Port Settings
EMC Celerra
Enable 'ARB' and 'D' flags
IBM AIX
5875 Defaults, Unchanged
INM i
Enable 'AS4' flag
LINUX
Enable 'D' flag
Hewlett-Packard OpenVMS
Enable 'OVMS' flag
Hewlett-Packard True64
Enable 'OVMS' flag, Disable 'SC3 flag
Hewlett-Packard HP-UX 11i V1
Enable 'V' flag, Disable 'SC3' and 'OS07'
&V2
flag
Hewlett-Packard HP-UX 11i V3
Enable 'V' flag, Disable 'SC3'

Microsoft Windows Server 2008

Microsoft Windows Server 2003
Oracle Solaris
VMware ESX

5875 Defaults, Unchanged

5875 Defaults, Unchanged
5875 Defaults, Unchanged
5875 Defaults, Unchanged

3. What are some additional considerations?

Directly connecting servers to the array, not using switches or other SAN components,
you are using Fibre-Channel Arbitrated Loop. This requires that the PP flag to be DISABLED
and a Loop ID assigned to a particular number ranging from 0-126. FC-AL is used in certain
markets and with some specialty servers.

Using Fujitsu (formerly Siemens), Novell Netware or Teradata systems, please look at
the support matrix at EMC.com for specific recommendations.

Use of Clusters can impact port flag settings. Please look at ELab's Host Connectivity
Guide of the particular operating system for information for details of a cluster's required port
flag settings.
4. Port Flag usage notes:

These new default values are in some cases different from the 5874 settings. Connection
of a server to both 5874 and 5875 Enginuity may require changes to the 5875 defaults or the user
to set the flags at the Initiator to match the previously installed array with its existing connection
to the server. Note in general that changes of the flags will not become effective until the host is
rebooted or FC connectivity is bounced.

Script behavior can change based upon these new 5875 default Port Flag settings.
Existing scripts should be tested to ensure port flag settings and scripts are compatible.

Your VMAX is set to operate with the ACLX flag ENABLED, which enables masking of
specific devices to specific ports on the array. Note that user must still provision ACLX volumes
to enable hosts management via Solution Enabler.

The OS07 flag, ENABLED by default, can generate warnings (I/O retry) on Solaris,
OpenVMS, and some other hosts. Some high performance or sensitive applications may be
degraded by the additional retry. EMC believes we support the SCSI standard correctly, and
retries should not occur, but that said, it can happen on some systems. Such environments
should DISABLE this flag. Although the OS07 flag is on by default, specifically for
VMWARE ESX this setting may need to be disabled.
Note: that if you DISABLE the OS07 flag and wish to re-ENABLE it, you must be sure that

theSPC2 flag is ENABLED first. Also note that the SPC2 flag value should not be changed from
DISABLED to ENABLED on installed servers as it changes the disk name utilized by the server

and the disks will no longer be visible to that server.

If SCSI-3 persistent device reservation bit (PER) is used (note that this flag is assigned
to devices and not to ports and so is not covered by this document), then the SC3 bit should be
set. Please refer to the Elab Host Connectivity Guide for documentation on the platforms that
can utilize the SCSI-3 reservation bit.
For additional information regarding to port flags, specific flags for a given server, impacts of
changing flags and any other known issues, please go to EMC.com and search PRIMUS for
additional port flag information.

5. How did 5875 Enginuity Port Flag Defaults Change from earlier releases?

The following is the table of default settings for the commonly set flags for 5875 Enginuity
versus other recent Enginuity releases:
Flag
PP

UWN
EAN
C
SPC2
OS07
ACLX
SC3

Enginuity 5773
Enabled
Enabled
Enabled
Enabled
Enabled

Enginuity 5874
Enabled
Enabled
Enabled
Enabled
Enabled

Disabled
Disabled
Disabled

Disabled
Disabled
Disabled

Enginuity 5875
Enabled
Enabled
Enabled
Enabled
Enabled
Enabled
Enabled
Enabled

6. What are the port flag descriptions?

Flags come in two types. The first type are flags set only on the arrays storage port. These flags
affect the connectivity behavior of all servers connected to that port. The second type of flags
can be set per storage port or initiator group, a group defined on the array as one or more
initiators (HBAs) for a given server. With this type, the flags settings for the initiator group
override, for the group, the settings on the port itself. In this way, a port can be shared by servers
with different flag requirements.
The following table list where the flags should be set:

Flag Flag Description

PP

When enabled, specifies a point-to-point

(direct or switched)

Flags that can be set

Flags set only at
at
the Storage Portor
the StoragePort Initiator Groups
YES
N/A

topology in the initialization sequence. When

disabled (default),
it is initialized as an arbitrated loop.
When enabled, allows two fibre ports to
handshake
and settle on an optimal speed for data
EAN transfer.
YES
When this flag is set, it enables the Initiator
ACLX Groups/Access Logix1 function on the port. YES
When enabled, ensures unique World Wide
Names (WWNs)
within the fibre environment by
incorporating Symmetrix serial
numbers and port numbers for each WWN.
Enabled by default
for all environment configuration changes
and new environments.
When disabled, WWNs do not have to be
UWN changed.
YES
When enabled along with the Hard
Addressing flag, the
Fibre Channel director only uses hardassigned addressing
when it initializes the loop, otherwise, softassigned
addressing is used during loop initialization
(the default).
YES
NP
When enabled, this flag causes the serial
number
of a specific volume to be the same
regardless of
C
the port on which it is visible.
N/A
When enabled along with the Disk_Array
flag
for HP-UX hosts, the volume setaddressing
mode
is selected. VSA mode allows octal
V
addressing.
N/A
Typical behavior on a SCSI bus reset is to
propagate the reset to
all channels that share devices mapped to the
port that received
the reset. When this bit is set, the reset only
occurs to the port on
ARB which it is received, and is not broadcasted. N/A

N/A
N/A

N/A

N/A

YES

YES

YES

When this flag is enabled, a Unit Attention

(UA) that is propagated
from another director does not flush the
queue for this device on this
director. Used for hosts that do not expect the
queue to be flushed on
D
a SCSI 0629 sense (only on a Hard Reset). N/A
When this flag is set, it alters the Inquiry
data returned by any device
on the port to report that Symmetrix supports
SCSI -3. If this flag is not
set, it reports that SCSI-2 is the supported
protocol. This flag also enable
the support for Auto Contingence Allegiance
SC3 which is required by IBM AIX.
N/A
The effect of the OS07 setting can be
summarized as follows:
1. Automated device discovery of newly
configured devices and/or device paths.
2. Automated creation of DSFs (device
special files) for newly configured devices.
3. Automated detection and adaptation of
device attribute changes including LUN
OS07 4. Capacity change event.
N/A
This flag enables the support for SPC-2 SCSI
standard that
are recommended for FC, FCoE, and iSCSI
SPC2 connectivity
N/A
This flag should be enabled for any AS/400
(IBM i)
AS4 hosts connecting to the port.
N/A
OVM Enabled for an OpenVMS Fibre Channel
S
connection.
N/A

YES

YES

YES

YES

YES
YES

EMC VMAX Notes On Setting FA Port Flags For VMware

June 4, 2014 David Ring VMAX, VMwareEAN, EMC, esx, FA Port, FLAGS, OS2007, SCSI-3,SPC2, VMAX, VMWARE

When configuring a VMAX for VMware ESX connectivity, I will always firstly
consult the EMC Simple Support Matrix EMC Symmetrix VMAX
Series Director Bit Settings found here for the latest listing of port
settings required. Below I will list the relevant VMware flags to be enabled on
the Director FA ports which have ESX Hosts zoned:
ACLX Allows Initiator Group/Access Logix function on the port.
Common Serial Number (C) When enabled, this flag causes the serial
number of a specific volume to be the same regardless of the port on which it
is visible and also allows multi-path configurations.
Unique WWN (UWN) When enabled, ensures unique World Wide
Names (WWNs) within the fibre environment by incorporating Symmetrix
serial numbers and port numbers for each WWN. Enabled by default for all
environment configuration changes and new environments.
Init Point to Point (PP) Specifies a point-to-point (direct or switched)
topology.
Enable Auto Negotiate (EAN) With EAN enabled this allows the FA
port to Auto Negotiate the port speed with the switch port it is connected to.
SCSI-3 Interface (SC3) SC3 provides SCSI-3 (Full range of SCSI-3
commands) support for the ESX hosts. Also note that SC3 is required for
Linux VMs with RDMs attached.
SCSI Support1 (OS2007) OS2007 enables automatic discovery of
newly created LUNs/Paths. When Enabled, this flag provides a stricter
compliance with SCSI standards for managing device identifiers, multi-port
targets, unit attention reports, and the absence of a device at LUN 0. If using
VPLEX 5.2 and higher, the OS2007 flag MUST be enabled.
SCSI Primary Commands (SPC2) This Flag is required in order for
the FA port to conform to industry standards resulting in an NAA identifier
being used for the LUNs presented to the ESX hosts. To change the status the
Port must be offline.
Note: These Flags may already be enabled by default for the associated
Director Ports and no additional configuration change is required. Consulting
the ESM for default settings will verify this.

FA Port Settings from Factory Bin File:

Green = Enabled
Yellow = Disabled

Setting director port attributes via Unisphere:

1. Select the Symmetrix system.
2. Select System > Dashboard.
3. From the Hardware panel, click Front End Directors to open the Front End
Directors list view.
4. Select the director, and click Set Port Attributes to open the Set Port
Attributes dialog box.

5. Optional:Select a port whose flag settings you want to copy.

VIA SYMCLI to enable flags on FA Port 1F:1, create a text file as per the
following example:
symconfigure -sid xxxx -file c:\file.txt
where file.txt has the following contents:
set port 1F:1 Common_Serial_Number=enable, SCSI_3=enable,
SPC2_Protocol_Version=enable, SCSI_Support1=enable;
View 1F:1 Port configuration include -v in order to view the Flag settings:
symcfg -sid xxxx list -fa 1F -p 1 -v
List all Ports:
symcfg -sid xxxx list -v -dir all

While all the port flags described above can be set at a Director Port level, in
some cases you may not want the enabled flags to affect different host types
connected to the Port such as Linux or Windows, in this case it may be a better
option to set the Flags at an Initiator Group level. By enabling the relevant
host type Flag settings at the Initiator Group level you are only affecting the
ESX Hosts included in the Initiator Group.
The Initiator Group VMware Specific flags are : C, SC3, OS2007, SPC2
In this way the flag settings for the initiator group override flag settings on the
port itself, thus allowing for the port to be shared by servers with differing flag
requirements.
To set the required flags at an IG level use the following symaccess cmd:
symaccess -sid xxxx -type initiator -name IGName set ig_flags on
C,OS2007,SC3,SPC2
There is also an option to set the Flags at the host initiator (WWN) level. For
example setting the flags on a host initiator connected to port 1F:1 then use the
following symmask commands:
symmask -sid xxxx -wwn xxxxxxxxxxxxx -dir 1F -p 1 set hba_flags on
C,OS2007,SC3,SPC2 -enable
symmask refresh
To display what director flags have been configured per host initiator:
symmaskdb -sid xxxx list database -v
Note: If there is a requirement to enable SPC-2 in a live VMware environment
then please reference EMC Support material; as doing so may result in a LUN
UUID change, depending on the version of VMware. One such solution to
overcome this problem was to create a new path and datastore then perform
an svmotion. Also changing the OS2007, SPC2, SC3 FA flags then the hosts
affected by these changes will require a reboot as the SCSI protocols in use will
have changed at the FA level and the HBA needs to reinitialize communication
with the FA.

How to check bits / flags

$ symmaskdb -sid <####> list database -v -wwn <HBA port pWWN> (e.g 10000000c951c57c)

$ symcfg list -sid 0426 -v -dir all | grep -E "[a-zA-Z1_ ]+\(.*\)" | grep abled | sort | uniq

Auto_Negotiate(EAN)

: Disabled

Auto_Negotiate(EAN)

: Enabled

Avoid_Reset_Broadcast(ARB) : Disabled
Common_Serial_Number(C)

: Enabled

Disable_Q_Reset_on_UA(D)

: Enabled

Environ_Set(E)

: Disabled

HP3000_Mode(B)

: Disabled

Init_Point_to_Point(PP)

: Enabled

Negotiate_Reset(N)

: Disabled

Non_Participating(NP)

: Disabled

OpenVMS(OVMS)
SCSI_Support1(OS2007)

: Disabled
: Disabled

SPC2_Protocol_Version(SPC2) : Disabled

Sequent(SEQ)

: Disabled

Siemens(S)

: Disabled

Soft_Reset(S)

: Disabled

Sunapee(SCL)

: Disabled

Unique_WWN(UWN)
VCM_State(VCM)

: Enabled
: Enabled

Volume_Set_Addressing(V)

: Disabled

Volume_Set_Addressing(V)

: Enabled

Device Masking (VCM) Config State

: Enable