Fibre Channel SAN Configure Guide

Fibre Channel SAN Configuration Guide
ESX Server 3.5, ESX Server 3i version 3.5 VirtualCenter 2.5
This document supports the version of each product listed and supports all subsequent versions until the document is replaced by a new edition. To check for more recent editions of this document, see http://www.vmware.com/support/pubs.
You can find the most up-to-date technical documentation on the VMware Web site at: http://www.vmware.com/support/ The VMware Web site also provides the latest product updates. If you have comments about this documentation, submit your feedback to: docfeedback@vmware.com
Copyright 20062010 VMware, Inc. All rights reserved. This product is protected by U.S. and international copyright and intellectual property laws. VMware products are covered by one or more patents listed at http://www.vmware.com/go/patents. VMware is a registered trademark or trademark of VMware, Inc. in the United States and/or other jurisdictions. All other marks and names mentioned herein may be trademarks of their respective companies.
VMware, Inc. 3401 Hillview Ave. Palo Alto, CA 94304 www.vmware.com

2 VMware, Inc.
Contents
AboutThisBook
1 OverviewofVMwareESXServer 13
IntroductiontoESXServer 14 SystemComponents 14 SoftwareandHardwareCompatibility 15 UnderstandingVirtualization 16 CPU,Memory,andNetworkVirtualization 16 VirtualSCSI 17 DiskConfigurationOptions 18 VirtualMachineFileSystem 19 RawDeviceMapping 19 VirtualSCSIHostBusAdapters 20 InteractingwithESXServerSystems 20 VMwareVirtualCenter 20 ESXServer3ServiceConsole 21 VirtualizationataGlance 22
2 UsingESXServerwithFibreChannelSAN 25
StorageAreaNetworkConcepts 26 OverviewofUsingESXServerwithSAN 28 BenefitsofUsingESXServerwithSAN 28 ESXServerandSANUseCases 29 FindingFurtherInformation 30 SpecificsofUsingSANArrayswithESXServer 31 SharingaVMFSAcrossESXServers 31 MetadataUpdates 32 LUNDisplayandRescan 32 HostType 33 LevelsofIndirection 33 DataAccess:VMFSorRDM 34 ThirdPartyManagementApplications 35
VMware, Inc.
ZoningandESXServer 35 AccessControl(LUNMasking)andESXServer 36 UnderstandingVMFSandSANStorageChoices 36 ChoosingLargerorSmallerLUNs 36 MakingLUNDecisions 37 PredictiveScheme 37 AdaptiveScheme 37 TipsforMakingLUNDecisions 38 UnderstandingDataAccess 39 PathManagementandFailover 41 ChoosingVirtualMachineLocations 43 DesigningforServerFailure 44 UsingVMwareHA 44 UsingClusterServices 44 ServerFailoverandStorageConsiderations 45 OptimizingResourceUse 46 UsingVMotiontoMigrateVirtualMachines 46 UsingVMwareDRStoMigrateVirtualMachines 47
3 RequirementsandInstallation 49
GeneralESXServerSANRequirements 50 RestrictionsforESXServerwithaSAN 50 SettingLUNAllocations 51 SettingFibreChannelHBA 51 Recommendations 52 ESXServerBootfromSANRequirements 53 InstallationandSetupSteps 54
4 SettingUpSANStorageDeviceswithESXServer 57
SetupOverview 58 Testing 58 SupportedDevices 59 GeneralSetupConsiderations 59 EMCCLARiiONStorageSystems 60 EMCCLARiiONAX100andRDM 60 AX100DisplayProblemswithInactiveConnections 61 PushingHostConfigurationChangestotheArray 61 EMCSymmetrixStorageSystems 61
VMware, Inc.
Contents
IBMTotalStorageDS4000StorageSystems 62 ConfiguringtheHardwareforSANFailoverwithDS4000StorageServers 62 VerifyingtheStorageProcessorPortConfiguration 63 DisablingAutoVolumeTransfer 64 ConfiguringStorageProcessorSenseData 65 IBMTotalStorageDS4000andPathThrashing 66 IBMTotalStorage8000 66 HPStorageWorksStorageSystems 66 HPStorageWorksMSA 66 SettingtheProfileNametoLinux 66 HubControllerIssues 68 HPStorageWorksEVA 68 HPStorageWorksXP 69 HitachiDataSystemsStorage 69 NetworkApplianceStorage 69
5 UsingBootfromSANwithESXServerSystems 71
BootfromSANOverview 72 HowBootfromaSANWorks 72 BenefitsofBootfromSAN 73 GettingReadyforBootfromSAN 73 BeforeYouBegin 74 LUNMaskinginBootfromSANMode 74 PreparingtheSAN 75 MinimizingtheNumberofInitiators 76 SettingUptheFCHBAforBootfromSAN 76 SettingUptheQLogicFCHBAforBootfromSAN 76 EnablingtheQLogicHBABIOS 76 EnablingtheSelectableBoot 77 SelectingtheBootLUN 77 SettingUpYourSystemtoBootfromCDROMFirst SettingUptheEmulexFCHBAforBootfromSAN 78
78
6 ManagingESXServerSystemsThatUseSANStorage 81
IssuesandSolutions 82 GuidelinesforAvoidingProblems 83 GettingInformation 83 ViewingHBAInformation 83 ViewingDatastoreInformation 84
VMware, Inc.
ResolvingDisplayIssues 85 UnderstandingLUNNamingintheDisplay 85 ResolvingIssueswithLUNsThatAreNotVisible 86 UsingRescan 87 RemovingDatastores 88 AdvancedLUNDisplayConfiguration 88 ChangingtheNumberofLUNsScannedUsingDisk.MaxLUN 88 MaskingLUNsUsingDisk.MaskLUNs 89 ChangingSparseLUNSupportUsingDisk.SupportSparseLUN 90 NPortIDVirtualization 90 HowNPIVBasedLUNAccessWorks 90 RequirementsforUsingNPIV 91 AssigningWWNstoVirtualMachines 92 Multipathing 95 ViewingtheCurrentMultipathingState 95 SettingaLUNMultipathingPolicy 98 DisablingandEnablingPaths 99 SettingthePreferredPathforFixedPathPolicy 100 PathManagementandManualLoadBalancing 100 Failover 102 SettingtheHBATimeoutforFailover 103 SettingDeviceDriverOptionsforSCSIControllers 103 SettingOperatingSystemTimeout 104 VMkernelConfiguration 104 SharingDiagnosticPartitions 104 AvoidingandResolvingProblems 105 OptimizingSANStoragePerformance 106 StorageArrayPerformance 106 ServerPerformance 107 ResolvingPerformanceIssues 108 MonitoringPerformance 108 ResolvingPathThrashing 108 UnderstandingPathThrashing 109 EqualizingDiskAccessBetweenVirtualMachines 110 RemovingVMFS2Drivers 111 RemovingNFSDrivers 111 ReducingSCSIReservations 111 SettingMaximumQueueDepthforHBAs 112 AdjustingQueueDepthforaQLogicHBA 112 AdjustingQueueDepthforanEmulexHBA 113
VMware, Inc.
Contents
SANStorageBackupConsiderations 114 SnapshotSoftware 115 UsingaThirdPartyBackupPackage 115 ChoosingYourBackupSolution 116 LayeredApplications 116 ArrayBased(ThirdParty)Solution 116 FileBased(VMFS)Solution 117 VMFSVolumeResignaturing 117 MountingOriginal,Snapshot,orReplicaVMFSVolumes 118 UnderstandingResignaturingOptions 118 State1EnableResignature=0,DisallowSnapshotLUN=1(default) 119 State2EnableResignature=1,(DisallowSnapshotLUNisnotrelevant) 119 State3EnableResignature=0,DisallowSnapshotLUN=0 119
A MultipathingChecklist 121 B Utilities 123

esxtopandresxtopUtilities 124 storageMonitorUtility 124 Options 124 Examples 125
Index 127
VMware, Inc.
VMware, Inc.
About This Book
Thismanual,theFibreChannelSANConfigurationGuide,explainshowtousea VMwareESXServersystemwithastorageareanetwork(SAN).Themanualdiscusses conceptualbackground,installationrequirements,andmanagementinformationinthe followingmaintopics:
UnderstandingESXServerIntroducesESXServersystemsforSAN administrators. UsingESXServerwithaSANDiscussesrequirements,noticeabledifferencesin SANsetupifESXServerisused,andhowtomanageandtroubleshootthetwo systemstogether. EnablingyourESXServersystemtobootfromaLUNonaSANDiscusses requirements,limitations,andmanagementofbootfromSAN.
NOTEThismanualsfocusisSANoverFibreChannel(FC).ItdoesnotdiscussiSCSI orNFSstoragedevices.ForinformationaboutiSCSIstorage,seetheiSCSISAN ConfigurationGuide.Forinformationaboutothertypesofstorage,seetheESXServer3i ConfigurationGuideandESXServer3ConfigurationGuide. TheFibreChannelSANConfigurationGuidecoversbothESXServer3.5andESXServer 3iversion3.5.Foreaseofdiscussion,thisbookusesthefollowingproductnaming conventions:

FortopicsspecifictoESXServer3.5,thisbookusesthetermESXServer3. FortopicsspecifictoESXServer3iversion3.5,thisbookusestheterm ESX Server 3i. Fortopicscommontobothproducts,thisbookusesthetermESXServer.
VMware, Inc.
Whentheidentificationofaspecificreleaseisimportanttoadiscussion,thisbook referstotheproductbyitsfull,versionedname. WhenadiscussionappliestoallversionsofESXServerforVMwareInfrastructure 3,thisbookusesthetermESXServer3.x.
Intended Audience
TheinformationpresentedinthismanualiswrittenforexperiencedWindowsorLinux systemadministratorsandwhoarefamiliarwithvirtualmachinetechnology datacenteroperations.
Document Feedback
VMwarewelcomesyoursuggestionsforimprovingourdocumentation.Ifyouhave comments,sendyourfeedbackto: docfeedback@vmware.com
VMware Infrastructure Documentation

TheVMwareInfrastructuredocumentationconsistsofthecombinedVMware VirtualCenterandESXServerdocumentationset.
Abbreviations Used in Figures

ThefiguresinthismanualusetheabbreviationslistedinTable 1. Table 1. Abbreviations
Abbreviation database datastore dsk# hostn SAN tmplt user# VC VM# Description VirtualCenterdatabase Storageforthemanagedhost Storagediskforthemanagedhost VirtualCentermanagedhosts Storageareanetworktypedatastoresharedbetweenmanagedhosts Template Userwithaccesspermissions VirtualCenter Virtualmachinesonamanagedhost
10
VMware, Inc.
About This Book
Technical Support and Education Resources

Thefollowingsectionsdescribethetechnicalsupportresourcesavailabletoyou.You canaccessthemostcurrentversionsofthismanualandotherbooksbygoingto: http://www.vmware.com/support/pubs
Online and Telephone Support

Useonlinesupporttosubmittechnicalsupportrequests,viewyourproductand contractinformation,andregisteryourproducts.Goto http://www.vmware.com/support. Customerswithappropriatesupportcontractsshouldusetelephonesupportforthe fastestresponseonpriority1issues.Goto http://www.vmware.com/support/phone_support.html.
Support Offerings
FindouthowVMwaresupportofferingscanhelpmeetyourbusinessneeds.Goto http://www.vmware.com/support/services.
VMware Education Services

VMwarecoursesofferextensivehandsonlabs,casestudyexamples,andcourse materialsdesignedtobeusedasonthejobreferencetools.Formoreinformationabout VMwareEducationServices,gotohttp://mylearn1.vmware.com/mgrreg/index.cfm.
VMware, Inc.
11
12
VMware, Inc.
Overview of VMware ESX Server
YoucanuseESXServerinconjunctionwithaFibreChannelstorageareanetwork (SAN),aspecializedhighspeednetworkthatusesFibreChannel(FC)protocolto transmitdatabetweenyourcomputersystemsandhighperformancestorage subsystems.UsingESXServertogetherwithaSANprovidesextrastoragefor consolidation,improvesreliability,andhelpswithdisasterrecovery. TouseESXServereffectivelywithaSAN,youmusthaveaworkingknowledgeofESX ServersystemsandSANconcepts.ThischapterpresentsanoverviewofESXServer concepts.ItismeantforSANadministratorsnotfamiliarwithESXServersystemsand consistsofthefollowingsections:

IntroductiontoESXServeronpage 14 UnderstandingVirtualizationonpage 16 InteractingwithESXServerSystemsonpage 20 VirtualizationataGlanceonpage 22
ForindepthinformationonVMwareESXServer,includingdocumentation,hardware compatibilitylists,whitepapers,andmore,gototheVMwareWebsiteat http://www.vmware.com.
VMware, Inc.
13
Introduction to ESX Server

TheESXServerarchitectureallowsadministratorstoallocatehardwareresourcesto multipleworkloadsinfullyisolatedenvironmentscalledvirtualmachines.
System Components
AnESXServersystemhasthefollowingkeycomponents:
VirtualizationlayerThislayerprovidestheidealizedhardwareenvironment andvirtualizationofunderlyingphysicalresourcestothevirtualmachines.It includesthevirtualmachinemonitor(VMM),whichisresponsiblefor virtualization,andVMkernel. Thevirtualizationlayerschedulesthevirtualmachineoperatingsystemsand,if youarerunninganESXServer3host,theserviceconsole.Thevirtualizationlayer manageshowtheoperatingsystemsaccessphysicalresources.TheVMkernel needsitsowndriverstoprovideaccesstothephysicaldevices.VMkerneldrivers aremodifiedLinuxdrivers,eventhoughtheVMkernelisnotaLinuxvariant.
HardwareinterfacecomponentsThevirtualmachinecommunicateswith hardwaresuchasCPUordiskbyusinghardwareinterfacecomponents.These componentsincludedevicedrivers,whichenablehardwarespecificservice deliverywhilehidinghardwaredifferencesfromotherpartsofthesystem. UserinterfaceAdministratorscanviewandmanageESXServerhostsand virtualmachinesinseveralways:
AVMwareInfrastructureClient(VIClient)canconnectdirectlytotheESX Serverhost.Thisisappropriateifyourenvironmenthasonlyonehost. AVIClientcanalsoconnecttoaVirtualCenterServerandinteractwithall ESXServerhoststhatVirtualCenterServermanages. TheVIWebAccessClientallowsyoutoperformmanymanagementtasksby usingabrowserbasedinterface. Onrareoccasions,whenyouneedtohavecommandlineaccess,youcanuse thefollowingoptions:
WithESXServer3,theserviceconsolecommandlineinterface.See AppendixAintheESXServer3ConfigurationGuide. WithESXServer3i,theremotecommandlineinterfaces(RCLIs).See AppendixAintheESXServer3iConfigurationGuide.
14
VMware, Inc.
Chapter 1 Overview of VMware ESX Server
Figure 11showshowthecomponentsinteract.TheESXServerhosthasfourvirtual machinesconfigured.Eachvirtualmachinerunsitsownguestoperatingsystemand applications.Administratorsmonitorthehostandthevirtualmachinesinthe followingways:

UsingaVIClienttoconnecttoanESXServerhostdirectly. UsingaVIClienttoconnecttoaVirtualCenterManagementServer.The VirtualCenterServercanmanageanumberofESXServerhosts.
Figure 1-1. Virtual Infrastructure Environment

VI Web Access VI Client VI Client VI Client
ESX Server host applications applications applications applications guest operating guest operating guest operating guest operating system system system system virtual machine virtual machine virtual machine virtual machine VMware virtualization layer host operating system
VirtualCenter Server
memory
CPUs
storage array
disk
network
Software and Hardware Compatibility

IntheVMwareESXServerarchitecture,theoperatingsystemofthevirtualmachine (theguestoperatingsystem)interactsonlywiththestandard,x86compatiblevirtual hardwarethatthevirtualizationlayerpresents.ThisarchitectureallowsVMware productstosupportanyx86compatibleoperatingsystem. Inpractice,VMwareproductssupportalargesubsetofx86compatibleoperating systemsthataretestedthroughouttheproductdevelopmentcycle.VMware documentstheinstallationandoperationoftheseguestoperatingsystemsandtrains itstechnicalpersonnelinsupportingthem.
VMware, Inc.
15
Mostapplicationsinteractonlywiththeguestoperatingsystem,notwiththe underlyinghardware.Asaresult,youcanrunapplicationsonthehardwareofyour choiceaslongasyouinstallavirtualmachinewiththeoperatingsystemthatthe applicationrequires.
Understanding Virtualization
TheVMwarevirtualizationlayeriscommonacrossVMwaredesktopproducts(suchas VMwareWorkstation)andserverproducts(suchasVMwareESXServer).Thislayer providesaconsistentplatformfordevelopment,testing,delivery,andsupportof applicationworkloadsandisorganizedasfollows:
Eachvirtualmachinerunsitsownoperatingsystem(theguestoperatingsystem) andapplications. Thevirtualizationlayerprovidesthevirtualdevicesthatmaptosharesofspecific physicaldevices.ThesedevicesincludevirtualizedCPU,memory,I/Obuses, networkinterfaces,storageadaptersanddevices,humaninterfacedevices, and BIOS.
CPU, Memory, and Network Virtualization

AVMwarevirtualmachineofferscompletehardwarevirtualization.Theguest operatingsystemandapplicationsrunningonavirtualmachinecanneverdetermine directlywhichphysicalresourcestheyareaccessing(suchaswhichphysicalCPUthey arerunningoninamultiprocessorsystem,orwhichphysicalmemoryismappedto theirpages).Thefollowingvirtualizationprocessesoccur:
CPUvirtualizationEachvirtualmachineappearstorunonitsownCPU(oraset ofCPUs),fullyisolatedfromothervirtualmachines.Registers,thetranslation lookasidebuffer,andothercontrolstructuresaremaintainedseparatelyforeach virtualmachine. MostinstructionsareexecuteddirectlyonthephysicalCPU,allowing resourceintensiveworkloadstorunatnearnativespeed.Thevirtualizationlayer safelyperformsprivilegedinstructions. SeetheResourceManagementGuide.
16
VMware, Inc.
MemoryvirtualizationAcontiguousmemoryspaceisvisibletoeachvirtual machine.However,theallocatedphysicalmemorymightnotbecontiguous. Instead,noncontiguousphysicalpagesareremappedandpresentedtoeach virtualmachine.Withunusuallymemoryintensiveloads,servermemory becomesovercommitted.Inthatcase,someofthephysicalmemoryofavirtual machinemightbemappedtosharedpagesortopagesthatareunmappedor swappedout. ESXServerperformsthisvirtualmemorymanagementwithouttheinformation thattheguestoperatingsystemhasandwithoutinterferingwiththeguest operatingsystemsmemorymanagementsubsystem. SeetheResourceManagementGuide.
NetworkvirtualizationThevirtualizationlayerguaranteesthateachvirtual machineisisolatedfromothervirtualmachines.Virtualmachinescantalktoeach otheronlythroughnetworkingmechanismssimilartothoseusedtoconnect separatephysicalmachines. Theisolationallowsadministratorstobuildinternalfirewallsorothernetwork isolationenvironments,allowingsomevirtualmachinestoconnecttotheoutside, whileothersareconnectedonlythroughvirtualnetworkstoothervirtual machines. SeetheESXServer3ConfigurationGuideorESXServer3iConfigurationGuide.
Virtual SCSI
InanESXServerenvironment,eachvirtualmachineincludesfromonetofourvirtual SCSIhostbusadapters(HBAs).ThesevirtualadaptersappearaseitherBuslogicorLSI LogicSCSIcontrollers.TheyaretheonlytypesofSCSIcontrollersthatavirtual machinecanaccess. EachvirtualdiskthatavirtualmachinecanaccessthroughoneofthevirtualSCSI adaptersresidesintheVMFSorisarawdisk.
VMware, Inc.
17
Figure 12givesanoverviewofstoragevirtualization.ItillustratesstorageusingVMFS andstorageusingrawdevicemapping(RDM). Figure 1-2. SAN Storage Virtualization

ESX Server virtual machine 1
SCSI controller (Buslogic or LSI Logic) virtual disk 1 virtual disk 2
VMware virtualization layer HBA
VMFS LUN1 .vmdk LUN2 RDM LUN5
Disk Configuration Options

YoucanconfigurevirtualmachineswithmultiplevirtualSCSIdrives.Foralistof supporteddrivers,seetheStorage/SANCompatibilityGuideat www.vmware.com/support/pubs/vi_pubs.html.Theguestoperatingsystemcanplace limitationsonthetotalnumberofSCSIdrives. AlthoughallSCSIdevicesarepresentedasSCSItargets,thefollowingphysical implementationalternativesexist:
Virtualmachine.vmdkfilestoredonaVMFSvolume.SeeVirtualMachineFile Systemonpage 19. DevicemappingtoaSANLUN(logicalunitnumber).SeeRawDeviceMapping onpage 19.
18
VMware, Inc.
LocalSCSIdevicepassedthroughdirectlytothevirtualmachine(forexample,a localtapedrive).
Fromthestandpointofthevirtualmachine,eachvirtualdiskappearsasifitwerea SCSIdriveconnectedtoaSCSIadapter.Whethertheactualphysicaldiskdeviceis beingaccessedthroughSCSI,iSCSI,RAID,NFS,orFibreChannelcontrollersis transparenttotheguestoperatingsystemandtoapplicationsrunningonthevirtual machine.
Virtual Machine File System

Inasimpleconfiguration,thevirtualmachinesdisksarestoredasfileswithinaVirtual MachineFileSystem(VMFS).WhenguestoperatingsystemsissueSCSIcommandsto theirvirtualdisks,thevirtualizationlayertranslatesthesecommandstoVMFSfile operations. ESXServersystemsuseVMFStostorevirtualmachinefiles.TominimizediskI/O overhead,VMFSisoptimizedtorunmultiplevirtualmachinesasoneworkload.VMFS alsoprovidesdistributedlockingforyourvirtualmachinefiles,sothatyourvirtual machinescanoperatesafelyinaSANenvironmentwheremultipleESXServerhosts shareasetofLUNs. VMFSisfirstconfiguredaspartoftheESXServerinstallation.Whenyoucreateanew VMFS3volume,itmustbe1200MBorlarger.SeetheInstallationGuide.Itcanthenbe customized,asdiscussedintheESXServer3ConfigurationGuideorESXServer3i ConfigurationGuide. AVMFSvolumecanbeextendedover32physicalstorageextentsofthesamestorage type.Thisabilityallowspoolingofstorageandflexibilityincreatingthestorage volumenecessaryforyourvirtualmachine.Youcanextendavolumewhilevirtual machinesarerunningonthevolumeaddingnewspacetoyourVMFSvolumesasyour virtualmachineneedsit.
Raw Device Mapping

Arawdevicemapping(RDM)isaspecialfileinaVMFSvolumethatactsasaproxyfor arawdevice.TheRDMprovidessomeoftheadvantagesofavirtualdiskintheVMFS filesystemwhilekeepingsomeadvantagesofdirectaccesstophysicaldevices. RDMmightberequiredifyouuseMicrosoftClusterService(MSCS)orifyourunSAN snapshotorotherlayeredapplicationsonthevirtualmachine.RDMsbetterenable systemstousethehardwarefeaturesinherenttoSANarrays.ForinformationonRDM, seeRawDeviceMappingintheESXServer3ConfigurationGuideorESXServer3i ConfigurationGuide,orSetupforMicrosoftClusterServiceforinformationaboutMSCS.
VMware, Inc.
19
Virtual SCSI Host Bus Adapters

VirtualSCSIhostbusadapters(HBAs)allowvirtualmachinesaccesstologicalSCSI devices,justasaphysicalHBAallowsaccesstophysicalstoragedevices.However,the virtualSCSIHBAdoesnotallowstorageadministrators(suchasSANadministrators) accesstothephysicalmachine.YoucanhidemanyvirtualHBAsbehindasingle(or multiple)FCHBAs.
Interacting with ESX Server Systems

AdministratorsinteractwithESXServersystemsinoneofthefollowingways:
Withaclient(VIClientorVIWebAccess).Clientscanbeconnecteddirectlytothe ESXServerhost,oryoucanmanagemultipleESXServerhostssimultaneouslyby usingtheVirtualCenterManagementServer. WithESXServer3,useaserviceconsole.InESXServer3.x,useoftheservice consoleisnotnecessaryandisdiscouragedbecauseyoucanperformmost administrativeoperationsusingaVIClientorVIWebAccess.Forscripted management,usetheVirtualInfrastructureSDK. Formoreinformationontheserviceconsole,seeESXServer3ServiceConsole onpage 21.
WithESXServer3i,usearemotecommandlineinterfaces(RCLIs).BecauseESX Server3idoesnotincludetheserviceconsole,configurationofanESXServer3i hostisusuallydonebyusingtheVIClient.However,ifyouwanttousethesame configurationsettingswithmultipleESXServer3ihosts,orifyouneed commandlineaccessforotherreasons,theRCLIsareavailable. SeetheESXServer3iConfigurationGuide.
VMware Virtual Center

YoucanaccessaVirtualCenterServerthroughaVIClientorVIWebAccess.
TheVirtualCenterServeractsasacentraladministratorforESXServerhosts connectedonanetwork.Theserverdirectsactionsuponthevirtualmachinesand VMwareESXServer. TheVIClientrunsonMicrosoftWindows.Inamultihostenvironment, administratorsusetheVIClienttomakerequeststotheVirtualCenterserver, whichinturnaffectsitsvirtualmachinesandhosts.Inasingleserver environment,theVIClientconnectsdirectlytoanESXServerhost.
20
VMware, Inc.
VIWebAccessallowsyoutoconnecttoaVirtualCenterServerbyusinganHTML browser.
Figure 13showstheConfigurationtabofaVIClientdisplaywithStorageselected. TheselectedESXServerhostconnectstoSANLUNsandtolocalharddisks.The differenceinthedisplayisvisibleonlybecauseofthenamesthatwerechosenduring setup. Figure 1-3. Storage Information Displayed in VI Client, Configuration Tab
ESX Server 3 Service Console

TheserviceconsoleistheESXServer3commandlinemanagementinterface.ESX Server3idoesnotprovideaserviceconsole.TheserviceconsolesupportsESX Server 3 systemmanagementfunctionsandinterfaces.TheseincludeHTTP,SNMP,andAPI interfaces,aswellasothersupportfunctionssuchasauthenticationand lowperformancedeviceaccess. BecauseVirtualCenterfunctionalityisenhancedtoallowalmostalladministrative operations,serviceconsolefunctionalityisnowlimited.Theserviceconsoleisused onlyunderspecialcircumstances. NOTEForscriptedmanagement,usetheVirtualInfrastructureSDK.
VMware, Inc.
21
TheserviceconsoleisimplementedusingamodifiedLinuxdistribution.However,the serviceconsoledoesnotcorresponddirectlytoaLinuxcommandprompt. ThefollowingESXServer3managementprocessesandservicesrunintheservice console:
Hostdaemon(hostd)Performsactionsintheserviceconsoleonbehalfofthe serviceconsoleandtheVIClient. Authenticationdaemon(vmauthd)AuthenticatesremoteusersoftheVIClient andremoteconsolesbyusingtheusernameandpassworddatabase.Youcanalso useanyotherauthenticationstorethatyoucanaccessusingtheserviceconsoles PluggableAuthenticationModule(PAM)capabilities.Havingmultiplepassword storagemechanismspermitstheuseofpasswordsfromaWindowsdomain controller,LDAPorRADIUSserver,orsimilarcentralauthenticationstorein conjunctionwithVMwareESXServerforremoteaccess. SNMPserver(netsnmpd)ImplementstheSNMPtrapsanddatastructuresthat anadministratorcanusetointegrateanESXServersystemintoanSNMPbased systemmanagementtool.
Inadditiontotheseservices,whicharesuppliedbyVMware,theserviceconsolecanbe usedtorunothersystemwideorhardwaredependentmanagementtools.Thesetools canincludehardwarespecifichealthmonitors(suchasIBMDirectororHPInsight Manager),fullsystembackupanddisasterrecoverysoftware,andclusteringandhigh availabilityproducts. NOTETheserviceconsoleisnotguaranteedtobeavailableforgeneralpurposeLinux hardwaremonitoring.ItisnotequivalenttoaLinuxshell.
Virtualization at a Glance
ESXServervirtualizestheresourcesofthephysicalsystemforthevirtualmachinesto use. Figure 14illustrateshowmultiplevirtualmachinessharephysicaldevices.Itshows twovirtualmachines,eachconfiguredwiththefollowing:

OneCPU Anallocationofmemoryandanetworkadapter(NIC) Twovirtualdisks
22
VMware, Inc.
Figure 1-4. Virtual Machines Sharing Physical Resources

virtual machine 1 network memory CPU adapter disk disk 1 2 virtual machine 2 disk disk network CPU memory 1 2 adapter
VMFS
raw disk
storage array
network adapter
CPU memory physical resources
ThevirtualmachineseachuseoneoftheCPUsontheserverandaccessnoncontiguous pagesofmemory,withpartofthememoryofonevirtualmachinecurrentlyswapped todisk(notshown).Thetwovirtualnetworkadaptersareconnectedtotwophysical networkadapters. Thedisksaremappedasfollows:
Disk1ofvirtualmachine1ismappeddirectlytoarawdisk.Thisconfigurationcan beadvantageousundercertaincircumstances. Disk2ofvirtualmachine1andbothdisksofvirtualmachine2resideontheVMFS, whichislocatedonaSANstoragearray.VMFSmakessurethatappropriate lockingandsecurityisinplaceatalltimes.
VMware, Inc.
23
24
VMware, Inc.
Using ESX Server with Fibre Channel SAN
WhenyousetupESXServerhoststouseFCSANarraystorage,specialconsiderations arenecessary.Thischapterprovidesintroductoryinformationabouthowtouse ESX ServerwithaSANarrayanddiscussesthesetopics:

StorageAreaNetworkConceptsonpage 26 OverviewofUsingESXServerwithSANonpage 28 SpecificsofUsingSANArrayswithESXServeronpage 31 UnderstandingVMFSandSANStorageChoicesonpage 36 UnderstandingDataAccessonpage 39 PathManagementandFailoveronpage 41 ChoosingVirtualMachineLocationsonpage 43 DesigningforServerFailureonpage 44 OptimizingResourceUseonpage 46
VMware, Inc.
25
Storage Area Network Concepts

IfyouareanESXServeradministratorplanningtosetupESXServerhoststoworkwith SANs,youmusthaveaworkingknowledgeofSANconcepts.Youcanfindinformation aboutSANinprintandontheInternet.Twowebbasedresourcesare:

www.searchstorage.com www.snia.org
Becausethisindustrychangesconstantly,checktheseresourcesfrequently. IfyouarenewtoSANtechnology,readthissectiontofamiliarizeyourselfwiththe basicterminologySANConfigurationGuideuses.TolearnaboutbasicSANconcepts, seetheSANConceptualandDesignBasicswhitepaperat http://www.vmware.com/support/pubs. NOTESANadministratorscanskipthissectionandcontinuewiththerestofthis chapter. Astorageareanetwork(SAN)isaspecializedhighspeednetworkthatconnects computersystems,orhostservers,tohighperformancestoragesubsystems.TheSAN componentsincludehostbusadapters(HBAs)inthehostservers,switchesthathelp routestoragetraffic,cables,storageprocessors(SPs),andstoragediskarrays. ASANtopologywithatleastoneswitchpresentonthenetworkformsaSANfabric. Totransfertrafficfromhostserverstosharedstorage,theSANusesFibreChannel(FC) protocolthatpackagesSCSIcommandsintoFibreChannelframes.
26
VMware, Inc.
Chapter 2 Using ESX Server with Fibre Channel SAN
Inthecontextofthisdocument,aportistheconnectionfromadevicetotheSAN.Each nodeintheSAN,ahost,storagedevice,andfabriccomponent,hasoneormoreports thatconnectittotheSAN.Portscanbeidentifiedinanumberofways:
WWPN(WorldWidePortName)Agloballyuniqueidentifierforaportthat allowscertainapplicationstoaccesstheport.TheFCswitchesdiscovertheWWPN ofadeviceorhostandassignaportaddresstothedevice. ToviewtheWWPNbyusingaVIClient,clickthehostsConfigurationtaband chooseStorageAdapters.Youcanthenselectthestorageadapterthatyouwantto seetheWWPNfor.
Port_ID(orportaddress)IntheSAN,eachporthasauniqueportIDthatserves astheFCaddressfortheport.ThisenablesroutingofdatathroughtheSANtothat port.TheFCswitchesassigntheportIDwhenthedevicelogsintothefabric.The portIDisvalidonlywhilethedeviceisloggedon.
WhenNPortIDVirtualization(NPIV)isused,asingleFCHBAport(Nport)can registerwiththefabricbyusingseveralWWPNs.ThisallowsanNporttoclaim multiplefabricaddresses,eachofwhichappearsasauniqueentity.Inthecontextofa SANbeingusedbyESXServerhosts,thesemultiple,uniqueidentifiersallowthe assignmentofWWNstoindividualvirtualmachinesaspartoftheirconfiguration.See NPortIDVirtualizationonpage 90. Whentransferringdatabetweenthehostserverandstorage,theSANusesa multipathingtechnique.Multipathingallowsyoutohavemorethanonephysicalpath fromtheESXServerhosttoaLUNonastoragearray. IfapathoranycomponentalongthepathHBA,cable,switchport,orstorage processorfails,theserverselectsanotheroftheavailablepaths.Theprocessof detectingafailedpathandswitchingtoanotheriscalledpathfailover.
VMware, Inc.
27
Storagediskarrayscanbeofthefollowingtypes:
Anactive/activediskarray,whichallowsaccesstotheLUNssimultaneouslythrough allthestorageprocessorsthatareavailablewithoutsignificantperformance degradation.Allthepathsareactiveatalltimes(unlessapathfails). Anactive/passivediskarray,inwhichonestorageprocessor(SP)isactivelyservicing agivenLUN.TheotherSPactsasbackupfortheLUNandcanbeactivelyservicing otherLUNI/O.I/Ocanbesentonlytoanactiveprocessor.IftheprimarySPfails, oneofthesecondarystorageprocessorsbecomesactive,eitherautomaticallyor throughadministratorintervention.
Torestrictserveraccesstostoragearraysnotallocatedtothatserver,theSANuses zoning.Typically,zonesarecreatedforeachgroupofserversthataccessasharedgroup ofstoragedevicesandLUNs.ZonesdefinewhichHBAscanconnecttowhichSPs. Devicesoutsideazonearenotvisibletothedevicesinsidethezone. ZoningissimilartoLUNmasking,whichiscommonlyusedforpermission management.LUNmaskingisaprocessthatmakesaLUNavailabletosomehostsand unavailabletootherhosts.Usually,LUNmaskingisperformedattheSPorserverlevel.
Overview of Using ESX Server with SAN

SupportforFCHBAsallowsanESXServersystemtobeconnectedtoaSANarray.You canthenuseSANarrayLUNstostorevirtualmachineconfigurationinformationand applicationdata.UsingESXServerwithaSANimprovesflexibility,efficiency,and reliability.Italsosupportscentralizedmanagement,aswellasfailoverandload balancingtechnologies.
Benefits of Using ESX Server with SAN

UsingaSANwithESXServerallowsyoutoimproveyourenvironmentsfailure resilience:
YoucanstoredataredundantlyandconfiguremultipleFCfabrics,eliminatinga singlepointoffailure.Yourenterpriseisnotcrippledwhenonedatacenter becomesunavailable. ESXServersystemsprovidemultipathingbydefaultandautomaticallysupportit foreveryvirtualmachine.SeePathManagementandFailoveronpage 41. UsingaSANwithESXServersystemsextendsfailureresistancetotheserver. WhenyouuseSANstorage,allapplicationscaninstantlyberestartedafterhost failure.SeeDesigningforServerFailureonpage 44.
28
VMware, Inc.
UsingESXServerwithaSANmakeshighavailabilityandautomaticloadbalancing affordableformoreapplicationsthanifdedicatedhardwareisusedtoprovidestandby services:
Becausesharedcentralstorageisavailable,buildingvirtualmachineclustersthat useMSCSbecomespossible.SeeServerFailoverandStorageConsiderationson page 45. Ifvirtualmachinesareusedasstandbysystemsforexistingphysicalservers, sharedstorageisessentialandaSANisthebestsolution. UsetheVMwareVMotioncapabilitiestomigratevirtualmachinesseamlessly fromonehosttoanother. UseVMwareHighAvailability(HA)inconjunctionwithaSANforacoldstandby solutionthatguaranteesanimmediate,automaticresponse. UseVMwareDistributedResourceScheduler(DRS)tomigratevirtualmachines fromonehosttoanotherforloadbalancing.BecausestorageisonaSANarray, applicationscontinuerunningseamlessly. IfyouuseVMwareDRSclusters,putanESXServerhostintomaintenancemode tohavethesystemmigrateallrunningvirtualmachinestootherESXServerhosts. Youcanthenperformupgradesorothermaintenanceoperations.
ThetransportabilityandencapsulationofVMwarevirtualmachinescomplementsthe sharednatureofSANstorage.WhenvirtualmachinesarelocatedonSANbased storage,youcanshutdownavirtualmachineononeserverandpowerituponanother serverortosuspenditononeserverandresumeoperationonanotherserveronthe samenetworkinamatterofminutes.Thisabilityallowsyoutomigratecomputing resourceswhilemaintainingconsistentsharedaccess.
ESX Server and SAN Use Cases

UsingESXServersystemsinconjunctionwithSANiseffectiveforthefollowingtasks:
MaintenancewithzerodowntimeWhenperformingmaintenance,useVMware DRSorVMotiontomigratevirtualmachinestootherservers.Ifsharedstorageis ontheSAN,youcanperformmaintenancewithoutinterruptionstotheuser. LoadbalancingUseVMotionorVMwareDRStomigratevirtualmachinesto otherhostsforloadbalancing.IfsharedstorageisonaSAN,youcanperformload balancingwithoutinterruptiontotheuser.
VMware, Inc.
29
StorageconsolidationandsimplificationofstoragelayoutIfyouareworking withmultiplehosts,andeachhostisrunningmultiplevirtualmachines,thehosts storageisnolongersufficientandexternalstorageisneeded.ChoosingaSANfor externalstorageresultsinasimplersystemarchitecturewhilegivingyoutheother benefitslistedinthissection.StartbyreservingalargeLUNandthenallocate portionstovirtualmachinesasneeded.LUNreservationandcreationfromthe storagedeviceneedstohappenonlyonce. DisasterrecoveryHavingalldatastoredonaSANcangreatlyfacilitateremote storageofdatabackups.Inaddition,youcanrestartvirtualmachinesonremote ESXServerhostsforrecoveryifonesiteiscompromised.
Finding Further Information

Inadditiontothisdocument,anumberofotherresourcescanhelpyouconfigureyour ESXServersysteminconjunctionwithaSAN:
Useyourstoragearrayvendorsdocumentationformostsetupquestions.Your storagearrayvendormightalsoofferdocumentationonusingthestoragearrayin anESXServerenvironment. TheVMwareDocumentationWebsiteathttp://www.vmware.com/support/pubs/. TheiSCSISANConfigurationGuidediscussestheuseofESXServerwithiSCSI storageareanetworks. TheVMwareI/OCompatibilityGuideliststhecurrentlyapprovedHBAs,HBA drivers,anddriverversions. TheVMwareStorage/SANCompatibilityGuidelistscurrentlyapprovedstorage arrays. TheVMwareReleaseNotesgiveinformationaboutknownissuesandworkarounds. TheVMwareKnowledgeBaseshaveinformationoncommonissuesand workarounds.
30
VMware, Inc.
Specifics of Using SAN Arrays with ESX Server

UsingaSANinconjunctionwithanESXServerhostdiffersfromtraditionalSANusage inavarietyofways,discussedinthissection.
Sharing a VMFS Across ESX Servers

ESXServerVMFSisdesignedforconcurrentaccessfrommultiplephysicalmachines andenforcestheappropriateaccesscontrolsonvirtualmachinefiles.Forbackground informationonVMFS,seeVirtualMachineFileSystemonpage 19. VMFScan:
CoordinateaccesstovirtualdiskfilesESXServerusesfilelevellocks,whichthe VMFSdistributedlockmanagermanages. CoordinateaccesstoVMFSinternalfilesysteminformation(metadata)ESX ServerusesshortlivedSCSIreservationsaspartofitsdistributedlockingprotocol. SCSIreservationsarenotheldduringmetadataupdatestotheVMFSvolume.
BecausevirtualmachinesshareacommonVMFS,itmightbedifficulttocharacterize peakaccessperiodsoroptimizeperformance.Planvirtualmachinestorageaccessfor peakperiods,butdifferentapplicationsmighthavedifferentpeakaccessperiods.The morevirtualmachinesshareaVMFS,thegreateristhepotentialforperformance degradationbecauseofI/Ocontention. NOTEVMwarerecommendsthatyouloadbalancevirtualmachinesoverservers, CPU,andstorage.Runamixofvirtualmachinesoneachgivenserverandstorageso thatnotallexperiencehighdemandinthesameareaatthesametime.
VMware, Inc.
31
Figure 21showsseveralESXServersystemssharingthesameVMFSvolume. Figure 2-1. Accessing Virtual Disk Files

ESX Server A ESX Server B ESX Server C
VM1
VM2
VM3
VMFS volume disk1 disk2 disk3
virtual disk files
Metadata Updates
AVMFSholdsfiles,directories,symboliclinks,RDMs,andsoon,andcorresponding metadatafortheseobjects.Metadataisaccessedeachtimetheattributesofafileare accessedormodified.Theseoperationsinclude,butarenotlimitedtothefollowing:

Creating,growing,orlockingafile Changingafilesattributes Poweringavirtualmachineonoroff
LUN Display and Rescan

ASANisdynamic,andwhichLUNsareavailabletoacertainhostcanchangebased onanumberoffactors,includingthefollowing:

NewLUNscreatedontheSANstoragearrays ChangestoLUNmasking ChangesinSANconnectivityorotheraspectsoftheSAN
32
VMware, Inc.
TheVMkerneldiscoversLUNswhenitboots,andthoseLUNsarethenvisibleintheVI Client.IfchangesaremadetotheLUNs,youmustrescantoseethosechanges. CAUTIONAfteryoucreateanewVMFSdatastoreorextendanexistingVMFS datastore,youmustrescantheSANstoragefromallESXServerhoststhatcouldseethat particulardatastore.Ifthisisnotdone,theshareddatastoremightbecomeinvisibleto someofthosehosts.
Host Type
ALUNhasaslightlydifferentbehaviordependingonthetypeofhostthatisaccessing it.Usually,thehosttypeassignmentdealswithoperatingsystemspecificfeaturesor issues.ESXServerarraysaretypicallyconfiguredwithahosttypeofLinuxor,if available,ESXorVMwarehosttype. SeeChapter 6,ManagingESXServerSystemsThatUseSANStorage,onpage 81and theVMwareknowledgebases.
Levels of Indirection
IfyouareusedtoworkingwithtraditionalSANs,thelevelsofindirectioncaninitially beconfusing.
Youcannotdirectlyaccessthevirtualmachineoperatingsystemthatusesthe storage.Withtraditionaltools,youcanmonitoronlytheVMwareESXServer operatingsystem,butnotthevirtualmachineoperatingsystem.Youusethe VI Clienttomonitorvirtualmachines. Eachvirtualmachineis,bydefault,configuredwithonevirtualharddiskandone virtualSCSIcontrollerduringinstallation.YoucanmodifytheSCSIcontrollertype andSCSIbussharingcharacteristicsbyusingtheVIClienttoeditthevirtual machinesettings,asshowninFigure 22.Youcanalsoaddharddiskstoyour virtualmachine.SeetheBasicSystemAdministration.
VMware, Inc.
33
Figure 2-2. Setting the SCSI Controller Type
TheHBAvisibletotheSANadministrationtoolsispartoftheESXServersystem, notthevirtualmachine. YourESXServersystemperformsmultipathingforyou.Multipathingsoftware, suchasPowerPath,inthevirtualmachineisnotsupportedandnotrequired.
Data Access: VMFS or RDM

Typically,avirtualdiskisplacedonaVMFSdatastoreduringvirtualmachinecreation. WhenguestoperatingsystemsissueSCSIcommandstotheirvirtualdisks,the virtualizationlayertranslatesthesecommandstoVMFSfileoperations.SeeVirtual MachineFileSystemonpage 19. AnalternativetoVMFSisusingRDMs.RDMsarespecialfilesinaVMFSvolumethat actasaproxyforarawdevice.TheRDMgivessomeoftheadvantagesofavirtualdisk intheVMFS,whilekeepingsomeadvantagesofdirectaccesstoaphysicaldevice.See RawDeviceMappingonpage 19.
34
VMware, Inc.
Third-Party Management Applications

MostSANhardwareispackagedwithSANmanagementsoftware.Thissoftware typicallyrunsonthestoragearrayoronasingleserver,independentoftheserversthat usetheSANforstorage.Usethisthirdpartymanagementsoftwareforanumberof tasks:
StoragearraymanagementincludingLUNcreation,arraycachemanagement, LUNmapping,andLUNsecurity. Settingupreplication,checkpoints,snapshots,ormirroring.
WhenyoudecidetoruntheSANmanagementsoftwareonavirtualmachine,yougain thebenefitsofrunningavirtualmachineincludingfailoverusingVMotionand VMwareHA,andsoon.Becauseoftheadditionallevelofindirection,however,the managementsoftwaremightnotbeabletoseetheSAN.Thisproblemcanberesolved byusinganRDM.SeeLayeredApplicationsonpage 116. NOTEWhetheravirtualmachinecanrunmanagementsoftwaresuccessfullydepends onthestoragearray.
Zoning and ESX Server

ZoningprovidesaccesscontrolintheSANtopology.ZoningdefineswhichHBAscan connecttowhichSPs.WhenaSANisconfiguredbyusingzoning,thedevicesoutside azonearenotvisibletothedevicesinsidethezone. Zoninghasthefollowingeffects:

ReducesthenumberoftargetsandLUNspresentedtoanESXServersystem. Controlsandisolatespathsinafabric. CanpreventnonESXServersystemsfromseeingaparticularstoragesystem,and frompossiblydestroyingESXServerVMFSdata. Canbeusedtoseparatedifferentenvironments(forexample,atestfroma productionenvironment).
VMware, Inc.
35
Whenyouusezoning,keepinmindthefollowingitems:
ESXServerhoststhatusesharedstorageforvirtualmachinefailoverorload balancingmustbeinonezone. Ifyouhaveaverylargedeployment,youmightneedtocreateseparatezonesfor differentareasoffunctionality.Forexample,youcanseparateaccountingfrom humanresources. Itdoesnotworkwelltocreatemanysmallzonesof,forexample,twohostswith fourvirtualmachineseach.
NOTECheckwiththestoragearrayvendorforzoningbestpractices.
Access Control (LUN Masking) and ESX Server

AccesscontrolallowsyoutolimitthenumberofESXServerhosts(orotherhosts)that canseeaLUN.Accesscontrolcanbeusefulto:

ReducethenumberofLUNspresentedtoanESXServersystem. PreventnonESXServersystemsfromseeingESXServerLUNsandfrompossibly destroyingVMFSvolumes.
Understanding VMFS and SAN Storage Choices

ThissectiondiscussestheavailableVMFSandSANstoragechoicesandgivesadviceon howtomakethem.
Choosing Larger or Smaller LUNs

WhenyousetupstorageforyourESXServersystems,chooseoneoftheseapproaches:

ManyLUNswithoneVMFSvolumeoneachLUN ManyLUNswithasingleVMFSvolumespanningallLUNs
YoucanhaveonlyoneVMFSvolumeperLUN.Youcan,however,decidetouseone largeLUNormultiplesmallLUNs. Youmightwantfewer,largerLUNsforthefollowingreasons:
MoreflexibilitytocreatevirtualmachineswithoutgoingbacktotheSAN administratorformorespace. Moreflexibilityforresizingvirtualdisks,takingsnapshots,andsoon. FewerLUNstoidentifyandmanage.

VMware, Inc.
36
Youmightwantmore,smallerLUNsforthefollowingreasons:

DifferentapplicationsmightneeddifferentRAIDcharacteristics. Moreflexibility(themultipathingpolicyanddisksharesaresetperLUN). UseofMicrosoftClusterService,whichrequiresthateachclusterdiskresourceis onitsownLUN.
Making LUN Decisions

Whenthestoragecharacterizationforavirtualmachineisnotavailable,useoneofthe followingapproachestodecideonLUNsizeandnumberofLUNstouse:

Predictivescheme Adaptivescheme
Predictive Scheme
Inthepredictivescheme,you:

CreateseveralLUNswithdifferentstoragecharacteristics. BuildaVMFSvolumeoneachLUN(labeleachvolumeaccordingtoits characteristics). LocateeachapplicationintheappropriateRAIDforitsrequirements. Usedisksharestodistinguishhighpriorityfromlowpriorityvirtualmachines. DisksharesarerelevantonlywithinagivenESXServerhost.Thesharesassigned tovirtualmachinesononeESXServerhosthavenoeffectonvirtualmachineson otherESXServerhosts.
Adaptive Scheme
Intheadaptivescheme,you:

CreatealargeLUN(RAID1+0orRAID5),withwritecachingenabled. BuildaVMFSonthatLUN. PlaceseveraldisksontheVMFS. Runtheapplicationsanddeterminewhetherdiskperformanceisacceptable. Ifperformanceisacceptable,youcanplaceadditionalvirtualdisksontheVMFS. Ifperformanceisnotacceptable,createanew,largerLUN,possiblywitha differentRAIDlevel,andrepeattheprocess.Youcanusecoldmigrationsothat youdonotlosevirtualmachineswhenrecreatingtheLUN.
VMware, Inc.
37
Tips for Making LUN Decisions

WhenmakingyourLUNdecision,keepinmindthefollowing:
EachLUNshouldhavethecorrectRAIDlevelandstoragecharacteristicfor applicationsinvirtualmachinesthatuseit. OneLUNmustcontainonlyonesingleVMFSvolume. IfmultiplevirtualmachinesaccessthesameLUN,usedisksharestoprioritize virtualmachines.
To use disk shares to prioritize virtual machines 1 2 3 4 StartaVIClientandconnecttoaVirtualCenterServer. Selectthevirtualmachinefromtheinventory,rightclick,andchooseEditSettings. ClicktheResourcestabandclickDisk. RightclicktheSharescolumnforthedisktomodify,andselecttherequiredvalue fromthedropdownmenu.
Sharesisavaluethatrepresentstherelativemetricforcontrollingdiskbandwidth toallvirtualmachines.ThevaluesLow,Normal,High,andCustomarecompared tothesumofallsharesofallvirtualmachinesontheserverand,onanESXServer 3 host,theserviceconsole.Shareallocationsymbolicvaluescanbeusedtoconfigure theirconversionintonumericvalues.
38
VMware, Inc.
Understanding Data Access

Virtualmachinesaccessdatabyusingoneofthefollowingmethods:
VMFSInasimpleconfiguration,thevirtualmachinesdisksarestoredas.vmdk fileswithinanESXServerVMFSdatastore.Whenguestoperatingsystemsissue SCSIcommandstotheirvirtualdisks,thevirtualizationlayertranslatesthese commandstoVMFSfileoperations. Inadefaultsetup,thevirtualmachinealwaysgoesthroughVMFSwhenitaccesses afile,whetherthefileisonaSANorahostslocalharddrives.SeeVirtual MachineFileSystemonpage 19.
RDMAnRDMisamappingfileinsidetheVMFSthatactsasaproxyforaraw device.TheRDMgivestheguestoperatingsystemaccesstotherawdevice. RDMisrecommendedwhenavirtualmachinemustinteractdirectlywitha physicaldiskontheSAN.Thisisthecase,forexample,whenyouwanttoissue diskarraysnapshotcreationcommandsfromyourguestoperationsystemor, morerarely,ifyouhavealargeamountofdatathatyoudonotwanttomoveonto avirtualdisk.RDMisalsorequiredforMicrosoftClusterServicesetup.Seethe VMwaredocumentSetupforMicrosoftClusterService.
VMware, Inc.
39
Figure 23illustrateshowvirtualmachinesaccessdatabyusingVMFSorRDM. Figure 2-3. How Virtual Machines Access Data

ESX Server virtual machine 1
SCSI controller (Buslogic or LSI Logic) virtual disk 1 virtual disk 2
VMware virtualization layer HBA
VMFS LUN1 .vmdk LUN2 RDM LUN5
FormoreinformationaboutVMFSandRDMs,seetheESXServer3ConfigurationGuide orESXServer3iConfigurationGuide. WhenavirtualmachineinteractswithaSAN,thefollowingprocesstakesplace: 1 2 Whentheguestoperatingsysteminavirtualmachineneedstoreadorwriteto SCSIdisk,itissuesSCSIcommandstothevirtualdisk. Devicedriversinthevirtualmachinesoperatingsystemcommunicatewiththe virtualSCSIcontrollers.VMwareESXServersupportstwotypesofvirtualSCSI controllers:BusLogicandLSILogic. ThevirtualSCSIControllerforwardsthecommandtotheVMkernel. TheVMkernel:
3 4
LocatesthefileintheVMFSvolumethatcorrespondstotheguestvirtual machinedisk. Mapstherequestsfortheblocksonthevirtualdisktoblocksonthe appropriatephysicaldevice.

VMware, Inc.
40
SendsthemodifiedI/OrequestfromthedevicedriverintheVMkerneltothe physicalHBA(hostHBA).
ThehostHBA:
Convertstherequestfromitsbinarydataformtotheopticalformrequiredfor transmissiononthefiberopticcable. PackagestherequestaccordingtotherulesoftheFCprotocol. TransmitstherequesttotheSAN.
DependingonwhichporttheHBAusestoconnecttothefabric,oneoftheSAN switchesreceivestherequestandroutesittothestoragedevicethatthehostwants toaccess. Fromthehostsperspective,thisstoragedeviceappearstobeaspecificdisk,butit mightbealogicaldevicethatcorrespondstoaphysicaldeviceontheSAN.The switchmustdeterminewhichphysicaldeviceismadeavailabletothehostforits targetedlogicaldevice.
Path Management and Failover

ESXServersupportsmultipathingtomaintainaconstantconnectionbetweenthe servermachineandthestoragedeviceincaseofthefailureofanHBA,switch,SP,or FCcable.Multipathingsupportdoesnotrequirespecificfailoverdrivers. Tosupportpathswitching,theservertypicallyhastwoormoreHBAsavailablefrom whichthestoragearraycanbereachedbyusingoneormoreswitches.Alternatively, thesetupcouldincludeoneHBAandtwostorageprocessorssothattheHBAcanuse adifferentpathtoreachthediskarray. InFigure 24,multiplepathsconnecteachserverwiththestoragedevice.Forexample, ifHBA1orthelinkbetweenHBA1andtheFCswitchfails,HBA2takesoverand providestheconnectionbetweentheserverandtheswitch.TheprocessofoneHBA takingoverforanotheriscalledHBAfailover.
VMware, Inc.
41
Figure 2-4. Multipathing and Failover

ESX Server ESX Server
HBA2
HBA1
HBA3
HBA4
switch
switch
SP1
SP2
storage array
Similarly,ifSP1failsorthelinksbetweenSP1andtheswitchesbreaks,SP2takesover andprovidestheconnectionbetweentheswitchandthestoragedevice.Thisprocessis calledSPfailover.VMwareESXServersupportsHBAandSPfailoverwithits multipathingcapability. Youcanchooseamultipathingpolicyforyoursystem,eitherFixedorMostRecently Used.IfthepolicyisFixed,youcanspecifyapreferredpath.EachLUN(disk)thatis visibletotheESXServerhostcanhaveitsownpathpolicy.Forinformationonhowto viewthecurrentmultipathingstateandhowtosetthemultipathingpolicy,see Multipathingonpage 95.
42
VMware, Inc.
VirtualmachineI/Omightbedelayedforatmostsixtysecondswhilefailovertakes place,particularlyonanactive/passivearray.ThisdelayisnecessarytoallowtheSAN fabrictostabilizeitsconfigurationaftertopologychangesorotherfabricevents.Inthe caseofanactive/passivearraywithpathpolicyFixed,paththrashingmightbea problem.SeeResolvingPathThrashingonpage 108. Avirtualmachinewillfailinanunpredictablewayifallpathstothestoragedevice whereyoustoredyourvirtualmachinedisksbecomeunavailable.
Choosing Virtual Machine Locations

Whenyouareworkingonoptimizingperformanceforyourvirtualmachines,storage locationisanimportantfactor.Thereisalwaysatradeoffbetweenexpensivestorage thatoffershighperformanceandhighavailabilityandstoragewithlowercostand lowerperformance.Storagecanbedividedintodifferenttiersdependingonanumber offactors:
HightierOffershighperformanceandhighavailability.Mightofferbuiltin snapshotstofacilitatebackupsandPointinTime(PiT)restorations.Supports replication,fullSPredundancy,andfibredrives.Useshighcostspindles. MidtierOffersmidrangeperformance,loweravailability,someSPredundancy, andSCSIdrives.Mightoffersnapshots.Usesmediumcostspindles. LowertierOfferslowperformance,littleinternalstorageredundancy.Useslow endSCSIdrivesorSATA(seriallowcostspindles).
Notallapplicationsneedtobeonthehighestperformance,mostavailablestorageat leastnotthroughouttheirentirelifecycle. Ifyouneedsomeofthefunctionalityofthehightier,suchassnapshots,butdonotwant topayforit,youmightbeabletoachievesomeofthehighperformancecharacteristics insoftware.Forexample,youcancreatesnapshotsinsoftware. Whenyoudecidewheretoplaceavirtualmachine,askyourselfthesequestions:

Howcriticalisthevirtualmachine? Whatareitsperformanceandavailabilityrequirements? Whatareitspointintime(PiT)restorationrequirements? Whatareitsbackuprequirements? Whatareitsreplicationrequirements?
VMware, Inc.
43
Avirtualmachinemightchangetiersthroughoutitslifecyclebecauseofchangesin criticalityorchangesintechnologythatpushhighertierfeaturestoalowertier. Criticalityisrelative,andmightchangeforavarietyofreasons,includingchangesin theorganization,operationalprocesses,regulatoryrequirements,disasterplanning, andsoon.
Designing for Server Failure

TheRAIDarchitectureofSANstorageinherentlyprotectsyoufromfailureatthe physicaldisklevel.Adualfabric,withduplicationofallfabriccomponents,protects theSANfrommostfabricfailures.Thefinalstepinmakingyourwholeenvironment failureresistantistoprotectagainstserverfailure.ESXServersystemsfailoveroptions arediscussedinthefollowingsections.
Using VMware HA
VMwareHAallowsyoutoorganizevirtualmachinesintofailovergroups.Whenahost fails,allitsvirtualmachinesareimmediatelystartedondifferenthosts.HArequires SANstorage. Whenavirtualmachineisrestoredonadifferenthost,itlosesitsmemorystatebutits diskstateisexactlyasitwaswhenthehostfailed(crashconsistentfailover).Shared storage,suchasaSAN,isrequiredforHA.SeetheResourceManagementGuide. NOTEYoumustbelicensedtouseVMwareHA.
Using Cluster Services

Serverclusteringisamethodoftyingtwoormoreserverstogetherbyusinga highspeednetworkconnectionsothatthegroupofserversfunctionsasasingle, logicalserver.Ifoneoftheserversfails,theotherserversintheclustercontinue operating,pickinguptheoperationsthatthefailedserverperforms. VMwaretestsMicrosoftClusterServiceinconjunctionwithESXServersystems,but otherclustersolutionsmightalsowork.Differentconfigurationoptionsareavailable forachievingfailoverwithclustering:
ClusterinaboxTwovirtualmachinesononehostactasfailoverserversforeach other.Whenonevirtualmachinefails,theothertakesover.Thisconfigurationdoes notprotectagainsthostfailures.Itismostcommonlydoneduringtestingofthe clusteredapplication. ClusteracrossboxesAvirtualmachineonanESXServerhosthasamatching virtualmachineonanotherESXServerhost.

VMware, Inc.
44
Physicaltovirtualclustering(N+1clustering)AvirtualmachineonanESX Serverhostactsasafailoverserverforaphysicalserver.Becausevirtualmachines runningonasinglehostcanactasfailoverserversfornumerousphysicalservers, thisclusteringmethodprovidesacosteffectiveN+1solution.
SeeSetupforMicrosoftClusterService. Figure 25showsdifferentconfigurationoptionsavailableforachievingfailoverwith clustering. Figure 2-5. Clustering Using a Clustering Service
virtual machine virtual machine virtual machine virtual machine virtual machine virtual machine virtual machine cluster in a box cluster across boxes physical to virtual clustering
Server Failover and Storage Considerations

Foreachtypeofserverfailover,youmustconsiderstorageissues:
Approachestoserverfailoverworkonlyifeachserverhasaccesstothesame storage.Becausemultipleserversrequirealotofdiskspace,andbecausefailover forthestoragearraycomplementsfailoverfortheserver,SANsareusually employedinconjunctionwithserverfailover. WhenyoudesignaSANtoworkinconjunctionwithserverfailover,allLUNsthat areusedbytheclusteredvirtualmachinesmustbeseenbyallESXServerhosts. ThisrequirementiscounterintuitiveforSANadministrators,butisappropriate whenusingvirtualmachines.
VMware, Inc.
45
AlthoughaLUNisaccessibletoahost,allvirtualmachinesonthathostdonot necessarilyhaveaccesstoalldataonthatLUN.Avirtualmachinecanaccessonly thevirtualdisksforwhichitwasconfigured.Incaseofaconfigurationerror, virtualdisksarelockedwhenthevirtualmachinebootssothatnocorruption occurs. NOTEAsarule,whenyouareusingbootfromaSAN,onlytheESXServersystemthat isbootingfromaLUNshouldseeeachbootLUN.Anexceptioniswhenyouaretrying torecoverfromafailurebypointingasecondESXServersystemtothesameLUN.In thiscase,theSANLUNinquestionisnotreallybootingfromSAN.NoESXServer systemisbootingfromitbecauseitiscorrupted.TheSANLUNisanonbootLUNthat ismadevisibletoanESXServersystem.
Optimizing Resource Use

VMwareInfrastructureallowsyoutooptimizeresourceallocationbymigratingvirtual machinesfromoverusedhoststounderusedhosts.Thefollowingoptionsexist:

MigratevirtualmachinesmanuallybyusingVMotion. MigratevirtualmachinesautomaticallybyusingVMwareDRS.
YoucanuseVMotionorDRSonlyifthevirtualdisksarelocatedonsharedstorage accessibletomultipleservers.Inmostcases,SANstorageisused.Foradditional informationonVMotion,seeBasicSystemAdministration.Foradditionalinformationon DRS,seetheResourceManagementGuide.
Using VMotion to Migrate Virtual Machines

VMotionallowsadministratorstomanuallymigratevirtualmachinestodifferent hosts.Administratorscanmigratearunningvirtualmachinetoadifferentphysical serverconnectedtothesameSANwithoutserviceinterruption.VMotionmakesit possibleto:
Performzerodowntimemaintenancebymovingvirtualmachinesaroundsothat theunderlyinghardwareandstoragecanbeservicedwithoutdisruptinguser sessions. Continuouslybalanceworkloadsacrossthedatacentertomosteffectivelyuse resourcesinresponsetochangingbusinessdemands.
46
VMware, Inc.
Figure 26illustrateshowyoucanuseVMotiontomigrateavirtualmachine. Figure 2-6. Migration with VMotion

ESX Server VMotion technology applications guest operating system virtual machine applicat s plicat p t tions guest ope st e s erating system system em virtual achine virtual ma u a e applications guest operating system virtual machine applications guest operating system virtual machine ESX Server
Using VMware DRS to Migrate Virtual Machines

VMwareDRShelpsimproveresourceallocationacrossallhostsandresourcepools. DRScollectsresourceusageinformationforallhostsandvirtualmachinesinaVMware clusterandgivesrecommendations(ormigratesvirtualmachines)inoneoftwo situations:
InitialplacementWhenyoufirstpoweronavirtualmachineinthecluster,DRS eitherplacesthevirtualmachineormakesarecommendation. LoadbalancingDRStriestoimproveresourceuseacrosstheclusterby performingautomaticmigrationsofvirtualmachines(VMotion)orbyproviding recommendationsforvirtualmachinemigrations.
SeetheResourceManagementGuide.
VMware, Inc.
47
48
VMware, Inc.
Requirements and Installation
ThischapterdiscusseshardwareandsystemrequirementsforusingESXServer systemswithSANstorage.Thechapterconsistsofthefollowingsections:

GeneralESXServerSANRequirementsonpage 50 ESXServerBootfromSANRequirementsonpage 53 InstallationandSetupStepsonpage 54
Thischapterlistsonlythemostbasicrequirements.Fordetailedinformationabout settingupyoursystem,readChapter 4,SettingUpSANStorageDeviceswithESX Server,onpage 57.
VMware, Inc.
49
General ESX Server SAN Requirements

InpreparationforconfiguringyourSANandsettingupyourESXServersystemtouse SANstorage,reviewthefollowingrequirementsandrecommendations:
Hardwareandfirmware.OnlyalimitednumberofSANstoragehardwareand firmwarecombinationsaresupportedinconjunctionwithESXServersystems.For anuptodatelist,seetheStorage/SANCompatibilityGuide. OneVMFSvolumeperLUN.ConfigureyoursystemtohaveonlyoneVMFS volumeperLUN.WithVMFS3,youdonothavetosetaccessibility. Unlessyouareusingdisklessservers,donotsetupthediagnosticpartitionona SANLUN. InthecaseofdisklessserversthatbootfromaSAN,ashareddiagnosticpartition isappropriate.SeeSharingDiagnosticPartitionsonpage 104.
VMwarerecommendsthatyouuseRDMsforaccesstoanyrawdiskfromanESX Server2.5orlatermachine.FormoreinformationonRDMs,seetheESXServer3 ConfigurationGuideorESXServer3iConfigurationGuide. Multipathing.Formultipathingtoworkproperly,eachLUNmustpresentthe sameLUNIDnumbertoallESXServerhosts. Queuesize.MakesuretheBusLogicorLSILogicdriverintheguestoperating systemspecifiesalargeenoughqueue.Youcansetthequeuedepthforthe physicalHBAduringsystemsetup.Forsupporteddrivers,seetheStorage/SAN CompatibilityGuide. SCSItimeout.OnvirtualmachinesrunningMicrosoftWindows,consider increasingthevalueoftheSCSITimeoutValueparametertoallowsWindowsto bettertoleratedelayedI/Oresultingfrompathfailover.SeeSettingOperating SystemTimeoutonpage 104.
Restrictions for ESX Server with a SAN

ThefollowingrestrictionsapplywhenyouuseESXServerwithaSAN:
ESXServerdoesnotsupportFCconnectedtapedevices.TheVMware ConsolidatedBackupproxycanmanagethesedevices.SeetheVirtualMachine BackupGuide. YoucannotusevirtualmachinemultipathingsoftwaretoperformI/Oload balancingtoasinglephysicalLUN.
50
VMware, Inc.
Chapter 3 Requirements and Installation
Youcannotusevirtualmachinelogicalvolumemanagersoftwaretomirrorvirtual disks.DynamicdisksonaMicrosoftWindowsvirtualmachineareanexception, butrequirespecialconfiguration.
Setting LUN Allocations

WhenyousetLUNallocations,notethefollowingpoints:
Storageprovisioning.ToensurethattheESXServersystemrecognizestheLUNs atstartuptime,provisionallLUNStotheappropriateHBAsbeforeyouconnect theSANtotheESXServersystem. VMwarerecommendsthatyouprovisionallLUNstoallESXServerHBAsatthe sametime.HBAfailoverworksonlyifallHBAsseethesameLUNs.
VMotionandVMwareDRS.WhenyouuseVirtualCenterandVMotionorDRS, makesurethattheLUNsforthevirtualmachinesareprovisionedtoallESXServer hosts.Thisprovidesthegreatestfreedominmovingvirtualmachines. Active/activecomparedtoactive/passivearrays.WhenyouuseVMotionorDRS withanactive/passiveSANstoragedevice,makesurethatallESXServersystems haveconsistentpathstoallstorageprocessors.Notdoingsocancausepath thrashingwhenaVMotionmigrationoccurs.SeeResolvingPathThrashingon page 108. Foractive/passivestoragearraysnotlistedintheStorage/SANCompatibilityGuide, VMwaredoesnotsupportstorageportfailover.Inthosecases,youmustconnect theservertotheactiveportonthestoragearray.Thisconfigurationensuresthat theLUNsarepresentedtotheESXServerhost.
Setting Fibre Channel HBA

DuringFCHBAsetup,considerthefollowingpoints:
HBAdefaultsettings.FCHBAsworkcorrectlywiththedefaultconfiguration settings.Followtheconfigurationguidelinesgivenbyyourstoragearrayvendor. NOTEForbestresults,usethesamemodelofHBAinoneserver.Ensurethatthe firmwareleveloneachHBAisthesameinoneserver.HavingEmulexandQLogic HBAsinthesameservertothesametargetisnotsupported.
StaticloadbalancingacrossHBAs.YoucanconfiguresomeESXServersystems toloadbalancetrafficacrossmultipleHBAstomultipleLUNswithcertain active/activearrays.
VMware, Inc.
51
Todothis,assignpreferredpathstoyourLUNssothatyourHBAsarebeingused evenly.Forexample,ifyouhavetwoLUNs(AandB)andtwoHBAs(XandY),you cansetHBAXtobethepreferredpathforLUNA,andHBAYasthepreferred pathforLUNB.ThismaximizesuseofyourHBAs.PathpolicymustbesettoFixed forthiscase.SeeTosetthemultipathingpolicyusingaVIClientonpage 98.
Settingthetimeoutforfailover.Setthetimeoutvaluefordetectingwhenapath failsintheHBAdriver.VMwarerecommendsthatyousetthetimeoutto30 secondstoensureoptimalperformance.Tosetthevalue,followtheinstructionsin SettingtheHBATimeoutforFailoveronpage 103. Dedicatedadapterfortapedrives.Forbestresults,useadedicatedSCSIadapter foranytapedrivesthatyouareconnectingtoanESXServersystem.FCconnected tapedrivesarenotsupported.UsetheConsolidatedBackupproxy,asdiscussedin theVirtualMachineBackupGuide. ForadditionalinformationonbootfromaSANHBAsetup,seeChapter 5,Using BootfromSANwithESXServerSystems,onpage 71.
Recommendations
ConsiderthefollowingwhensettingupyourenvironmentwithESXServerhostsand aSAN:
UseRDMforavirtualdiskofavirtualmachinetousesomeofthehardware snapshottingfunctionsofthediskarray,ortoaccessadiskfromavirtualmachine andaphysicalmachineinacoldstandbyhostconfigurationfordataLUNs. UseRDMfortheshareddisksinaMicrosoftClusterServicesetup.SeetheSetup forMicrosoftClusterService. AllocatealargeLUNformultiplevirtualmachinestouseandsetitupasaVMFS. Youcanthencreateordeletevirtualmachinesdynamicallywithouthavingto requestadditionaldiskspaceeachtimeyouaddavirtualmachine. TomoveavirtualmachinetoadifferenthostusingVMotion,theLUNsthathold thevirtualdisksofthevirtualmachinesmustbevisiblefromallthehosts.
Foradditionalrecommendationsandtroubleshootinginformation,seeChapter 6, ManagingESXServerSystemsThatUseSANStorage,onpage 81.
52
VMware, Inc.
ESX Server Boot from SAN Requirements

WhenyouhaveSANstorageconfiguredwithyourESXServersystem,youcanplace theESXServerbootimageononeoftheLUNsontheSAN.Thisconfigurationmust meetspecificcriteria,discussedinthissection.SeeUsingBootfromSANwithESX ServerSystemsonpage 71. ToenableyourESXServersystemtobootfromaSAN,performthefollowingtasks:
Checkthatyourenvironmentmeetsthegeneralrequirements.SeeGeneralESX ServerSANRequirementsonpage 50. CompletethetaskslistedinTable 31.
Table 3-1. Boot from SAN Requirements

Requirement ESXServer system requirements HBA requirements Description ESXServer3.xisrecommended.WhenyouusetheESXServer3.xsystem, RDMsaresupportedinconjunctionwithbootfromSAN.ForanESXServer 2.5.xsystem,RDMsarenotsupportedinconjunctionwithbootfromSAN. TheHBABIOSforyourHBAFCcardmustbeenabledandcorrectly configuredtoaccessthebootLUN.SeeSettingFibreChannelHBAon page 51. TheHBAshouldbepluggedintothelowestPCIbusandslotnumber.This allowsthedriverstodetecttheHBAquicklybecausethedriversscanthe HBAsinascendingPCIbusandslotnumbers,regardlessoftheassociated virtualmachineHBAnumber. Forprecisedriverandversioninformation,seetheESXServerI/O CompatibilityGuide. BootLUN considerations
Whenyoubootfromanactive/passivestoragearray,theSPwhose WWNisspecifiedintheBIOSconfigurationoftheHBAmustbeactive. IfthatSPispassive,theHBAcannotsupportthebootprocess. TofacilitateBIOSconfiguration,maskeachbootLUNsothatonlyits ownESXServersystemcanseeit.EachESXServersystemshouldsee itsownbootLUN,butnotthebootLUNofanyotherESXServer system.
VMware, Inc.
53
Table 3-1. Boot from SAN Requirements (Continued)

Requirement SAN considerations Description
SANconnectionsmustbethroughaswitchfabrictopology.Bootfrom SANdoesnotsupportdirectconnect(thatis,connectionwithout switches)orFCarbitratedloopconnections. Redundantandnonredundantconfigurationsaresupported.Inthe redundantcase,ESXServercollapsestheredundantpathssothatonly asinglepathtoaLUNispresentedtotheuser.
Hardware specific considerations
IfyouarerunninganIBMeServerBladeCenterandusebootfromSAN,you mustdisableIDEdrivesontheblades. Foradditionalhardwarespecificconsiderations,seetheVMware knowledgebasearticlesandChapter 4,SettingUpSANStorageDevices withESXServer,onpage 57.
Installation and Setup Steps

Table 32givesanoverviewoftheinstallationandsetupsteps,withpointerstorelevant information. Table 3-2. Installation and Setup Steps
Step 1 Description DesignyourSANifitsnotalready configured.MostexistingSANs requireonlyminormodificationto workwithESXServer. CheckthatallSANcomponents meetrequirements. SetuptheHBAsfortheESXServer hosts. Reference Chapter 2,UsingESXServerwithFibre ChannelSAN,onpage 25.
Chapter 3,GeneralESXServerSAN Requirements,onpage 50. Storage/SANCompatibilityGuide. Forspecialrequirementsthatapplyonlyto bootfromSAN,seeChapter 3,ESXServer BootfromSANRequirements,onpage 53. SeealsoChapter 5,UsingBootfromSAN withESXServerSystems,onpage 71.
Performanynecessarystoragearray modification.
Foranoverview,seeChapter 4,SettingUp SANStorageDeviceswithESXServer,on page 57. Mostvendorshavevendorspecific documentationforsettingupaSANtowork withVMwareESXServer.
InstallESXServeronthehostsyou haveconnectedtotheSANandfor whichyouvesetuptheHBAs.
InstallationGuide.
54
VMware, Inc.
Table 3-2. Installation and Setup Steps (Continued)

Step 6 7 Description Createvirtualmachines. (Optional)Setupyoursystemfor VMwareHAfailoverorforusing MicrosoftClusteringServices. Upgradeormodifyyour environmentasneeded. Reference BasicSystemAdministration. ResourceManagementGuide. SetupforMicrosoftClusterService. Chapter 6,ManagingESXServerSystems ThatUseSANStorage,onpage 81givesan introduction. SearchtheVMwareknowledgebasearticles formachinespecificinformationand latebreakingnews.
VMware, Inc.
55
56
VMware, Inc.
Setting Up SAN Storage Devices with ESX Server
Thischapterdiscussesmanyofthestoragedevicessupportedinconjunctionwith VMwareESXServer.Foreachdevice,itliststhemajorknownpotentialissues,points tovendorspecificinformation(ifavailable),andincludesinformationfromVMware knowledgebasearticles. NOTEInformationinthisdocumentisupdatedonlywitheachrelease.New informationmightalreadybeavailable.ConsultthemostrecentStorage/SAN CompatibilityGuide,checkwithyourstoragearrayvendor,andexploretheVMware knowledgebasearticles. Thischapterdiscussesthefollowingtopics:

SetupOverviewonpage 58 GeneralSetupConsiderationsonpage 59 EMCCLARiiONStorageSystemsonpage 60 EMCSymmetrixStorageSystemsonpage 61 IBMTotalStorageDS4000StorageSystemsonpage 62 IBMTotalStorage8000onpage 66 HPStorageWorksStorageSystemsonpage 66 HitachiDataSystemsStorageonpage 69 NetworkApplianceStorageonpage 69
VMware, Inc.
57
Setup Overview
VMwareESXServersupportsavarietyofSANstoragearraysindifferent configurations.Notallstoragedevicesarecertifiedforallfeaturesandcapabilitiesof ESXServer,andvendorsmighthavespecificpositionsofsupportwithregardtoESX Server.Forthelatestinformationregardingsupportedstoragearrays,seethe Storage/SANCompatibilityGuide.
Testing
VMwaretestsESXServerwithstoragearraysinthefollowingconfigurations:
BasicconnectivityTestswhetherESXServercanrecognizeandoperatewiththe storagearray.Thisconfigurationdoesnotallowformultipathingoranytypeof failover. HBAfailoverTheserverisequippedwithmultipleHBAsconnectingtooneor moreSANswitches.TheserverisrobusttoHBAandswitchfailureonly. StorageportfailoverTheserverisattachedtomultiplestorageportsandis robusttostorageportfailuresandswitchfailures. BootfromSANTheESXServerhostbootsfromaLUNconfiguredontheSAN ratherthanfromtheserveritself. DirectconnectTheserverconnectstothearraywithoutusingswitches,using onlyanFCcable.Forallothertests,afabricconnectionisused.FCArbitratedLoop (AL)isnotsupported. ClusteringThesystemistestedwithMicrosoftClusterServicerunninginthe virtualmachine.SeetheSetupforMicrosoftClusterServicedocument.
58
VMware, Inc.
Chapter 4 Setting Up SAN Storage Devices with ESX Server
Supported Devices
Table 41listsstoragedevicessupportedwithESXServer3.xandpointswheretofind moreinformationaboutusingtheminconjunctionwithESXServer. Table 4-1. Supported SAN Arrays
Manufacturer EMC Device CLARiiONStorageSystem. AlsoavailablefromFSC. AlsoavailablefromDell,Inc.asthe Dell/EMCFCRAIDArrayfamily ofproducts. SymmetrixStorageSystem. IBM IBMTotalStorageDS4000systems (formerlyFAStTStoragesystem). AlsoavailablefromLSIEugenio andStorageTek. IBMTotalStorageEnterprise StorageSystems(previouslyShark Storagesystems). HewlettPackard Hitachi HPStorageWorks(MSA,EVA, and XP). HitachiDataSystemsStorage. AlsoavailablefromSunandas HP XP. Network Appliance NetworkApplianceFCSAN StorageSolutions. IBMTotalStorage8000on page 66. HPStorageWorksStorage Systemsonpage 66. HitachiDataSystemsStorageon page 69. NetworkApplianceStorageon page 69. EMCSymmetrixStorage Systemsonpage 61. IBMTotalStorageDS4000Storage Systemsonpage 62. Reference EMCCLARiiONStorage Systemsonpage 60.
General Setup Considerations

Forallstoragearrays,makesurethatthefollowingrequirementsaremet:
LUNsmustbepresentedtoeachHBAofeachhostwiththesameLUNIDnumber. Ifdifferentnumbersareused,theESXServerhostsdonotrecognizedifferent pathstothesameLUN. BecauseinstructionsonhowtoconfigureidenticalSANLUNIDsarevendor specific,consultyourstoragearraydocumentationformoreinformation.
Unlessspecifiedforindividualstoragearraysdiscussedinthischapter,setthehost typeforLUNspresentedtoESXServertoLinux,Linux Cluster,or,ifavailable, tovmwareoresx.

59
VMware, Inc.
IfyouareusingVMotion,DRS,orHA,makesurethatbothsourceandtargethosts forvirtualmachinescanseethesameLUNswithidenticalLUNIDs. SANadministratorsmightfinditcounterintuitivetohavemultiplehostsseethe sameLUNsbecausetheymightbeconcernedaboutdatacorruption.However, VMFSpreventsmultiplevirtualmachinesfromwritingtothesamefileatthesame time,soprovisioningtheLUNstoallrequiredESXServersystemisappropriate.
EMC CLARiiON Storage Systems

EMCCLARiiONstoragesystemsworkwithESXServermachinesinSAN configurations.Basicconfigurationstepsinclude: 1 2 3 4 5 6 Installingandconfiguringthestoragedevice. Configuringzoningattheswitchlevel. CreatingRAIDgroups. CreatingandbindingLUNs. RegisteringtheserversconnectedtotheSAN. CreatingstoragegroupsthatcontaintheserversandLUNs.
UsetheEMCsoftwaretoperformconfiguration.SeetheEMCdocumentation. Thisarrayisanactive/passivediskarray,sothefollowingrelatedissuesapply. Toavoidthepossibilityofpaththrashing,thedefaultmultipathingpolicyisMost RecentlyUsed,notFixed.TheESXServersystemsetsthedefaultpolicywhenit identifiesthearray.SeeResolvingPathThrashingonpage 108. AutomaticvolumeresignaturingisnotsupportedforAX100storagedevices.See VMFSVolumeResignaturingonpage 117. TousebootfromSAN,makesurethattheactiveSPischosenforthebootLUNstarget intheHBABIOS.
EMC CLARiiON AX100 and RDM

OnEMCCLARiiONAX100systems,RDMsaresupportedonlyifyouusethe NavisphereManagementSuiteforSANadministration.Navilightisnotguaranteedto workproperly. TouseRDMssuccessfully,agivenLUNmustbepresentedwiththesameLUNIDto everyESXServerhostinthecluster.Bydefault,theAX100doesnotsupportthis configuration.
60
VMware, Inc.
AX100 Display Problems with Inactive Connections

WhenyouuseanAX100FCstoragedevicedirectlyconnectedtoanESXServersystem, youmustverifythatallconnectionsareoperationalandunregisteranyconnections thatarenolongerinuse.Ifyoudont,ESXServercannotdiscovernewLUNsorpaths. Considerthefollowingscenario: 1 AnESXServersystemisdirectlyconnectedtoanAX100storagedevice.TheESX ServerhastwoFCHBAs.OneoftheHBAswaspreviouslyregisteredwiththe storagearrayanditsLUNswereconfigured,buttheconnectionsarenowinactive. WhenyouconnectthesecondHBAontheESXServerhosttotheAX100and registerit,theESXServerhostcorrectlyshowsthearrayashavinganactive connection.However,noneoftheLUNsthatwerepreviouslyconfiguredtothe ESXServerhostarevisible,evenafterrepeatedrescans.
Toresolvethisissue,removetheinactiveHBA,unregistertheconnectiontotheinactive HBA,ormakeallinactiveconnectionsactive.ThiscausesonlyactiveHBAstobeinthe storagegroup.Afterthischange,rescantoaddtheconfiguredLUNs.
Pushing Host Configuration Changes to the Array

WhenyouuseanAX100storagearray,nohostagentperiodicallychecksthehost configurationandpusheschangestothearray.Theaxnaviserverutil cliutilityis usedtoupdatethechanges.Thisisamanualoperationandshouldbeperformedas needed.
EMC Symmetrix Storage Systems

ThefollowingsettingsarerequiredforESXServeroperationsontheSymmetrix networkedstoragesystem:

Commonserialnumber(C) Autonegotiation(EAN)enabled Fibrepathenabledonthisport(VCM) SCSI3(SC3)set(enabled) Uniqueworldwidename(UWN) SPC2(Decal)(SPC2)SPC2flagisrequired
UseEMCsoftwaretoconfigurethestoragearray.SeeyourEMCdocumentation.
VMware, Inc.
61
TheESXServerhostconsidersanyLUNsfromaSymmetrixstoragearraywitha capacityof50MBorlessasmanagementLUNs.TheseLUNsarealsoknownaspseudo orgatekeeperLUNs.TheseLUNsappearintheEMCSymmetrixManagement Interfaceandshouldnotbeusedtoholddata.
IBM TotalStorage DS4000 Storage Systems

IBMTotalStorageDS4000systemsusedtobecalledIBMFAStT.Anumberofstorage arrayvendors(includingLSIandStorageTek)makeSANstoragearraysthatare compatiblewiththeDS4000. SeetheIBMRedbook,ImplementingVMwareESXServerwithIBMTotalStorageFAStTat http://www.redbooks.ibm.com/redbooks/pdfs/sg246434.pdf.Thissectionsummarizes howtoconfigureyourIBMTotalStorageStorageSystemtouseSANandMicrosoft ClusteringService.SeeSetupforMicrosoftClusterService. InadditiontonormalconfigurationstepsforyourIBMTotalStoragestoragesystem, youneedtoperformspecifictasks. YoumustalsomakesurethatmultipathingpolicyissettoMostRecentlyUsed.See ViewingtheCurrentMultipathingStateonpage 95.
Configuring the Hardware for SAN Failover with DS4000 Storage Servers
TosetupahighlyavailableSANfailoverconfigurationwithDS4000storagemodels equippedwithtwostorageprocessors,youneedthefollowinghardwarecomponents:

TwoFCHBAs,suchasQLogicorEmulex,oneachESXServermachine. TwoFCswitchesconnectingtheHBAstotheSAN(forexample,FCswitch1and FCswitch2). TwoSPs(forexample,SP1andSP2). EachSPmusthaveatleasttwoportsconnectedtotheSAN.
UsethefollowingconnectionsettingsfortheESXServerhost,asshowninFigure 41:
ConnecteachHBAoneachESXServermachinetoaseparateswitch.Forexample, connectHBA1toFCswitch1andHBA2toFCswitch2. OnFCswitch1,connectSP1toalowerswitchportnumberthanSP2,toensurethat SP1islistedfirst.Forexample,connectSP1toFCswitch1port1andSP2toFC switch1port2.
62
VMware, Inc.
OnFCswitch2,connectSP1toalowerswitchportnumberthanSP2,toensurethat SP1islistedfirst.Forexample,connectSP1toport1onFCswitch2andSP2toport 2onFCswitch2. Figure 4-1. SAN Failover

ESX Server 1 HBA1 HBA2 ESX Server 2 HBA3 HBA4
FC switch 1
FC switch 2
SP1 storage
SP2
ThisconfigurationprovidestwopathsfromeachHBA,sothateachelementofthe connectioncanfailovertoaredundantpath.Theorderofthepathsinthis configurationprovidesHBAandswitchfailoverwithouttheneedtotriggerSPfailover. ThestorageprocessorthatthepreferredpathsareconnectedtomustowntheLUNs.In theprecedingexampleconfiguration,SP1ownsthem. NOTETheprecedingexampleassumesthattheswitchesarenotconnectedthroughan InterSwitchLink(ISL)inonefabric.
Verifying the Storage Processor Port Configuration

YoucanverifytheSPportconfigurationbycomparingtheVIClientinformationwith theinformationintheDS4000subsystemprofile. To verify storage processor port configuration 1 2 3 ConnecttotheESXServerhostbyusingtheVIClient. SelectthehostandchoosetheConfigurationtab. ClickStorageAdaptersintheHardwarepanel.
VMware, Inc.
63
SelecteachstorageadaptertoseeitsWWPN.
SelectStoragetoseetheavailabledatastores.
ComparetheWWPNinformationtotheinformationlistedintheDS4000storage subsystemprofile.
Disabling Auto Volume Transfer

Toavoidthepossibilityofpaththrashing,disableAutoVolumeTransfer(AVT)onthe SANstorageprocessors.IfAVTisenabled,thetwostorageprocessorscanalternately takeownershipoftheLUNincertainsituations,resultinginperformancedegradation. AVTisalsoknownasADT(AutoDiskTransfer). SeeResolvingPathThrashingonpage 108. TodisableAVT,intheDS4000StorageManager,foreachportdefinedineachhost groupthatcontainsHBAsforoneormoreESXServermachines,setthehosttypeto LNXCLor,inlaterversions,toVMware. NOTEYoumustreboottheESXServerhostafteryouchangetheAVTconfiguration.
64
VMware, Inc.
Configuring Storage Processor Sense Data

StorageprocessorscanbeconfiguredtoreturneithertheUnit AttentionorNot Readymessagewhenquiescent.ADS4000SPthatisrunningWindowsasaguest operatingsystemshouldreturnNot Readysensedatawhenitisquiescent.Returning Unit AttentionmightcausetheWindowsguesttofailduringafailover. To configure the storage processors to return Not Ready sense data 1 DeterminetheindexfortheLNXCL hosttypebyusingthefollowingcommandsin ashellwindow: PressEnteraftereachcommand.
SMcli.exe <ip-addr-for-SPA> show hosttopology; <Enter> SMcli.exe <ip-addr-for-SPB> show hosttopology; <Enter>
Thefollowingcommandsassumethat13istheindexcorrespondingtoLNXCLin theNVSRAMhosttypedefinitions.IfyourstorageprocessorshaveLNXCLata differentindex,substitutethatindexfor13inthefollowingcommands. 2 ExecutethesecommandsforSPAtohaveitreturnNot Readysensedata. PressEnteronlyafteryouenterallcommands.

SMcli.exe <ip-addr-for-SPA> set controller [a] HostNVSRAMBYTE [13,0x12]=0x01; set controller [a] HostNVSRAMBYTE [13,0x13]=0x00; reset Controller [a]; <Enter>
ExecutethesecommandsforSPBtohaveitreturnNot Readysensedata. PressEnteronlyafteryouenterallcommands.

SMcli.exe <ip-addr-for-SPB> set controller [b] HostNVSRAMBYTE [13,0x12]=0x01; set controller [b] HostNVSRAMBYTE [13,0x13]=0x00; reset Controller [b]; <Enter>
NOTEIfyouusetheDS4000StorageManagerGUI,pastetheconfigurationcommands forbothstorageprocessorsintoasinglescriptandconfigurebothstorageprocessorsat thesametime.IfyouuseSMcli.exe,makeindividualconnectionstoeachSP.
VMware, Inc.
65
IBM TotalStorage DS4000 and Path Thrashing

WhenpaththrashingisdetectedonaDS4000orcompatibleSANarray,thefollowing warningisloggedtothevmkernellog.
FAStT SAN is path thrashing with another system. Check AVT setting.
IBM TotalStorage 8000

IBMTotalStorage8000systemsuseanactive/activearraythatdoesnotneedspecial configurationinconjunctionwithVMwareESXServer. TouseRDMssuccessfully,agivenLUNneedstobepresentedwiththesameLUNID toeveryESXServerhostinthecluster. IntheTotalStorageConfigurationManagementtool,selectUsesameIDforLUNin sourceandtarget. AutomaticresignaturingisnotsupportedforIBMTotalStorage8000systems. NOTEIfyouareconfiguringtheESXServerhosttousebootfromSANfromaLUN onanIBMTotalStorage8000array,disabletheinternalfibreportforthecorresponding bladeuntilinstallationisfinished.
HP StorageWorks Storage Systems

ThissectionincludesconfigurationinformationforthedifferentHPStorageWorks storagesystems. Foradditionalinformation,seetheHPActiveAnswerssectiononVMwareESXServer attheHPwebsite.
HP StorageWorks MSA
Thissectionlistsissuesofinterestifyouareusingtheactive/passiveversionoftheHP StorageWorksMSA.
Setting the Profile Name to Linux

TouseHPStorageWorksMSA1000andMSA1500withESXServersystems,configure theFCconnectionsbetweentheSANarrayandtheESXServerhostwiththeProfile NamesettoLinux.
66
VMware, Inc.
To set the Profile Name for a connection 1 CreateastaticconnectionontheMSA1000byusingtheMSA1000commandline interface. Forinformationoninstallingandconfiguringthecommandlineinterface,seethe HPStorageWorksMSA1000documentation. NOTEYoucannotcreateconnectionsettingsbyusingtheHPArrayConfiguration utility. 2 3 4 ConnecttheMSA1000commandlineinterfacetotheMSA1000. VerifythattheFCnetworkbetweentheMSA1000andtheESXServerhostis working. Startthecommandlineinterfaceandenterthefollowingattheprompt:
SHOW CONNECTIONS
TheoutputdisplaysaconnectionspecificationforeachFCWWNNandWWPN attachedtotheMSA1000:
Connection Name: <unknown> Host WWNN = 20:02:00:a0:b8:0c:d5:56 Host WWPN = 20:03:00:a0:b8:0c:d5:57 Profile Name = Default Unit Offset 0 Controller 1 Port 1 Status = Online Controller 2 Port 1 Status = Online
5 6
MakesurethehostsWWNNandWWPNshowthecorrectconnectionforeachFC adapterontheESXServermachine. Createastaticconnectionasfollows: ADD CONNECTION ESX_CONN_1 WWNN=20:02:00:a0:b8:0c:d5:56 WWPN=20:03:00:a0:b8:0c:d5:57 PROFILE=LINUX
Verifytheconnectionasfollows:
SHOW CONNECTIONS
TheoutputdisplaysasingleconnectionwiththeWWNNandWWPNpair 20:02:00:a0:b8:0c:d5:56and20:03:00:a0:b8:0c:d5:57andtheProfile NamesettoLinux:

Connection Name: ESX_CONN_1 Host WWNN = 20:02:00:a0:b8:0c:d5:56 Host WWPN = 20:03:00:a0:b8:0c:d5:57 Profile Name = Linux
VMware, Inc.
67
Unit Offset = 0 Controller 1 Port 1 Status = Online Controller 2 Port 1 Status = Online
NOTEMakesureWWNN=20:02:00:a0:b8:0c:d5:56andWWPN= 20:03:00:a0:b8:0c:d5:57displayasingleconnection. ThereshouldbenoconnectionwiththeConnectionNameunknownforWWNN= 20:02:00:a0:b8:0c:d5:56andWWPN =20:03:00:a0:b8:0c:d5:57. 8 Addstaticconnections(withdifferentconnectionnamevalues)foreachWWNN andWWPNontheESXServerhost.
Hub Controller Issues

TheESXServersystemmightnotfunctioncorrectlywiththeMSAhubcontroller.Use the2/8internalswitchorthesingleportcontrollerinstead.
HP StorageWorks EVA
ThetwotypesofHPStorageWorksEVAsystemsare:EVA_GL,anactive/passive system;andEVA_XL,anactive/activesystem. SettheconnectiontypetoCustomwhenyoupresentaLUNtoanESXServerhost.The valueisoneofthefollowing:
(SEEUPDATE)ForHPEVAgl3000/5000(active/passive),usethehostmodetype 000000002200282E. ForHPEVAglfirmware4.001(active/activefirmwareforGLseries)andabove,use thehostmodetypeVMware. ForEVA4000/6000/8000active/activearrayswithfirmwarebelow5.031,usethe hostmodetype000000202200083E. ForEVA4000/6000/8000active/activearrayswithfirmware5.031andabove,use thehostmodetypeVMware.
Otherwise,EVAsystemsdonotrequirespecialconfigurationchangestoworkwithan ESXServersystem. SeetheVMwareInfrastructure3,HPStorageWorksBestPracticesat http://h71019.www7.hp.com/ActiveAnswers/downloads/VMware3_StorageWorks_Be stPractice.pdf.
68
VMware, Inc.
HP StorageWorks XP
ForHPStorageWorksXP,setthehostmodetoWindows(notLinux).Thissystemis availablefromHitachiDataSystems.
Hitachi Data Systems Storage

ThissectionintroducesthesetupforHitachiDataSystemsstorage.Thisstorage solutionisalsoavailablefromSunandasHPXPstorage.
LUNmaskingTomaskLUNsonanESXServerhost,usetheHDSStorage Navigatorsoftwareforbestresults. MicrocodeandconfigurationsCheckwithyourHDSrepresentativeforexact configurationsandmicrocodelevelsneededforinteroperabilitywithESXServer. Ifyourmicrocodeisnotsupported,interactionwithESXServerisusuallynot possible. ModesThemodesyousetdependonthemodelyouareusing,forexample:

9900and9900vusesNetwarehostmode. 9500vseriesusesHostmode1:standardandHostmode2:SUNCluster.
CheckwithyourHDSrepresentativeforhostmodesettingsforthemodelsnot listedhere.
Network Appliance Storage

WhenconfiguringaNetworkAppliancestoragedevice,firstsettheappropriateLUN typeandinitiatorgrouptypeforthestoragearray:

LUNtypeVMware(ifVMwaretypeisnotavailable,useLinux) InitiatorgrouptypeVMware(ifVMwaretypeisnotavailable,useLinux)
Youmustthenprovisionstorage. To provision storage from a Network Appliance storage device 1 UsingCLIortheFilerViewGUI,createanAggregateifrequired:

aggr create <vmware-aggr> <number of disks>
CreateaFlexibleVolume:
vol create <aggregate name> <volume size>
VMware, Inc.
69
CreateaQtreetostoreeachLUN:
qtree create <path>
CreateaLUN:
lun create -s <size> -t vmware <path>
Createaninitiatorgroup:
igroup create -f -t vmware <igroup name>
MaptheLUNtotheinitiatorgroupyoujustcreated:
lun map (<path>) <igroup name> <LUN ID>
ForadditionalinformationonhowtouseNetworkApplianceStoragewithVMware technology,seethefollowingNetworkAppliancedocuments:
NetworkAppliance&VMwareESXServer:InstantaneousBackup&Recovery withNetAppSnapshotTechnologyathttp://www.netapp.com/library/tr/3428.pdf. TechnicalCaseStudy:UsingaNetworkApplianceSANwithVMwaretoFacilitate StorageandServerConsolidationathttp://www.netapp.com/library/tr/3401.pdf.
70
VMware, Inc.
Using Boot from SAN with ESX Server Systems
ThischapterdiscussesthebenefitsofbootfromSANanddescribesthetasksyouneed toperformtohavetheESXServerbootimagestoredonaSANLUN. NOTESkipthischapterifyoudonotplantohaveyourESXServerhostbootfroma SAN. Thechapterdiscussesthefollowingtopics:

BootfromSANOverviewonpage 72 GettingReadyforBootfromSANonpage 73 SettingUptheFCHBAforBootfromSANonpage 76
VMware, Inc.
71
Boot from SAN Overview

BeforeyouconsiderhowtosetupyoursystemforbootfromSAN,decidewhetherit makessenseforyourenvironment. UsebootfromSAN:

Ifyoudonotwanttohandlemaintenanceoflocalstorage. Ifyouneedeasycloningofserviceconsoles(ESXServer3only). Indisklesshardwareconfigurations,suchasonsomebladesystems.
DonotusebootfromSAN:

IfyouareusingMicrosoftClusterService. IfI/OcontentionmightoccurbetweentheserviceconsoleandVMkernel(ESX Server3only).
NOTEWithESXServer2.5,youcouldnotusebootfromSANtogetherwithRDM.With ESXServer3.x,thisrestrictionisremoved.
How Boot from a SAN Works

WhenyousetupyoursystemtobootfromaSAN,thebootimageisnotstoredonthe ESXServersystemslocaldisk,butinsteadisstoredonaSANLUNasFigure 51shows. Figure 5-1. How Boot from a SAN Works
ESX Server service console VMkernel
HBA FC switch storage array
boot disk
72
VMware, Inc.
Chapter 5 Using Boot from SAN with ESX Server Systems
OnasystemsetuptobootfromaSAN:
TheHBABIOSmustdesignatetheFCcardasthebootcontroller.SeeSettingUp theFCHBAforBootfromSANonpage 76. TheFCcardmustbeconfiguredtoinitiateaprimitiveconnectiontothetargetboot LUN.
Benefits of Boot from SAN

InabootfromSANenvironment,theoperatingsystemisinstalledononeormore LUNsintheSANarray.Theserversareinformedaboutthebootimagelocation.When theserversarestarted,theybootfromtheLUNsontheSANarray. NOTEWhenyouusebootfromSANinconjunctionwithaVMwareESXServer system,eachservermusthaveitsownbootLUN. BootingfromaSANprovidesnumerousbenefits,including:
CheaperserversServerscanbemoredenseandruncoolerwithoutinternal storage. EasierserverreplacementYoucanreplaceserversandhavethenewserverpoint totheoldbootlocation. Lesswastedspace. EasierbackupprocessesThesystembootimagesintheSANcanbebackedup aspartoftheoverallSANbackupprocedures. ImprovedmanagementCreatingandmanagingtheoperatingsystemimageis easierandmoreefficient.
Getting Ready for Boot from SAN

InadditiontothegeneralESXServerwithSANconfigurationtasks,completethe followingtaskstoenableyourESXServerhosttobootfromSAN. To enable boot from SAN 1 2 EnsurethattheconfigurationsettingsmeetthebasicbootfromSANrequirements. Preparethehardwareelements. ThisincludesyourHBA,networkdevices,andstoragesystem.Refertothe productdocumentationforeachdevice.
VMware, Inc.
73
ConfigureLUNmaskingonyourSAN. ThisensuresthateachESXServerhosthasadedicatedLUNforthebootpartitions. ThebootLUNmustbededicatedtoasingleserver.
Choosethelocationforthediagnosticpartition. DiagnosticpartitionscanbeputonthesameLUNasthebootpartition.Core dumpsarestoredindiagnosticpartitions.
Therestofthissectionliststhetasksyouneedtocompletebeforeyoucansuccessfully bootyourESXServermachinefromSAN.
Before You Begin

Reviewthefollowing:
Therecommendationsorsamplesetupsforthetypeofsetupyouwant:

SingleorredundantpathstothebootLUN. FCswitchfabric. Anyspecificrecommendationsthatapplytothetypeofstoragearrayyou have.
Restrictionsandrequirements,including:

BootfromSANrestrictions. Thevendorsrecommendationforthestoragearraytobeusedforbooting fromaSAN. ThevendorsrecommendationfortheserverbootingfromaSAN.
FindtheWWNforthebootpathHBAbyusingoneofthefollowingmethods:

GointotheFCHBABIOSuponboot. FindtheWWNonthephysicalcard.ItissimilartoaMACaddress.
LUN Masking in Boot from SAN Mode

ProperLUNmaskingiscriticalinbootfromSANmode.

EachservercanseeonlyitsownbootLUN,notthebootLUNsofotherservers. Multipleserverscanshareadiagnosticpartition.YoucanuseLUNmaskingto achievethis.SeeSharingDiagnosticPartitionsonpage 104.
74
VMware, Inc.
Preparing the SAN

ThissectionliststhestepsforpreparingtheSANstoragearrayforbootfromSAN. Steps37arespecifictobootfromSAN,whilesteps1and2arenot. To prepare the SAN 1 ConnecttheFCandEthernetcables,referringtoanycablingguidethatappliesto yoursetup. ChecktheFCswitchwiring,ifthereisany. 2 Configurethestoragearray. a b c d e f
FromtheSANstoragearray,maketheESXServerhostvisibletotheSAN.This isoftenreferredtoascreatinganobject. FromtheSANstoragearray,setuptheESXServerhosttohavetheWWPNs ofthehostsFCadaptersasportnamesornodenames. CreateLUNs. AssignLUNs. RecordtheIPaddressesoftheFCswitchesandstoragearrays. RecordtheWWPNforeachSPandhostadapterinvolved. CAUTIONIfyouusescriptedinstallationtoinstallESXServerinbootfrom SANmode,youneedtotakespecialstepstoavoidunintendeddataloss.See VMwareknowledgebasearticle1540at http://www.vmware.com/support/kb/enduser/std_adp.php?p_faqid=1540.
ConfiguretheHBABIOSforbootfromSAN,asdiscussedinthefollowing sections:

SettingUptheQLogicFCHBAforBootfromSANonpage 76. SettingUptheEmulexFCHBAforBootfromSANonpage 78
BootyourESXServersystemfromtheESXServerinstallationCD. SeetheInstallationGuide.
TheQLogicBIOSusesasearchlistofpaths(wwpn:lun)tolocateabootimage.Ifoneof thewwpn:lunpathsisassociatedwithapassivepath(ascouldbethecasewith CLARiiONorIBMTotalStorageDS4000systems),theBIOSstayswiththepassivepath anddoesnotlocateanactivepath.IfyouarebootingyourESXServersystemfroma SANLUN,thebootfailswhilethehosttriestoaccessthepassivepath.
VMware, Inc.
75
Minimizing the Number of Initiators

Besurethezonecontainstheminimumnumberofhostandstorageportspossible.The EmulexandQLogicBIOScanbecomeunresponsiveifseveralotherinitiatorsareinthe samezoneandyoutrytoselectabootLUN. Forexample,iffifteeninitiatorsandfourSymmetrixportsareinonezone,youmight notbeabletoselectabootdevicefromeithertheEmulexorQLogicBIOSbecauseit becomesunresponsive.Ifyouzonethetwohostportstoseeonlythefourstorageports, youcanselectabootLUN.
Setting Up the FC HBA for Boot from SAN

ThissectiondiscusseshowtosetuptheHBAs.
Setting Up the QLogic FC HBA for Boot from SAN

ConfiguringtheQLogicHBABIOStobootESXServerfromaSANincludesthe followingtasks: NOTEIfyouareusinganIBMBladeCenter,disconnectallyourlocaldiskdrivesfrom theserver.
Enabling the QLogic HBA BIOS

WhenconfiguringtheQLogicHBABIOStobootESXServerfromSAN,startwith enablingtheQLogicHBABIOS. To enable the QLogic HBA BIOS 1 EntertheBIOSFast!UTILconfigurationutility: a b 2 Boottheserver. Whilebootingtheserver,pressCtrl-Q.
PerformtheappropriateactiondependingonthenumberofHBAs.
Ifyouhaveonlyonehostbusadapter(HBA),theFast!UTILOptionspage appears.SkiptoStep 3. IfyouhavemorethanoneHBA,selecttheHBAmanually: IntheSelectHostAdapterpage,usethearrowkeystopositionthecursoron theappropriateHBA. PressEnter.
a b
76
VMware, Inc.
3 4 5
IntheFast!UTILOptionspage,selectConfigurationSettingsandpressEnter. IntheConfigurationSettingspage,selectHostAdapterSettingsandpressEnter. SettheBIOStosearchforSCSIdevices: a b c IntheHostAdapterSettingspage,selectHostAdapterBIOS. PressEntertotogglethevaluetoEnabled. PressEsctoexit.
Enabling the Selectable Boot

Youneedtoenabletheselectableboot. To enable the selectable boot 1 2 3 ChooseSelectableBootSettingsandpressEnter. IntheSelectableBootSettingspage,chooseSelectableBoot. PressEntertotogglethevaluetoEnabled.
Selecting the Boot LUN

Ifyouareusinganactive/passivestoragearray,theselectedSPmustbeonthepreferred (active)pathtothebootLUN.IfyouarenotsurewhichSPisontheactivepath,use yourstoragearraymanagementsoftwaretofindout.ThetargetIDsarecreatedbythe BIOSandmightchangewitheachreboot. To select the boot LUN 1 2 3 Usethecursorkeystoselectthefirstentryinthelistofstorageprocessors. PressEntertoopentheSelectFibreChannelDevicepage. UsethecursorkeystoselectthechosenSPandpressEnter.
IftheSPhasonlyoneLUNattached,itisselectedasthebootLUN,andyou canskiptoStep 4. IftheSPhasmorethanoneLUNattached,theSelectLUNpageopens.Usethe arrowkeystopositiontotheselectedLUNandpressEnter. Ifanyremainingstorageprocessorsshowinthelist,positiontothoseentries andpressCtoclearthedata.
4 5
PressEsctwicetoexit. PressEntertosavethesetting.
VMware, Inc.
77
Setting Up Your System to Boot from CD-ROM First

BecausetheVMwareinstallationCDisintheCDROMdrive,setupyoursystemto bootfromCDROMfirst.Toachievethis,changethesystembootsequenceinyour systemBIOSsetup. Forexample,ontheIBMXSeries345server,dothefollowing: 1 2 3 4 Duringyoursystempowerup,enterthesystemBIOSConfiguration/SetupUtility. SelectStartupOptionsandpressEnter. SelectStartupSequenceOptionsandpressEnter. ChangetheFirstStartupDeviceto[CDROM].
YoucannowinstalltheESXServersystem,asdiscussedintheInstallationGuide.
Setting Up the Emulex FC HBA for Boot from SAN

ConfiguringtheEmulexHBABIOStobootESXServerfromSANincludesthe followingtasks:

ToenabletheBootBIOSprompt ToenabletheBIOS
To enable the BootBIOS prompt 1 FromtheESXServerserviceconsoleoraLinuxcommandprompt,runlputil. NOTEConsiderbootingtheESXServerhostfromaLinuxAdministrationCDthat loadstheEmulexdriver,thenrunlputilfromthere. 2 3 4 5 Select<3>FirmwareMaintenance. Selectanadapter. Select<6>BootBIOSMaintenance. Select<1>EnableBootBIOS.
To enable the BIOS 1 2 ReboottheESXServermachine. Press<ALT+E>attheEmulexprompt. a b Selectanadapter(withBIOSsupport). Select<2>ConfigureAdaptersParameters.
78
VMware, Inc.
c d e 3
Select<1>EnableorDisableBIOS. Select<1>toenableBIOS. Select<x>toexitand<N>toreturntothemainmenu.
FromtheEmulexmainmenu: a b c d e f g h Selectthesameadapter. Select<1>ConfigureBootDevices. SelectthelocationfortheBootEntry. Enterthetwodigitbootdevice. Enterthetwodigit(HEX)startingLUN(forexample,08). SelectthebootLUN. Select<1>WWPN.(BootthisdeviceusingWWPN,notDID). Select<x>toexitand<Y>toreboot.
4 5
BootintothesystemBIOSandmoveEmulexfirstinthebootcontrollersequence. RebootandinstallonaSANLUN.
VMware, Inc.
79
80
VMware, Inc.
Managing ESX Server Systems That Use SAN Storage
ThischaptercanhelpyouwithmanagingyourESXServersystem,withusingSAN storageeffectively,andwithtroubleshooting.Thischapterdiscussesthefollowing topics:

IssuesandSolutionsonpage 82 GuidelinesforAvoidingProblemsonpage 83 GettingInformationonpage 83 ResolvingDisplayIssuesonpage 85 AdvancedLUNDisplayConfigurationonpage 88 NPortIDVirtualizationonpage 90 Multipathingonpage 95 Failoveronpage 102 VMkernelConfigurationonpage 104 SharingDiagnosticPartitionsonpage 104 AvoidingandResolvingProblemsonpage 105 OptimizingSANStoragePerformanceonpage 106 ResolvingPerformanceIssuesonpage 108 SANStorageBackupConsiderationsonpage 114 LayeredApplicationsonpage 116 VMFSVolumeResignaturingonpage 117
VMware, Inc.
81
Issues and Solutions

Table 61liststheissuesthataremostfrequentlyencounteredandeitherexplainshow toresolvethemorpointstothelocationwheretheissueisdiscussed. Table 6-1. Issues and Solutions (SEE UPDATE)
Issue ALUNisnotvisibleintheVIClient. AsharedLUNandaVMFSfilesystem formattedonitisnotvisibletoallESXServer hoststhataccesstheLUN. Understandhowpathfailoverisperformedor changehowpathfailoverisperformed. Vieworchangethecurrentmultipathing policyorpreferredpath,ordisableorenablea path. IncreasetheWindowsdisktimeouttoavoid disruptionduringfailover. CustomizedriveroptionsfortheQLogicor EmulexHBA. TheservercannotaccessaLUN,oraccessis slow. Accessisslow. Solution SeeResolvingDisplayIssuesonpage 85. SeeResolvingIssueswithLUNsThatAre NotVisibleonpage 86. TheVIClientallowsyoutoperformthese actions.SeeMultipathingonpage 95. TheVIClientallowsyoutoperformthese actions.SeeMultipathingonpage 95. SeeSettingOperatingSystemTimeouton page 104. SeeSettingDeviceDriverOptionsforSCSI Controllersonpage 103. Paththrashingmightbetheproblem.See ResolvingPathThrashingonpage 108. IfyouhavealotofVMFSdatastore,andallof themareVMFS3,unloadtheVMFS2driver bytypingatacommandlineprompt: vmkload_mod -u vmfs2 Youwillseeasignificantincreaseinthe speedofmanagementoperationssuchas refreshingdatastoresandrescanningstorage adapters. Also,ifyoudonotuseNFSdatastores,you canunloadtheNFSdriverbytyping: vmkload_mod -u nfsclient YouaddedanewLUNoranewpathto storageandwanttoseeitintheVIClient. Youmustrescan.SeeUsingRescanon page 87.
82
VMware, Inc.
Chapter 6 Managing ESX Server Systems That Use SAN Storage
Guidelines for Avoiding Problems

Followtheseguidelinestoavoidpotentialproblems:
PlaceonlyoneVMFSvolumeoneachLUN.MultipleVMFSvolumesononeLUN isnotrecommended. Donotchangethepathpolicythesystemsetsforyou.Inparticular,workingwith anactive/passivearrayandsettingthepathpolicytoFixedcanleadtopath thrashing.
Getting Information
ThissectionexplainshowtofindinformationaboutHBAs,status,multipathing,andso on.Ifyouexperienceproblemswhenperformingthesetasks,seeResolvingDisplay Issuesonpage 85.
Viewing HBA Information

UsetheVIClienttodisplayallavailablestorageadaptersandtheirinformation. To see a list of HBA types 1 SelectthehostforwhichyouwanttoseetheHBAsandclicktheConfiguration tab. YoucanviewalistofallstoragedevicesfromtheSummarytab.However,you cannotseedetailsormanagethedevicefromthere.
VMware, Inc.
83
IntheHardwarepanel,chooseStorageAdapters. Thelistofstorageadaptersappears.Youcanselecteachadapterforadditional information.
Viewing Datastore Information

UsetheVIClienttodisplayallformatteddatastoresandreviewdetailsaboutaspecific datastore. To view all storage devices and details about them 1 2 Selectthehostforwhichyouwanttoseethestoragedevicesandclickthe Configurationtab. IntheHardwarepanel,chooseStorage. ThelistofdatastoresappearsintheStoragepanel. ThedisplayshowsthewholeVMFSfortheselectedhost.Onlystoragethatis formattedwithVMFSisincludedinthedisplay. 3 Toviewdetailsaboutanydatastore,selectit. TheDetailspaneldisplaysadditionalinformation.Thisincludesthelocationand capacity,numberofpaths,pathpolicy,andproperties.Italsoincludesextent information.
84
VMware, Inc.
AnextentisaVMFSformattedpartition(apieceofaLUN).Forexample,vmhba 0:0:14isaLUN,andvmhba0:0:14:1isapartition.OneVMFSvolumecanhave multipleextents. NOTETheabbreviationvmhbareferstothephysicalHBA(QLogicorEmulex)on theESXServersystem,nottotheSCSIcontrollerusedbythevirtualmachines. Figure 6-1. Datastore details
Extent
ClickPropertiestoviewandchangeproperties.
Resolving Display Issues

IfyouareusinganAX100storagearray,inactiveconnectionscancausedisplay problems.SeeAX100DisplayProblemswithInactiveConnectionsonpage 61.
Understanding LUN Naming in the Display

IntheVIClient,aLUNisdisplayedasasequenceofthreeorfournumbers,separated bycolons:
<SCSI HBA>:<SCSI target>:<SCSI LUN>:<disk partition>
Ifthelastnumberis0ornotdisplayed,thenamereferstotheentireLUN.
VMware, Inc. 85
ThefirstthreenumbersinanESXdevicenamemaychange,butstillrefertothesame physicaldevice.Forexample,vmhba1:2:3representsSCSILUN3,attachedtoSCSI target2,onSCSIHBA1.WhentheESXServersystemisrebooted,thedevicenamefor LUN3couldchangetovmhba1:1:3.Thenumbershavethefollowingmeaning:
Thefirstnumber,theSCSIHBA,changesifthereisanFCoriSCSInetworkoutage atthetimethesystemisbootedorrescannedandESXisrequiredtoaccessthe physicaldeviceoveradifferentSCSIHBA. Thesecondnumber,theSCSItarget,changesifthereisachangeinthemappings intheFCoriSCSItargetsvisibletotheESXServerhost. Thethirdnumber,theSCSILUN,neverchanges.
Resolving Issues with LUNs That Are Not Visible

YoucanusetheVIClienttoviewLUNs. Ifthedisplay(oroutput)differsfromwhatyouexpect,checkthefollowing:
CableconnectivityIfyoudonotseeaport,theproblemcouldbecable connectivityorzoning.Checkthecablesfirst. ZoningLimitsaccesstospecificstoragedevices,increasessecurity,anddecreases trafficoverthenetwork.Somestoragevendorsallowonlysingleinitiatorzones.In thatcase,anHBAcanbeinmultiplezonestoonlyonetarget.Othervendorsallow multipleinitiatorzones.Seeyourstoragevendorsdocumentationforzoning requirements.UsetheSANswitchsoftwaretoconfigureandmanagezoning. LUNmaskingIfanESXServerhostseesaparticularstoragedevicebutnotthe expectedLUNsonthatdevice,itmightbethatLUNmaskinghasnotbeensetup properly. TobootfromaSAN,ensurethateachESXServerhostseesonlyrequiredLUNs. DonotallowanyESXServerhosttoseeanybootLUNotherthanitsown.Usedisk arraysoftwaretomakesurethattheESXServerhostcanseeonlytheLUNsthatit issupposedtosee. EnsurethattheDisk.MaxLUNandDisk.MaskLUNssettingsallowyoutoview theLUNyouexpecttosee.SeeChangingtheNumberofLUNsScannedUsing Disk.MaxLUNonpage 88andMaskingLUNsUsingDisk.MaskLUNson page 89.
86
VMware, Inc.
StorageprocessorIfadiskarrayhasmorethanoneSP,makesurethattheSAN switchhasaconnectiontotheSPthatownstheLUNsyouwanttoaccess.Onsome diskarrays,onlyoneSPisactiveandtheotherSPispassiveuntilthereisafailure. IfyouareconnectedtothewrongSP(theonewiththepassivepath)youmightnot seetheexpectedLUNs,oryoumightseetheLUNsbutgeterrorswhentryingto accessthem.
Using Rescan
Performarescaneachtimeyoudooneofthefollowing:

ZoneanewdiskarrayontheSANtoanESXServerhost. CreatenewLUNsonaSANdiskarray. ChangetheLUNmaskingonanESXServerhostdiskarray.Afteryoumaskall pathstoaLUN,rescanalladapterswithpathstotheLUNinordertoupdatethe configuration. Reseatacable. MakeachangetoanESXServerhostinacluster. MakeachangetoadatastoreconfigurationonanESXServerhost,forexample, createanewdatastore,remove,upgrade,orresignatureadatastore,oraddan extent.
NOTEDonotrescanwhenapathisdown.Ifonepathfails,theothertakesoverand yoursystemcontinuestobefullyfunctional.If,however,yourescanatatimewhena pathisnotavailable,theESXServerhostremovesthepathfromitslistofpathstothe device.ThepathcannotbeusedbytheESXServerhostuntilthenexttimearescanis performedwhilethepathisactive. To perform a rescan 1 2 IntheVIClient,selectahostandclicktheConfigurationtab. IntheHardwarepanel,chooseStorageAdapters,andclickRescanabovethe StorageAdapterspanel. YoucanalsoselectanindividualadapterandclickRescantorescanjustthat adapter.
VMware, Inc.
87
Removing Datastores
(SEEUPDATE)UsingtheVIClient,youcanremoveadatastorefromtheESXServerhost. Beforeremovingthedatastore,migratevirtualmachinesthatresideonthisdatastore. To remove a datastore 1 2 3 4 IntheInventorypanel,selectthehost. ClicktheConfigurationtabandclickStoragetodisplayallstoragedevices. SelectthedatastoretoremoveandclickRemove. ClickRefreshtoupdatetheviewofavailablestorageoptions.
Advanced LUN Display Configuration

Thissectiondiscussesanumberofadvancedconfigurationoptions,includingchanging thenumberofLUNs,maskingLUNs,andchangingsparseLUNsupport.
Changing the Number of LUNs Scanned Using Disk.MaxLUN

Bydefault,theVMkernelscansforLUN0toLUN255foreverytarget(atotalof256 LUNs).YoucanchangetheDisk.MaxLunparametertochangethisnumber.This changemightimproveLUNdiscoveryspeed. NOTEYoucannotdiscoverLUNswithaLUNIDnumberthatishigherthan255. Reducingthevaluecanshortenbothrescantimeandboottime.Thetimetorescan LUNsdependsonseveralfactors,includingthetypeofstoragearrayandwhether sparseLUNsupportisenabled.SeeChangingSparseLUNSupportUsing Disk.SupportSparseLUNonpage 90. To change the value of Disk.MaxLUN 1 2 3 IntheVIClientinventorypanel,selectthehost. ClicktheConfigurationtab,andclickAdvancedSettings. Inthedialogboxthatappears,selectDisk.
88
VMware, Inc.
ScrolldowntoDisk.MaxLUN,changetheexistingvaluetothevalueofyour choice,andclickOK.
Masking LUNs Using Disk.MaskLUNs

TheDisk.MaskLUNsparameterallowsyoutomaskspecificLUNsonspecificHBAs. MaskedLUNsarenottouchedoraccessiblebytheVMkernel,evenduringinitial scanning. UsethisoptionwhenyouwanttopreventtheESXServersystemfromaccessingsome FCLUNs,butdonotwanttousetheFCswitchorFCdeviceLUNmasking mechanisms. To change the value of Disk.MaskLUNs (SEEUPDATE) 1 2 3 4 IntheVIClientinventorypanel,selectthehost. ClicktheConfigurationtab,andclickAdvancedSettings. Inthedialogboxthatappears,selectDisk. ScrolldowntoDisk.MaskLUNsandchangetheexistingvaluetothevalueofyour choiceusingthefollowingformat:
<adapter>:<target>:<comma separated LUN range list>
ClickOK.
VMware, Inc.
89
CAUTIONIfatarget,LUN,orvmhbanumberchangesbecauseofaserverorSAN reconfiguration,theincorrectLUNmaybemaskedorexposed.
Changing Sparse LUN Support Using Disk.SupportSparseLUN

WhenscanningforLUNsondevicesthatdonotsupporttheSCSI3standard,the VMkernelusesasequentialmethodprobingeachLUNwithinagivenLUNIDrange. Bydefault,theVMkernelisconfiguredtosupportsparseLUNs,aconfigurationwhere notallLUNsintherangearepresent. (SEEUPDATE)IfalltheLUNsarepresentintherange,youcandisablethe Disk.SupportSparesLUNparameter.Thischangedecreasesthetimeneededtoscan forLUNs.TheVMkernelstopsprobingforLUNsassoonasoneLUNintherangeis notpresent. YoudonotneedtochangetheDisk.SupportSparseLUNparameterforLUNsthat supportSCSI3standard.TheVMkernelusesamethodthatallowsittodiscoverall LUNsavailabletotheESXServerhostwithoutaneedtoscantheLUNssequentially. To disable sparse LUNs support 1 2 3 4 IntheVIClientinventorypanel,selectthehost. ClicktheConfigurationtab,andclickAdvancedSettings. IntheAdvancedSettingsdialogbox,selectDisk. ScrolldowntoDisk.SupportSparseLUN,changethevalueto0,andclickOK.
N-Port ID Virtualization
NPortIDVirtualization(NPIV)isanANSIT11standardthatdescribeshowasingle FibreChannelHBAportcanregisterwiththefabricusingseveralworldwideport names(WWPNs).ThisallowsafabricattachedNporttoclaimmultiplefabric addresses.EachaddressappearsasauniqueentityontheFibreChannelfabric.
How NPIV-Based LUN Access Works

SANobjects,suchasswitches,HBAs,storagedevices,orvirtualmachinescanbe assignedWorldWideName(WWN)identifiers.WWNsuniquelyidentifysuchobjects intheFibreChannelfabric.WhenvirtualmachineshaveWWNassignments,theyuse themforallRDMtraffic,sotheLUNspointedtobyanyoftheRDMsonthevirtual
90
VMware, Inc.
machinemustnotbemaskedagainstitsWWNs.Whenvirtualmachinesdonothave WWNassignments,theyaccessstorageLUNswiththeWWNsoftheirhostsphysical HBAs.ByusingNPIV,however,aSANadministratorcanmonitorandroutestorage accessonapervirtualmachinebasis.Thefollowingsectiondescribeshowthisworks. NPIVenablesasingleFCHBAporttoregisterseveraluniqueWWNswiththefabric, eachofwhichcanbeassignedtoanindividualvirtualmachine.Whenavirtual machinehasaWWNassignedtoit,thevirtualmachinesconfigurationfile(.vmx)is updatedtoincludeaWWNpair(consistingofaWorldWidePortName,WWPN,and aWorldWideNodeName,WWNN).Asthatvirtualmachineispoweredon,the VMkernelinstantiatesavirtualport(VPORT)onthephysicalHBAwhichisusedto accesstheLUN.TheVPORTisavirtualHBAthatappearstotheFCfabricasaphysical HBA,thatis,ithasitsownuniqueidentifier,theWWNpairthatwasassignedtothe virtualmachine.EachVPORTisspecifictothevirtualmachine,andtheVPORTis destroyedonthehostanditnolongerappearstotheFCfabricwhenthevirtual machineispoweredoff. IfNPIVisenabled,fourWWNpairs(WWPN&WWNN)arespecifiedforeachvirtual machineatcreationtime.WhenavirtualmachineusingNPIVispoweredon,ituses eachoftheseWWNpairsinsequencetotrytodiscoveranaccesspathtothestorage. ThenumberofVPORTsthatareinstantiatedequalsthenumberofphysicalHBAs presentonthehostuptothemaximumoffour.AVPORTiscreatedoneachphysical HBAthataphysicalpathisfoundon.Eachphysicalpathisusedtodeterminethe virtualpaththatwillbeusedtoaccesstheLUN.NotethatHBAsthatarenot NPIVawareareskippedinthisdiscoveryprocessbecauseVPORTscannotbe instantiatedonthem. NOTEIfauserhasfourphysicalHBAsaspathstothestorage,allphysicalpathsmust bezonedtothevirtualmachinebytheSANadministrator.Thisisrequiredtosupport multipathingeventhoughonlyonepathatatimewillbeactive.
Requirements for Using NPIV

BeforeyouattempttoimplementNPIVbyassigningWWNstoyourvirtualmachines, beawareofthefollowingrequirementsandlimitations:
NPIVcanonlybeusedforvirtualmachineswithRDMdisks.Virtualmachines withregularvirtualdisksusetheWWNsofthehostsphysicalHBAs.Formore informationonRDMs,seetheESXServer3ConfigurationGuideorESXServer3i ConfigurationGuide. ForthisimplementationofNPIV,thephysicalHBAsonanESXServerhost,using theirownWWNs,musthaveaccesstoallLUNsthataretobeaccessedbyvirtual machinesrunningonthathost.

91
VMware, Inc.
TheESXServerhostsphysicalHBAsmustsupportNPIV.Currently,thefollowing vendorsandtypesofHBAprovidethissupport:

QLogicany4GBHBA. Emulex4GBHBAsthathaveNPIVcompatiblefirmware.
OnlyfourWWNpairsaregeneratedpervirtualmachine. WhenavirtualmachineortemplatewithaWWNassignedtoitiscloned,the clonesdonotretaintheWWN. TheswitchesusedmustbeNPIVaware. WhenconfiguringanNPIVLUNforaccessatthestoragelevel,makesurethatthe NPIVLUNnumberandNPIVtargetIDmatchthephysicalLUNandTargetID. AlwaysusetheVIClienttomanipulatevirtualmachineswithWWNs.
Assigning WWNs to Virtual Machines

YoucanassignaWWNtoanewvirtualmachinewithanRDMdiskwhenyoucreate thisvirtualmachine,ortoanexistingvirtualmachineyoucantemporarilypoweroff. To create a virtual machine with an RDM 1 2 3 FromtheVIClient,clickInventoryinthenavigationbar,andexpandthe inventoryasneeded. Intheinventorylist,selectthemanagedhosttowhichyouwanttoaddanew virtualmachine. ChooseFile>New>VirtualMachine. TheNewVirtualMachinewizardappears. 4 5 6 7 SelectCustom,andclickNext. Typeavirtualmachinename,andclickNext. Selectafolderortherootofadatacenter,andclickNext. Iftheresourcepooloptionisavailable,expandthetreeuntilyoulocatethe resourcepoolinwhichyouwanttorunthevirtualmachine,highlightit,andclick Next. Selectadatastoreinwhichtostorethevirtualmachinefiles,andclickNext. UnderGuestoperatingsystem,selecttheoperatingsystemfamily(Microsoft Windows,Linux,NovellNetWare,Solaris,orOther).
8 9
92
VMware, Inc.
10 11 12 13 14
Choosetheversionfromthepulldownmenu,andclickNext. Selectthenumberofvirtualprocessorsinthevirtualmachinefromthepulldown list,andclickNext. Configurethevirtualmachinesmemorysizebyselectingthenumberof megabytes,andclickNext. Configurenetworkconnections,andclickNext. ChoosethetypeofSCSIadapteryouwanttousewiththevirtualmachine.
15 SelectRawDeviceMapping,andclickNext. 16 17 FromalistofSANdisksorLUNs,selectarawLUNyouwantyourvirtualmachine toaccessdirectly. SelectadatastorefortheRDMmappingfile. YoucanplacetheRDMfileonthesamedatastorewhereyourvirtualmachinefiles reside,orselectadifferentdatastore. NOTEIfyouwanttouseVMotionforavirtualmachinewithenabledNPIV,make surethattheRDMfileislocatedonthesamedatastorewherethevirtualmachine configurationfileresides.YoucannotperformStorageVMotion,orVMotion betweendatastores,whenNPIVisenabled. 18 Selectacompatibilitymode,eitherphysicalorvirtual.
Physicalcompatibilitymodeallowstheguestoperatingsystemtoaccessthe hardwaredirectly.PhysicalcompatibilityisusefulifyouareusingSANaware applicationsinthevirtualmachine.However,avirtualmachinewiththe physicalcompatibilityRDMcannotbecloned,madeintoatemplate,or migratedifthemigrationinvolvescopyingthedisk. VirtualcompatibilityallowstheRDMtobehaveasifitwereavirtualdisk,so youcanusesuchfeaturesassnapshotting,cloning,andsoon.
Dependingonyourchoice,subsequentscreensofferdifferentoptions. 19 20 OntheSpecifyAdvancedOptionspage,youcanchangethevirtualdevicenode andclickNext. AssignWWNstothevirtualmachinefollowingthestepsinToassignormodify WWNs.OntheReadytoCompleteNewVirtualMachinepage,selecttheEditthe virtualmachinesettingsbeforecompletioncheckboxandclickNext.
VMware, Inc.
93
AfteryoucreateavirtualmachinewithanRDM,youcanassignvirtualWWNstoit. YoucanalsomodifyWWNassignmentsforanexistingvirtualmachinewithanRDM. MakesuretopoweroffthevirtualmachineifyouwanttoedittheexistingWWNs. To assign or modify WWNs 1 2 EnsurethatyourSANadministratorhasprovisionedthestorageLUNACLto allowthevirtualmachinesESXServerhosttoaccessit. OpentheVirtualMachinePropertiesdialogbox:
Foranewvirtualmachine,aftercreatingthevirtualmachineasdescribedin TocreateavirtualmachinewithanRDM,ontheReadytoCompleteNew VirtualMachinepageselecttheEditthevirtualmachinesettingsbefore submittingthecreationtaskcheckbox,andclickContinue. Foranexistingvirtualmachine,selectthevirtualmachinefromtheinventory panel,andclicktheEditSettingslink.
3 4 5
SelecttheOptionstab. SelectFibreChannelNPIV. Inthedialogboxthatopens,selectoneofthefollowingoptions:
LeaveunchangedTheexistingWWNassignmentsareretained.The readonlyWWNAssignmentssectionofthisdialogboxdisplaysthenodeand portvaluesofanyexistingWWNassignments. GeneratenewWWNsNewWWNsaregeneratedandassignedtothe virtualmachine,overwritinganyexistingWWNs(thoseoftheHBAitselfare unaffected). RemoveWWNassignmentTheWWNsassignedtothevirtualmachineare removedanditusestheHBAWWNstoaccessthestorageLUN.Thisoption isnotavailableifyouarecreatinganewvirtualmachine. CAUTIONRemovingorchangingavirtualmachinesexistingWWN assignmentscausesittoloseconnectivitytothestorageLUNs
ClickOKtosaveyourchanges.
94
VMware, Inc.
Multipathing
Foranintroductiontomultipathingconcepts,seePathManagementandFailoveron page 41. NOTESANimplementationswithahighnumberofLUNsandpathstothoseLUNs cancauseESXServertorunoutofresourcesbeforeallofthepathsareenumerated.This preventsESXServerfromseeingallofthepathstothestorage.Toavoidthissituation, reducethepathcounttotheLUNs.
Viewing the Current Multipathing State

YoucanusetheVIClienttoviewthecurrentmultipathingstate. To view the current multipathing state 1 2 IntheVIClientinventorypanel,selectahostandclicktheConfigurationtab. IntheStoragepanel,selectoneofthedatastores. InformationaboutthatdatastoreappearsintheDetailspanel.
Toviewadditionalinformation,ortochangethemultipathingpolicy,select PropertiesabovetheDetailspanel.
VMware, Inc.
95
Ifthedatastorehasmultipleextents,intheExtentspanel,selecttheextentfor whichyouwanttovieworchangeinformation. TheExtentDevicepaneldisplaysinformationaboutthedevice,theVMFS datastoreonthisdevice,thepathselectionalgorithm,theavailablepaths,andthe activepath.
Thedisplayincludesinformationonthestatusofeachpathtothedevice.The followingpathinformationcanappear:
ActiveThepathisworkingandisthecurrentpathbeingusedfor transferringdata. DisabledThepathhasbeendisabledandnodatacanbetransferred. StandbyThepathisworkingbutisnotcurrentlyusedfordatatransfer. BrokenThesoftwarecannotconnecttothediskthroughthispath.
96
VMware, Inc.
IfyouareusingpathpolicyFixedandwanttoseewhichpathisthepreferredpath, clickManagePaths. Thepreferredpathismarkedwithanasterisk(*)inthefourthcolumn.
Table 62summarizeshowthebehaviorofanESXServersystemchanges,depending onthetypeofarrayandthefailoverpolicy. Table 6-2. Path Policy Effects

Policy/Controller MostRecentlyUsed Fixed Active/Active Administratoractionisrequired tofailbackafterpathfailure. VMkernelresumesusingthe preferredpathwhenconnectivity isrestored. Active/Passive Administratoractionisrequired tofailbackafterpathfailure. VMkernelattemptstoresume usingthepreferredpath.Thiscan causepaththrashingorfailure becauseanotherSPnowownsthe LUN.SeeResolvingPath Thrashingonpage 108.
VMware, Inc.
97
Setting a LUN Multipathing Policy

Bydefault,theESXServerhostusesonlyonepath,calledtheactivepath,to communicatewithaparticularstoragedeviceatanygiventime.Whenyouselectthe activepath,ESXserverfollowsthesemultipathingpolicies:
FixedTheESXServerhostalwaysusesthedesignatedpreferredpathtothedisk whenthatpathisavailable.Ifitcannotaccessthediskthroughthepreferredpath, ittriesthealternatepaths.Fixedisthedefaultpolicyforactive/activestorage devices. MostRecentlyUsedTheESXServerhostusesthemostrecentpathtothedisk untilthispathbecomesunavailable.Thatis,theESXServerhostdoesnot automaticallyrevertbacktothepreferredpath.MostRecentlyUsedisthedefault policyforactive/passivestoragedevicesandisrequiredforthosedevices. RoundRobinTheESXServerhostusesanautomaticpathselectionrotating throughallavailablepaths.Inadditiontopathfailover,roundrobinsupportsload balancingacrossthepaths. NOTERoundrobinloadbalancingisexperimentalandnotsupportedfor productionuse.SeetheRoundRobinLoadBalancingwhitepaper.
TheESXServerhostsetsthemultipathingpolicyaccordingtothemakeandmodelof thearrayitdetects.Ifthedetectedarrayisnotsupported,itistreatedasactive/active. Foralistofsupportedarrays,seetheStorage/SANCompatibilityGuide. NOTEManuallychangingMostRecentlyUsedtoFixedisnotrecommended.The systemsetsthispolicyforthosearraysthatrequireit. To set the multipathing policy using a VI Client 1 2 3 4 IntheVIClientinventorypanel,selectthehostandclicktheConfigurationtab. IntheHardwarepanel,selectStorage. Selectthedatastoreforwhichyouwanttochangethemultipathingpolicy,and clickPropertiesintheDetailspanel. IntheExtentpanel,selectthedeviceforwhichyouwanttomakethechange,and clickManagePathsintheExtentDevicepanelontheright. TheManagePathswizardopens.
98
VMware, Inc.
UnderPolicy,clickChange. TheSelectionPolicypageopens.
SelectthemultipathingpolicyandclickOK.
Disabling and Enabling Paths

Youcantemporarilydisablepathsformaintenanceorotherreasons.Youcandoso usingtheVIClient. To disable a path 1 2 3 4 IntheVIClientinventorypanel,selectthehostandclicktheConfigurationtab. IntheHardwarepanel,selectStorage. Selectthedeviceforwhichyouwanttodisableapath,andclickPropertiesinthe Detailspanel. IntheExtentpanel,selectthedeviceforwhichyouwanttomakethechange,and clickManagePathsintheExtentDevicepanelontheright. TheManagePathswizardopens. 5 6 Underpaths,selectthepathyouwanttodisable,andclickChange. SelecttheDisabledradiobuttontodisablethepath.
To enable a path Ifyouhavedisabledapath(forexample,formaintenance),youcanenableitby followingthestepsfordisablingapath,butclicktheEnabledradiobutton.
VMware, Inc.
99
Setting the Preferred Path for Fixed Path Policy

IfyouareusingpathpolicyFixed,specifythepreferredpaththattheserveruseswhen itisavailable. To set the preferred path 1 2 3 4 IntheVIClientInventorypane,selectthehostandclicktheConfigurationtab. IntheHardwarepanel,selectStorage. Selectthedeviceforwhichyouwanttodisableapath,andclickPropertiesinthe Detailspanel. IntheExtentpanel,selectthedeviceforwhichyouwanttomakethechange,and clickManagePathsintheExtentDevicepanelontheright. TheManagePathswizardopens. 5 6 UnderPaths,selectthepathyouwanttomakethepreferredpathandclick Change. InthePreferencepane,clickPreferred.
IfPreferredisnotavailable,makesurethatthepathpolicyisFixed. 7 ClickOKtwicetosaveyoursettingsandexitthedialogboxes.
Path Management and Manual Load Balancing

Balancingloadsamongavailablepathsimprovesperformance.Youcansetupyour systemtousedifferentpathstodifferentLUNsbychangingthepreferredpathforthe differentHBAs.Thisispossibleonlyforactive/activeSPs,andrequiresthatyouhave pathpolicysettoFixed.
100
VMware, Inc.
Ifapathfails,thesurvivingpathscarryallthetraffic.Pathfailovermighttakeaminute ormore,becausethefabricmightconvergewithanewtopologytotrytorestore service.ThisdelayisnecessarytoallowtheSANfabrictostabilizeitsconfiguration aftertopologychangesorotherfabricevents. Thefollowingexampledemonstrateshowmanualloadbalancingisperformed: Whenusinganactive/activearray,youcansetupyourenvironmentforloadbalancing. Assumethefollowingsetup,showninFigure 62:

Active/ActiveSPs AnESXServersystem FourFibreChannelHBAsineachserver Directorclasssoftware
Figure 6-2. Manual Load Balancing

ESX Server HBA1 HBA2 HBA3 HBA4
FC switch
SP1
SP2
storage array
Forloadbalancing,setthepreferredpathsasfollows.

LUN1:vmhba1:1:1 LUN2:vmhba2:1:2 LUN3:vmhba3:2:3 LUN4:vmhba4:2:4
SeeSettingthePreferredPathforFixedPathPolicyonpage 100forinformation. NOTELoadbalancingcanbeperformedwithasfewastwoHBAs,althoughthis exampleusesfour.
VMware, Inc.
101
Failover
PathfailoverreferstosituationswhentheactivepathtoaLUNischangedfromone pathtoanother,usuallybecauseofsomeSANcomponentfailurealongthecurrent path.AserverusuallyhasoneortwoHBAsandeachHBAseesoneortwostorage processorsonagivenSANarray.Youcandeterminetheactivepaththepath currentlyusedbytheserverbylookingattheLUNsproperties. Figure 6-3. Active and Standby Paths
Active and standby paths
WhenanFCcableispulled,I/Omightpausefor3060secondsuntiltheFCdriver determinesthatthelinkisdownanduntilfailoverhasoccurred.Asaresult,thevirtual machines(withtheirvirtualdisksinstalledonSANstorage)canappearunresponsive. Ifyouattempttodisplaythehost,itsstoragedevices,oritsadapter,theoperationmight appeartohang.Afterfailoveriscomplete,I/Oresumesnormally. Incaseofdisastrouseventsthatincludemultiplebreakages,allconnectionstoSAN storagedevicesmightbelost.Ifnoneoftheconnectionstothestoragedeviceis working,somevirtualmachinesmightencounterI/OerrorsontheirvirtualSCSIdisks.
102
VMware, Inc.
Setting the HBA Timeout for Failover

ThetimeoutvalueforI/OretryoperationsisusuallysetintheHBABIOSdriver.(You mightalsowanttochangeoperatingsystemtimeout,asdiscussedinSetting OperatingSystemTimeoutonpage 104.)VMwarerecommendsthatyousetthe timeoutvalueto30seconds. Toconfigurethisvalue:
ForQLogicHBAs,thetimeoutvalueis2*n + 5seconds,wherenisthevalueof thePortDownRetryCountparameteroftheBIOSoftheQLogiccard.Youcan changethepathfailuredetectiontimebychangingthevalueofthemodule parameterqlport_down_retry(whosedefaultvaluecomesfromtheBIOS setting).Therecommendedsettingforthisparameteris14. ForEmulexHBAs,youcanmodifythepathfailuredetectiontimebychangingthe valueofthemoduleparameterslpfc_linkdown_tmo(thedefaultis30)and lpfc_nodev_tmo(thedefaultis30).Thedriverusesthelargestofthetwo parameterstodeterminepathfailuredetectiontime.Therecommendedsettingfor eachisthedefault.
Tochangetheseparameters,youmustpassanextraoptiontothedriver,suchas qlport_down_retryorlpfc_linkdown_tmo.Thefollowingsectionexplainshowyou canpasstheseoptionstothedriver.
Setting Device Driver Options for SCSI Controllers

ThissectionsetsdevicedriveroptionsforQLogic,Emulex,orotherSCSIcarddrivers. To set device driver options for QLogic, Emulex, or other SCSI card drivers 1 Backupthefile/etc/vmware/esx.conf,andopenitforediting. ThefileincludesasectionforeachSCSIdevice,asinthefollowingexample:
/device/002:02.0/class = "0c0400" /device/002:02.0/devID = "2312" /device/002:02.0/irq = "19" /device/002:02.0/name = "QLogic Corp QLA231x/2340 (rev 02)" /device/002:02.0/options = "" /device/002:02.0/owner = "vmkernel" /device/002:02.0/subsysDevID = "027d" /device/002:02.0/subsysVendor = "1014" /device/002:02.0/vendor = "1077" /device/002:02.0/vmkname = "vmhba0"
2 3
VMware, Inc.
Findtheoptionslinerightunderthenamelineandmodifyitasappropriate. RepeatforeverySCSIadapterthatiscontrolledbythesamedriverifneeded.
103
Setting Operating System Timeout

YoumightwanttoincreasethestandarddisktimeoutvaluesothataWindowsguest operatingsystemisnotextensivelydisruptedduringfailover. ForWindows2000andWindowsServer2003guestoperatingsystems,youcanset operatingsystemtimeoutfusingtheregistry. To set operating system timeout for Windows servers 1 2 3 4 BackupyourWindowsregistry. SelectStart>Run,typeregedit.exeandclickOK. Intheleftpanelhierarchyview,doubleclickfirstHKEY_LOCAL_MACHINE, thenSystem,thenCurrentControlSet,thenServices,andthenDisk. SelecttheTimeOutValueandsetthedatavaluetox03c(hexadecimal)or60 (decimal). Afteryouvemadethischange,Windowswaitsatleast60secondsfordelayeddisk operationstocompletebeforeitgenerateserrors. 5 ClickOKtoexittheRegistryEditor.
VMkernel Configuration
WhenyouinstallyourESXServersystem,decidewheretoplacedifferentstorage elementssuchasthe/and/bootpartitionsoftheserviceconsole(ESXServer3only). ThedifferentcomponentsarediscussedinmoredetailintheInstallationGuide.
Sharing Diagnostic Partitions

IfyourESXServerhosthasalocaldisk,thatdiskismostappropriatelyusedforthe diagnosticpartition.Onereasonisthatifthereisanissuewithremotestoragethat causesacoredump,thecoredumpislostandresolvingtheissuebecomesmore difficult. However,fordisklessserversthatbootfromSAN,multipleESXServersystemscan shareonediagnosticpartitiononaSANLUN.IfmorethanoneESXServersystemis usingaLUNasadiagnosticpartition,thatLUNmustbezonedsothatalltheservers canaccessit. Eachserverneeds100MBofspace,sothesizeoftheLUNdetermineshowmanyservers canshareit.EachESXServersystemismappedtoadiagnosticslot.VMware recommendsatleast16slots(1600MB)ofdiskspaceifserversshareadiagnostic partition.
104 VMware, Inc.
Ifthereisonlyonediagnosticslotonthedevice,allESXServersystemssharingthat devicemaptothesameslot.Thiscaneasilycreateproblems.IftwoESXServersystems performacoredumpatthesametime,thecoredumpsareoverwrittenonthelastslot onthediagnosticpartition. Ifyouallocateenoughmemoryfor16slots,itisunlikelythatcoredumpsaremapped tothesamelocationonthediagnosticpartition,eveniftwoESXServersystems performacoredumpatthesametime.
Avoiding and Resolving Problems

Thissectiongivessometipsforavoidingandresolvingproblems:
Documenteverything.Includeinformationaboutzoning,accesscontrol,storage, switch,serverandFCHBAconfiguration,softwareandfirmwareversions,and storagecableplan. Planforfailure:
Takeyourtopologymapsandmakeseveralcopies.Foreachelement,consider whathappenstoyourSANiftheelementfails. Crossoffdifferentlinks,switches,HBAsandotherelementstoensureyou didntmissacriticalfailurepointinyourdesign.
DisconnecttheFibreChannelHBAsduringlocalinstallationwhenyouinstallan ESXServerhostonaproductionsystem. CAUTIONTheinstallerletsyoueraseanyaccessibledisks,includingSANLUNs inusebyotherservers.
EnsurethattheFibreChannelHBAsareinstalledinthecorrectslotsintheESX Serverhost,basedonslotandbusspeed.BalancePCIbusloadamongtheavailable bussesintheserver. Becomefamiliarwiththevariousmonitorpointsinyourstoragenetwork,atall visibilitypoints,includingESXServerperformancecharts,FCswitchstatistics, andstorageperformancestatistics.
VMware, Inc.
105
Optimizing SAN Storage Performance

ThetwomajorfactorsforoptimizingatypicalSANenvironmentarestoragearray performanceandserverperformance.Iftheenvironmentisproperlyconfigured,the SANfabriccomponents(particularlytheSANswitches)areonlyminorcontributors becauseoftheirlowlatenciesrelativetoserversandstoragearrays.Ensurethatthe pathsthroughtheswitchfabricarenotsaturated,thatis,theswitchfabricisrunningat thehighestthroughput.
Storage Array Performance

Ifthereareissueswithstoragearrayperformance,besuretoconsultyourstoragearray vendorsdocumentationforanyrelevantinformation. WhenassigningLUNs,rememberthateachLUNisaccessedbyanumberofESXServer hosts,andthatanumberofvirtualmachinescanrunoneachhost.OneLUNusedby anESXServerhostcanserviceI/Ofrommanydifferentapplicationsrunningon differentoperatingsystems.Becauseofthisdiverseworkload,theRAIDgroup containingtheESXServerLUNsshouldnotincludeLUNsusedbyotherhoststhatare notrunningESXServerforI/Ointensiveapplications. Makesureread/writecachingisenabled. LoadbalancingistheprocessofspreadingserverI/OrequestsacrossallavailableSPs andtheirassociatedhostserverpaths.Thegoalistooptimizeperformanceintermsof throughput(I/Opersecond,megabytespersecond,orresponsetimes). SANstoragearraysrequirecontinualredesignandtuningtoensurethatI/Oisload balancedacrossallstoragearraypaths.Tomeetthisrequirement,distributethepaths totheLUNsamongalltheSPstoprovideoptimalloadbalancing.Closemonitoring indicateswhenitisnecessarytomanuallyrebalancetheLUNdistribution.SeePath ManagementandManualLoadBalancingonpage 100foranexample. Tuningstaticallybalancedstoragearraysisamatterofmonitoringthespecific performancestatistics(suchasI/Ooperationspersecond,blockspersecond,and responsetime)anddistributingtheLUNworkloadtospreadtheworkloadacrossall theSPs. NOTEDynamicloadbalancingisnotcurrentlysupportedwithESXServer.
106
VMware, Inc.
Server Performance
Ensuringoptimalserverperformancerequireslookingatanumberoffactors.Each serverapplicationmusthaveaccesstoitsdesignatedstoragewith:

HighI/Orate(numberofI/Ooperationspersecond) Highthroughput(megabytespersecond) Minimallatency(responsetimes)
Becauseeachapplicationhasdifferentrequirements,youcanmeetthesegoalsby choosinganappropriateRAIDgrouponthestoragearray.Toachieveperformance goals:
PlaceeachLUNonaRAIDgroupthatprovidesthenecessaryperformancelevels. PayattentiontotheactivitiesandresourceutilizationofotherLUNSinthe assignedRAIDgroup.AhighperformanceRAIDgroupthathastoomany applicationsdoingI/Otoitmightnotmeetperformancegoalsrequiredbyan applicationrunningontheESXServerhost. MakesurethateachserverhasasufficientnumberofHBAstoallowmaximum throughputforalltheapplicationshostedontheserverforthepeakperiod.I/O spreadacrossmultipleHBAsprovidehigherthroughputandlesslatencyforeach application. ToprovideredundancyintheeventofHBAfailure,makesuretheserveris connectedtoadualredundantfabric. WhenallocatingLUNsorRAIDgroupsforESXServersystems,multipleoperating systemswilluseandsharethatresource.Asaresult,theperformancerequired fromeachLUNinthestoragesubsystemcanbemuchhigherifyouareworking withESXServersystemsthanifyouareusingphysicalmachines.Forexample,if youexpecttorunfourI/Ointensiveapplications,allocatefourtimesthe performancecapacityfortheESXServerLUNs. WhenusingmultipleESXServersystemsinconjunctionwithaVirtualCenter Server,theperformanceneededfromthestoragesubsystemincreases correspondingly. ThenumberofoutstandingI/OsneededbyapplicationsrunningonanESXServer systemshouldmatchthenumberofI/OstheHBAandstoragearraycanhandle.
VMware, Inc.
107
Resolving Performance Issues

Thissectiondiscussesperformancemonitoringandpossiblewaysofresolving performanceissues. Forbestperformance,placeeachvirtualmachineontheappropriatetierofstorage.See ChoosingVirtualMachineLocationsonpage 43forinformation.
Monitoring Performance
TheVIClientoffersextensivefacilitiesforcollectingperformanceinformation.The informationisthengraphicallydisplayedintheVIClient.Forinformation,seetheBasic SystemAdministration.TheVIClientupdatesitsdisplayperiodically. WithESXServer3,youcanalsousetheesxtoptool,availablefromtheserviceconsole. Forinformationaboutesxtop,seetheResourceManagementGuide,orlookattheman pagefromtheserviceconsole.Youcanuseesxtoptomonitorperformanceinrealtime. IfyouareusingESXServer3i,similarfunctionalityisprovidedbytheresxtoptool.
Resolving Path Thrashing

IfyourserverisunabletoaccessaLUN,oraccessisveryslow,youmighthavea problemwithpaththrashing(alsocalledLUNthrashing).Paththrashingmightoccur whentwohostsaccesstheLUNthroughdifferentSPsand,asaresult,theLUNisnever actuallyavailable. Usually,onlyspecificSANconfigurationsinconjunctionwiththefollowingconditions cancausethepaththrashing:

Youareworkingwithanactive/passivearray. PathpolicyissettoFixed. TwohostsaccesstheLUNusingoppositepathorder.Forexample,HostAissetup toaccessthelowernumberedLUNthroughSPA.HostBissetuptoaccessthe lowernumberedLUNthroughSPB.
PaththrashingcanalsooccurifHostAlostacertainpathandcanuseonlypathstoSP AwhilehostBlostotherpathsandcanuseonlypathstoSPB. Thisproblemcanalsooccuronadirectconnectarray(suchasAX100)withHBA failoverononeormorenodes.
108
VMware, Inc.
Paththrashingisaproblemthatyoutypicallywontexperiencewithotheroperating systems:
NoothercommonoperatingsystemusessharedLUNsformorethantwoservers (thatsetupistypicallyreservedforclustering). Forclustering,onlyoneserverisissuingI/Osatatime.Paththrashingdoesnot becomeaproblem.
Incontrast,multipleESXServersystemsmaybeissuingI/OtothesameLUN concurrently. To resolve path thrashing
EnsureallhostssharingthesamesetofLUNsonthoseactive/passivearraysaccess thesamestorageprocessorsimultaneously. CorrectanycablinginconsistenciesbetweendifferentESXServerhostsandSAN targetssothatallHBAsseethesametargetsinthesameorder. MakesurethepathpolicyissettoMostRecentlyUsed(thedefault).
Understanding Path Thrashing

InallarraystheSPsarelikeindependentcomputersthathaveaccesstosomeshared storage.Algorithmsdeterminehowconcurrentaccessishandled.
Foractive/passivearrays,allthesectorsonthestoragethatmakeupagivenLUN canbeaccessedbyonlyoneSPatatime.Theownershipispassedaroundbetween thestorageprocessors.ThereasonisthatstoragearraysusecachesandSPAmust notwritesomethingtodiskthatinvalidatestheSPBcache.BecausetheSPhasto flushthecachewhenitsdonewithitsoperation,ittakesalittletimetomovethe ownership.Duringthattime,noI/OtotheLUNcanbeprocessedbyeitherSP. Foractive/activearrays,thealgorithmsallowmorefinegrainedaccesstothe storageandsynchronizecaches.AccesscanhappenconcurrentlythroughanySP withoutextratimerequired.
ArrayswithAVTareactive/passivearraysthatattempttolooklikeactive/activearrays bypassingtheownershipoftheLUNtothevariousSPsasI/Oarrives.Thisapproach worksinaclusteringsetup,butifmanyESXServersystemsaccessthesameLUN concurrentlythroughdifferentSPs,theresultisLUNthrashing.
VMware, Inc.
109
Considerhowpathselectionworks:

Onanactive/activearraythesystemstartssendingI/Odownthenewpath. Foractive/passivearrays,theESXServersystemchecksallstandbypaths.TheSP attheendofthepaththatiscurrentlyunderconsiderationsendsinformationto thesystemonwhetheritcurrentlyownstheLUN.
IftheESXServersystemfindsanSPthatownstheLUN,thatpathisselected andI/Oissentdownthatpath. IftheESXServerhostcannotfindsuchpath,theESXServerhostpicksoneof thepathsandsendstheSP(attheotherendofthepath)acommandtomove theLUNownershiptothisSP.
Paththrashingcanoccurasaresultofthispathchoice:IfserverAcanreachaLUN onlythroughoneSP,andserverBcanreachthesameLUNonlythroughadifferent SP,theybothcontinuouslycausetheownershipoftheLUNtomovebetweenthe twoSPs,effectivelypingpongingtheownershipoftheLUN.Becausethesystem movestheownershipquickly,thestoragearraycannotprocessanyI/O(orcan processonlyverylittle).Asaresult,anyserversthatdependontheLUNstart timingoutI/O.
Equalizing Disk Access Between Virtual Machines

Youcanadjustthemaximumnumberofoutstandingdiskrequestswiththe Disk.SchedNumReqOutstandingparameterintheVIClient.Whentwoormore virtualmachinesareaccessingthesameLUN,thisparametercontrolsthenumberof outstandingrequestseachvirtualmachinecanissuetotheLUN.Adjustingthelimit canhelpequalizediskaccessbetweenvirtualmachines. ThislimitisinapplicablewhenonlyonevirtualmachineisactiveonaLUN.Inthat case,thebandwidthislimitedbythequeuedepthofthestorageadapter. To set the number of outstanding disk requests 1 2 3 4 5 IntheVIClient,selectthehostintheinventorypanel. ClicktheConfigurationtabandclickAdvancedSettings. ClickDiskintheleftpanelandscrolldowntoDisk.SchedNumReqOutstanding. ChangetheparametervaluetothenumberofyourchoiceandclickOK. Reboottheserver.
110
VMware, Inc.
Thischangecanimpactdiskbandwidthscheduling,butexperimentshaveshown improvementsfordiskintensiveworkloads. IfyouadjustthisvalueintheVMkernel,youmightalsowanttoadjustthequeuedepth inyourstorageadapter.SeeSettingMaximumQueueDepthforHBAsonpage 112.
Removing VMFS-2 Drivers

IfyouhavealotofVMFSdatastores,andallofthemareVMFS3,youcanpotentially improveperformancebyunloadingtheVMFS2driver.Atacommandlineprompt, type:
vmkload_mod -u vmfs2
Asignificantincreaseinthespeedofcertainmanagementoperationslikerefreshing datastoresandrescanningstorageadaptersshouldresult.However,becausethis commandisvalidforthecurrentbootonly,youneedtorepeatitaftereachreboot.
Removing NFS Drivers

IfyoudonotuseNFSdatastores,youcanunloadtheNFSdriverbytypingthe followingatacommandlineprompt.Thecommandisvalidforthecurrentbootonly, asaresult,youneedtorepeatitaftereachreboot.
vmkload_mod -u nfsclient
Reducing SCSI Reservations

OperationsthatrequiregettingafilelockorametadatalockinVMFSresultin shortlivedSCSIreservations.SCSIreservationslockanentireLUN.ExcessiveSCSI reservationsbyaservercancauseperformancedegradationonotherserversaccessing thesameVMFS. Examplesofoperationsthatrequiregettingfilelocksormetadatalocksinclude:

Virtualmachinepoweron. VMotion. Virtualmachinesrunningwithvirtualdisksnapshots. Fileoperationsthatrequireopeningfilesordoingmetadataupdates.(See MetadataUpdatesonpage 32.)
VMware, Inc.
111
Therecanbeperformancedegradationifsuchoperationsarehappeningfrequentlyon multipleserversaccessingthesameVMFS.Forinstance,itisnotrecommendedtorun manyvirtualmachinesfrommultipleserversthatareusingvirtualdisksnapshotson thesameVMFS.LimitthenumberofVMFSfileoperationswhenmanyvirtual machinesarerunningontheVMFS.
Setting Maximum Queue Depth for HBAs

YourESXServershouldhavereasonablequeuedepths.However,ifyouarenot satisfiedwiththeperformanceofyourHBAs,youcanchangetheirmaximumqueue depth.
Adjusting Queue Depth for a QLogic HBA

YoucanadjustthemaximumqueuedepthforaQLogicqla2x00seriesadapterbyusing thefollowingprocedure. To set maximum queue depth for a QLogic HBA 1 2 Logintotheserviceconsoleasroot. VerifywhichQLogicHBAmoduleiscurrentlyloaded:
vmkload_mod -l | grep qla2300
DependingonthemodeloftheHBA,themodulecanbeoneofthefollowing:

qla2300_707(ESXServer3.0.x) qla2300_707_vmw(ESXServer3.5)
Runthefollowingcommands. Theexampleshowstheqla2300_707module.Usetheappropriatemodulebased ontheoutcomeofStep 2. a b esxcfg-module -s ql2xmaxqdepth=64 qla2300_707 esxcfg-boot b
Inthiscase,theHBArepresentedbyql2xwillhaveitsLUNqueuedepthsetto64. 4 Reboot.
112
VMware, Inc.
Adjusting Queue Depth for an Emulex HBA

YoucanadjustthemaximumqueuedepthforanEmulexHBAwiththefollowing procedure. To change the queue depth of an Emulex HBA 1 2 Logintotheserviceconsoleasroot. VerifywhichEmulexHBAmoduleiscurrentlyloaded:
vmkload_mod -l | grep lpfcdd
DependingonthemodeloftheHBA,themodulecanbeoneofthefollowing:

lpfcdd_7xx lpfcdd_732.oThis4GBdriverisincludedwithESXServer3.x.Insome cases,youmightneedtodowngradetoa2GBdriver.See http://kb.vmware.com/kb/1560391.
ForasingleinstanceofanEmulexHBAonthesystem,runthefollowing commands. Theexampleshowsthelpfcdd_7xxmodule.Usetheappropriatemodulebased ontheoutcomeofStep 2. a b esxcfg-module -s lpfc0_lun_queue_depth=16 lpfcdd_7xx esxcfg-boot b
Inthiscase,theHBArepresentedbylpfc0willhaveitsLUNqueuedepthset to 16. 4 FormultipleinstancesofanEmulexHBAbeingpresentonthesystem,runthe followingcommands: a b esxcfg-module -s "lpfc0_lun_queue_depth=16 lpfc1_lun_queue_depth=16" lpfcdd_7xx esxcfg-boot b
Inthiscase,bothHBAslpfc0andlpfc1willhavetheirLUNqueuedepthsset to 16. 5 Reboot.
VMware, Inc.
113
SAN Storage Backup Considerations

WithintheSANenvironment,backupshavetwogoals.Thefirstgoalistoarchive onlinedatatoofflinemedia.Thisprocessisrepeatedperiodicallyforallonlinedataon atimeschedule.Thesecondgoalistoprovideaccesstoofflinedataforrecoveryfrom aproblem.Forexample,databaserecoveryoftenrequiresretrievalofarchivedlogfiles thatarenotcurrentlyonline. Schedulingabackupdependsonanumberoffactors:
Identificationofcriticalapplicationsthatrequiremorefrequentbackupcycles withinagivenperiodoftime. Recoverypointandrecoverytimegoals.Considerhowpreciseyourrecoverypoint needstobe,andhowlongyouarewillingtowaitforit. Therateofchange(RoC)associatedwiththedata.Forexample,ifyouareusing synchronous/asynchronousreplication,theRoCaffectstheamountofbandwidth requiredbetweentheprimaryandsecondarystoragedevices. OverallimpactonSANenvironment,storageperformance(whilebackingup), andotherapplications. IdentificationofpeaktrafficperiodsontheSAN(backupsscheduledduringthose peakperiodscanslowtheapplicationsandthebackupprocess). Timetoscheduleallbackupswithinthedatacenter. Timeittakestobackupanindividualapplication. Resourceavailabilityforarchivingdata;usuallyofflinemediaaccess(tape).
Includearecoverytimeobjectiveforeachapplicationwhenyoudesignyourbackup strategy.Thatis,considerthetimeandresourcesnecessarytoreprovisionthedata.For example,ifascheduledbackupstoressomuchdatathatrecoveryrequiresa considerableamountoftime,examinethescheduledbackup.Performthebackupmore frequently,sothatlessdataisbackedupatatimeandtherecoverytimedecreases. Ifaparticularapplicationrequiresrecoverywithinacertaintimeframe,thebackup processneedstoprovideatimescheduleandspecificdataprocessingtomeetthis requirement.Fastrecoverycanrequiretheuseofrecoveryvolumesthatresideon onlinestoragetominimizeoreliminatetheneedtoaccessslowofflinemediafor missingdatacomponents.
114
VMware, Inc.
Snapshot Software
Snapshotsoftwareallowsanadministratortomakeaninstantaneouscopyofanysingle virtualdiskdefinedwithinthedisksubsystem.Snapshotsoftwareisavailableat differentlevels:
ESXServerhostsallowyoutocreatesnapshotsofvirtualmachines.Thissoftware isincludedinthebasicESXServerpackage. Thirdpartybackupsoftwaremightallowformorecomprehensivebackup proceduresandmightcontainmoresophisticatedconfigurationoptions.
Administratorsmakesnapshotsforavarietyofreasons,including:

Backup. Disasterrecovery. Availabilityofmultipleconfigurations,versions,orboth. Forensics(lookingatasnapshottofindthecauseofproblemswhileyoursystem isrunning). Datamining(lookingatacopyofyourdatatoreduceloadonproductionsystems).
Using a Third-Party Backup Package

Ifyouareusingthirdpartybackupsoftware,makesurethatthesoftwareissupported withESXServerhosts.SeetheBackupSoftwareCompatibilityGuide. Usingthirdpartysoftwarehastheadvantageofauniformenvironment.However,you havetoconsiderthattheadditionalcostofthethirdpartysnapshottingsoftwarecan becomehigherasyourSANgrows. Ifyouusesnapshotstobackupyourdata,considerthefollowingpoints:
SomevendorssupportsnapshotsforbothVMFSandRDMs.Ifbotharesupported, youcanmakeeitherasnapshotofthewholevirtualmachinefilesystemforahost, orsnapshotsfortheindividualvirtualmachines(oneperdisk). SomevendorssupportsnapshotsonlyforasetupusingRDM.IfonlyRDMis supported,youcanmakesnapshotsofindividualvirtualmachines.
Seeyourstoragevendorsdocumentation. NOTEESXServersystemsalsoincludeaConsolidatedBackupcomponent,whichis discussedindetailintheVirtualMachineBackupGuide.
VMware, Inc.
115
Choosing Your Backup Solution

Whenchoosingyourbackupsolution,considerthatabackupcanbeoneorallofthese:

Crashconsistent Filesystemconsistent Applicationconsistent
VMwareoffersafilesystemconsistentbackup.Inmostcases,afilesystemconsistent backupallowsyoutocompletelyrecoverfromfailure.However,ifyourapplications requiresynchronizationacrossfilesystemsorwithadatabase,theVMwaresolution mightnotprovideenoughconsistency.Inthesecases,youshouldinvestigatea thirdpartybackupsolutiontoseewhetheritbettersuitsyourneeds.
Layered Applications
SANadministratorscustomarilyusespecializedarraybasedsoftwareforbackup, disasterrecovery,datamining,forensics,andconfigurationtesting. Storageproviderstypicallysupplytwotypesofadvancedservicesfortheir LUNssnapshottingandreplication.
SnapshottingcreatesspacewithefficientcopiesofLUNsthatsharecommon blocksofdata.Ingeneral,snapshottingisusedlocallyonthesamearrayasthe primaryLUNforquickbackups,applicationtesting,forensics,ordatamining. ReplicationcreatesfullcopiesofLUNs.Replicasareusuallymadetoseparate arrays,possiblyseparatesitestoprotectagainstmajoroutagesthatincapacitateor destroyanentirearrayorsite.
WhenyouuseanESXServersysteminconjunctionwithaSAN,youneedtodecide whetherarraybasedorhostbasedtoolsaremoresuitableforyourparticularsituation.
Array-Based (Third-Party) Solution

Whenconsideringanarraybasedsolution,considerthefollowingpoints: NOTEESXServersystemsalsoincludeaconsolidatedbackupcomponent,whichis discussedindetailintheVirtualMachineBackupGuide.
Arraybasedsolutionsusuallyresultinmorecomprehensivestatistics.WithRDM, dataalwaystakesthesamepath,whichresultsineasierperformance management.
116
VMware, Inc.
SecurityismoretransparenttothestorageadministratorwhenyouuseRDMand anarraybasedsolutionbecausewithRDM,virtualmachinesmoreclosely resemblephysicalmachines. Ifyouuseanarraybasedsolution,physicalcompatibilityRDMsareoftenusedfor thestorageofvirtualmachines.IfyoudonotintendtouseRDM,youshouldcheck thestoragevendordocumentationtoseeifoperationsonLUNswithVMFS volumesaresupported.Furthermore,ifyouusearrayoperationsonVMFSLUNs, youshouldcarefullyreadthesectiononresignaturing.
File-Based (VMFS) Solution

WhenconsideringafilebasedsolutionusingVMwaretoolsandVMFS(insteadofthe arraytools),beawareofthefollowingpoints:
UsingVMwaretoolsandVMFSisbetterforprovisioning:onelargeLUNis allocatedandmultiple.vmdk filescanbeplacedonthatLUN.WithRDM,anew LUNisrequiredforeachvirtualmachine. SnapshottingisincludedwithyourESXServerhostatnoextracost.Thefilebased solutionisthereforemorecosteffectivethanthearraybasedsolution. ForESXServeradministrators,usingVMFSiseasier. ESXServeradministratorswhousethefilebasedsolutionaremoreindependent fromtheSANadministrator.
VMFS Volume Resignaturing

ESXserversneedtobeabletodifferentiatebetweentheirVMFSvolumesandusea volumesignaturetodoso.WhenaVMFSvolumeisreplicatedorsnapshotted,the resultingLUNcopyhasthesamesignatureasthesource.WhentwoLUNsseenbyan ESXServerhavethesamesignature,theESXServermusthandletheconditionto preventdowntimecausedbyconfusionoverwhichLUNitshouldbeusingtoaccess theregisteredvirtualmachines.ResignaturingisafeatureintroducedinESXServer3.0 tosolvethisproblem. NOTEWhenaLUNneedstoberesignatured,analertappearsinthevmkernellog.If youencountersuchanalert,setyourresignaturingoptionsappropriately,asdescribed inthefollowingsections.
VMware, Inc.
117
Mounting Original, Snapshot, or Replica VMFS Volumes

Youcanmountoriginal,snapshot,orreplicaVMFSvolumesonthesameESXServer host. To mount original, snapshot, or replica VMFS volumes 1 Performtherequiredstoragetasks: a b 2 3 4 5 Makethearraysnapshotorreplica. MaskorzonethesnapshotorreplicatotheESXServer.
IntheVIClient,selectthehostintheinventorypanel. ClicktheConfigurationtabandclickAdvancedSettings. SelectLVMintheleftpanel,thensettheLVM.EnableResignatureoptionto1. RescanforanynewLUNsorVMFSvolumes.Volumesthataredetectedtobe snapshotsorreplicasareresignatured. Afterrescan,thecopiedVMFSvolumeappearsas /vmfs/volumes/snap-<DIGIT>-<old-label>. Ifthe.vmxfileforanyofthevirtualmachinesorthe.vmsdfileforvirtualmachine snapshotscontains/vmfs/volumes/<label or UUID>/paths,youmustchange theseitemstoreflecttheresignaturedvolumepath.
SettheLVM.EnableResignatureoptionto0afterresignaturingiscomplete.
Understanding Resignaturing Options

ThissectiondiscusseshowtheEnableResignatureandDisallowSnapshotLUN optionsinteractandexplainsthethreestatesthatresultfromchangingtheseoptions:
State1:EnableResignature=0,DisallowSnapshotLUN=1(theESXServer3.x default) State2:EnableResignature=1(DisallowSnapshotLUNisnotrelevant) State3:EnableResignature=0,DisallowSnapshotLUN=0(ESXServer2.xbehavior)
118
VMware, Inc.
State 1 - EnableResignature=0, DisallowSnapshotLUN=1 (default)

Inthisstate:
YoucannotbringsnapshotsorreplicasofVMFSvolumesbythearrayintotheESX ServerhostregardlessofwhetherornottheESXServerhasaccesstotheoriginal LUN. LUNsformattedwithVMFSmusthavethesameIDforeachESXServerhost.
State 2 - EnableResignature=1, (DisallowSnapshotLUN is not relevant)

Inthisstate,youcansafelybringsnapshotsorreplicasofVMFSvolumesintothesame serversastheoriginalandtheyareautomaticallyresignatured.
State 3 - EnableResignature=0, DisallowSnapshotLUN=0

ThisissimilartoESXServer2.xbehavior.Inthisstate,theESXServerassumesthatit seesonlyonereplicaorsnapshotofagivenLUNandnevertriestoresignature.Thisis idealinaDRscenariowhereyouarebringingareplicaofaLUNtoanewclusterofESX Servers,possiblyonanothersitethatdoesnothaveaccesstothesourceLUN.Insucha case,theESXServerusesthereplicaasifitistheoriginal. DonotusethissettingifyouarebringingsnapshotsorreplicasofaLUNintoaserver withaccesstotheoriginalLUN.Thiscanhavedestructiveresultsincluding:
IfyoucreatesnapshotsofaVMFSvolumeoneormoretimesanddynamically bringoneormoreofthosesnapshotsintoanESXServer,onlythefirstcopyis usable.Theusablecopyismostlikelytheprimarycopy.Afterreboot,itis impossibletodeterminewhichvolume(thesourceoroneofthesnapshots)is usable.Thisnondeterministicbehaviorisdangerous. IfyoucreateasnapshotofaspannedVMFSvolume,anESXServerhostmight reassemblethevolumefromfragmentsthatbelongtodifferentsnapshots.Thiscan corruptyourfilesystem.
VMware, Inc.
119
120
VMware, Inc.
Multipathing Checklist
Thisappendixprovidesachecklistofmultipathingsetuprequirementsfordifferent storagearrays. Table A-1. Multipathing Setup Requirements

Component Allstoragearrays Topology IBMTotalStorageDS 4000(formerlyFastT) Comments Writecachemustbedisabledifnotbatterybacked. NosinglefailureshouldcausebothHBAandSPfailover,especiallywith activepassivestoragearrays. DefaulthosttypemustbeLNXCL or VMwareinlaterversions. HosttypemustbeLNXCL or VMwareinlaterversions. AVT(AutoVolumeTransfer)isdisabledinthishostmode. HDS99xxand95xxV family HDS9500Vfamily(Thunder)requirestwohostmodes:

HostMode1:Standard. HostMode2:Sun Cluster
HDS99xxfamily(Lightning)andHDSTabma(USP)requirehostmodesetto Netware. EMCSymmetrix EMCClariion EnabletheSPC2andSC3settings.ContactEMCforthelatestsettings. AllInitiatorrecordsmusthave:

FailoverMode=1 InitiatorType=ClariionOpen ArrayCommPath=Enabledor1
HPMSA
HosttypemustbeLinux. SettheconnectiontypeforeachHBAporttoLinux.
VMware, Inc.
121
Table A-1. Multipathing Setup Requirements (Continued)

Component HPEVA Comments (SEEUPDATE)ForEVA3000/5000firmware4.001andabove,and EVA4000/6000/8000firmware5.031andabove,setthehosttypetoVMware. Otherwise,setthehostmodetypetoCustom.Thevalueis:

EVA3000/5000firmware3.x:000000002200282E EVA4000/6000/8000:000000202200083E
HPXP NetApp ESXServer Configuration
ForXP128/1024/10000/12000,thehostmodeshouldbesetto0C (Windows), thatis,zeroC(Windows). Nospecificrequirements (SEEUPDATE)SetthefollowingAdvancedSettingsfortheESXServerhost:

SetDisk.UseLunResetto1 SetDisk.UseDeviceResetto0
AmultipathingpolicyofMostRecentlyUsedmustbesetforallLUNshosting clustereddisksforactivepassivearrays.AmultipathingpolicyofMost RecentlyUsedorFixedmaybesetforLUNsonactiveactivearrays. AllFCHBAsmustbeofthesamemodel.
122
VMware, Inc.
Utilities
Inmostcases,theVIClientiswellsuitedformonitoringanESXServerhostconnected toSANstorage.Advancedusersmight,attimes,wanttousesomecommandline utilitiesforadditionaldetails. Thisappendixprovidesinformationonthefollowingutilities:

esxtopandresxtopUtilitiesonpage 124 storageMonitorUtilityonpage 124
VMware, Inc.
123
esxtop and resxtop Utilities

Theesxtopandresxtop commandlinetoolsprovideafinegrainedlookatESX Serverresourceutilizationinrealtime.Fordetailedinformationabouttheutilities,see theResourceManagementGuide.
storageMonitor Utility
ThestorageMonitorutilitymonitorsSCSIsenseerrorsexperiencedbystorage devicesattachedtoVMwareESXServer.Theutilitygatherssenseerrorinformationby periodicallypollingthestorageMonitorrunninginsidetheVMkernel,andsends errorinformationtostandardoutput,afile,orthesystemlog.Itformatserror informationbeforesendingittooutput.Forexample,itconvertssenseerrorcodesto correspondingtextasperSCSI3specification. Ifnoconfigurationfileisspecified,storageMonitorparsesthedefaultconfiguration file/etc/vmware/storageMonitor.conftofiltercertainerrorsandallowothererrors tobedisplayed.YoucanrunstorageMonitorininteractivemodeordaemonmode usingthe-doption.
Options
YoucaninvokestorageMonitorfromtheESXServercommandlineusingoneofthe followingoptions. Table B-1. storageMonitor Command-Line Options
Option <config-file> Description Allowsyoutospecifyaconfigurationfile.Ifthisoptionisleft unspecified,thedefaultisused. TheconfigurationfilespecifieswhichtypeoferrorsstorageMonitor shouldallowandwhichonesitshouldfilterbeforedisplayingthem. Thedefaultconfigurationfileillustratestheformatoftheentries. -d SpecifiesthatstorageMonitorshouldberunindaemonmode.When thisoptionisspecifiedalloutputgoeseithertosyslogortoalogfile specifiedbytheuser.Ifthe-soptionisalsospecified,outputiswritten tostandardoutaswell. Displayshelpinformation. Whenthisoptionisspecified,outputfromtheprogramiswrittento <log_file>.Thisoptionisvalidonlyifthe-doptionisalsospecified.
-h -l <log_file>
124
VMware, Inc.
Appendix B Utilities
Table B-1. storageMonitor Command-Line Options (Continued)

Option -p <poll_interval> Description Allowsyoutospecifytheinterval(inseconds)usedforpollingkernel residentstorageandforretrievingthestatusorerrorsofthestorage devices.Ifthisoptionisnotspecified,thedefaultpollingintervalof10 secondsisused. SpecifiesthatstorageMonitorshouldsendoutputtostandardout. ThisoptionisonlyvalidifyoustartstorageMonitorindaemon mode(-doptionisspecified).
-s
Examples
storageMonitor -p 60
Setsthepollingintervalto60seconds.Sendsoutputtostandardout(because storageMonitorisnotrunningindaemonmode).Usesthefiltersspecifiedinthe defaultconfigurationfilebeforesendingtheoutput.

storageMonitor -d -c myconf.conf
RunsstorageMonitorindaemonmodeusingtheconfigurationfilemyconf.conf. Writesoutputtosyslog.Bydefault,syslogislocatedat/var/log/storageMonitor.
storageMonitor -d -l mylog.log -s
RunsstorageMonitorindaemonmodeusingthedefaultconfigurationfile.Sends outputtomylog.loginsteadofsyslog.Alsowritesoutputtostandardoutbecausethe -soptionisspecified.
VMware, Inc.
125
126
VMware, Inc.
Index
Symbols
* next to path 97 .vmdk file 18
B
backups 50 considerations 114 solution 116 third-party backup package 115 benefits 28 BIOS enabling for BFS 78 enabling Qlogic HBA for BFS 76 boot BIOS prompt, enabling for BFS 78 boot from CD-ROM 78 boot from SAN benefits 73 boot LUN considerations 53 conceptual overview 72 diagnostic partitions 74 Emulex FC HBA 78 enabling Qlogic HBA BIOS 76 ESX Server requirements 53 HBA requirements 53 introduction 72 LUN masking 74 preparing installation 73 Qlogic FC HBA 76 requirements 53 boot LUN 77 boot LUN, selecting 77 BusLogic queue depth 50 SCSI controller 17
A
access control 36 equalizing disk access 110 active path status 96 active/active disk arrays 51, 66, 97, 100 active/passive disk arrays 43, 51, 77, 97 boot from SAN 53 HP StorageWorks MSA 66 path policy reset 83 path thrashing 109, 110 advanced LUN display configuration 88 allocations 51 applications,layered 116 array-based (third-party) solution 116 asterisk next to path 97 authentication daemon 22 auto volume transfer 64 avoiding problems 105 AVT 64, 109 AX100 display problems 61 inactive connections 61
VMware, Inc.
127
C
cables connectivity issues 86 reseating and rescan 87 cant see LUN 85 CD-ROM, booting from 78 changing disk.maskLuns 89 changing disk.maxLun 88 changing disk.supportSparseLun 90 cluster across boxes 44 cluster in a box 44 cluster services 44 commands, SDK 20 configuration options 18 storage processor sense data 65 configuring hardware for SAN failover DS4000 62 consolidated backup proxy 50 CPU virtualization 16 current multipathing state 95
D
data access 39 RDM 34 VMFS 34 datastores, removing 88 dead paths 96 design for server failure 44 details about storage devices 84 device driver options 103 Emulex 103 Qlogic 103 device drivers 14 diagnostic partitions 50 boot from SAN 74 sharing 104 direct connect 58
128
disabled path status 96 disabling auto volume transfer 64 disabling paths 99 disallowSnapshotLUN 119 disaster recovery 30 disk access, equalizing 110 disk arrays active/active 51, 97, 100 active/passive 51, 66, 77, 97, 109 zoning disk array to ESX Server 87 disk shares 38 disk.maskLuns 89 disk.maxLun 88 Disk.SchedNumReqOutstanding parameter 110 disk.supportSparseLun 90 disks, configuration options 18 display problems, AX100 61 distributed locking 19 drivers device drivers 103 VMFS-2 111 DRS 47 DS4000, configuring hardware for SAN failover 62 dump partitions 50 sharing 104
E
EMC CLARiiON 60 EMC Symmetrix 61 pseudo LUNs 62 Emulex FC HBA boot from SAN 78 device driver options 103 lpfc_linkdown_tmo 103 NPIV support 92 enableResignature 119 enabling BIOS for BFS 78
VMware, Inc.
Index
enabling boot BIOS prompt for BFS 78 enabling paths 99 enabling selectable boot 77 equalizing disk access 110 ESX Server benefits 28 introduction 14 sharing VMFS 31 ESX Server 3 14, 72, 104, 108 ESX Server 3i 14, 108 ESX Server and SAN, requirements 50 esxtop utility 124 EVA (HP StorageWorks) 68 extents 19 definition 85
F
failover 41, 45, 102 FAStT storage 62 HBA 103 I/O delay 43 zones 36 failure 44 FC HBA setup 51 finding information 30 Fixed path policy 43, 97, 98 path thrashing 109 preferred path 100
list of types 83 Qlogic 76, 92 queue depth 112 setup 51 static load balancing 51 timeout 103 types 83 high-tier storage 43 Hitachi Data Systems storage 69 microcode 69 host daemon 22 host type 59 hostd 22 HP StorageWorks 66 EVA 68 MSA 66 XP 69 hub controller issues 68
I
I/O delay 43, 50 IBM TotalStorage DS4000 62 path thrashing 66 IBM TotalStorage Enterprise Storage Systems 66 indirection 33 installation preparing for boot from SAN 73 SAN 53 steps 54 interacting with ESX Server systems 20 Inter-Switch Link 63 iSCSI 19 ISL 63 issues 82 hub controller 68 performance 108 visibility 85
G
getting information 83
H
HA 44 hardware compatibility 15 HBA Emulex 78, 92 enabling Qlogic HBA BIOS for BFS 76
VMware, Inc.
129
L
layered applications 116 LDAP 22 levels of indirection 33 Linux host type 59 profile name 66 service console 22 VMkernel 14 Linux Cluster host type 59 list of HBA types 83 load balancing 29, 51 manual 100 locations of virtual machines 43 locking 19 metadata updates 32 lower-tier storage 43 lpfc_linkdown_tmo parameter 103 lpfc_nodedev_tmo parameter 103 LSI Logic SCSI controller 17 LSILogic queue depth 50 LUN discovery, VMkernel 33 LUN masking 36, 86 boot from SAN 74 LUN not visible cable connectivity 86 issues 86 masking 86 SP visibility 87 zoning 86 LUNs 1 VMFS volume 50 allocations 51 boot LUN 77 cant see 85 changing number scanned 88 creating, and rescan 87 decisions 37
disk.maskLuns 89 display and rescan 32 display configuration 88 fewer, larger vs. more, smaller 36 masking changes and rescan 87 multipathing policy 98 NPIV-based access 90 number scanned 88 removing 88 selecting boot LUN 77 setting multipathing policy 98 sparse 90
M
maintenance 29 Manage Paths dialog box 99 management applications 35 manual load balancing 100 mapping file 19 masking 36 LUN not visible 86 using disk.maskLuns 89 maximum HBA queue depth 112 maxLun 88 memory virtualization 16 metadata updates 32 microcode, Hitachi Data Systems storage 69 Microsoft Cluster Service 19, 29, 58 mid-tier storage 43 monitoring performance 108 Most Recently Used path policy 97, 98 path thrashing 109 MRU path policy 97 MSA (HP StorageWorks) 66 MSCS 29, 58 multipathing 95 multipathing policy 98 multipathing software 34
VMware, Inc.
130
Index
multipathing state 95 multiple extents 85
N
N+1 clustering 45 net-snmpd 22 Netware host mode 69 Network Appliance storage 69 provisioning storage 69 network virtualization 16 NFS 19 N-Port ID Virtualization (NPIV) 27, 90 requirements 91 number of extents 19 number of outstanding disk requests 110
O
operating system timeout 104 optimizing resource utilization 46 outstanding disk requests 110
P
PAM 22 parameters, lpfc_nodedev_tmo 103 passive disk arrays 51, 66, 77, 97 path thrashing 109 path failover 41 path failure rescan 87 path management 41, 100 path policies Fixed 43, 97, 98 Most Recently Used 98 MRU 97 Round Robin 98 path policy reset active/passive disk array 83 path status 96 path thrashing 60, 108, 109 IBM TotalStorage DS4000 66
paths disabling 99 enabling 99 preferred 97, 100 performance 106, 107 issues 108 monitoring 108 optimizing 106 removing VMFS-2 drivers 111 SCSI reservations 31 physical to virtual clustering 45 pitfalls 83 Pluggable Authentication Module 22 port address 27 Port_ID 27 PortDownRetryCount parameter 103 ports, configuration 63 preferred path 97, 100 prioritizing virtual machines 38 problems avoiding 105 hub controller 68 performance 108 visibility 85 profile name, Linux 66
Q
Qlogic FC HBA boot from SAN 76 device driver options 103 NPIV support 92 PortDownRetryCount 103 Qlogic HBA BIOS, enabling for BFS 76 queue depth 112
VMware, Inc.
131
R
RADIUS 22 raw device mapping 19, 39, 60 mapping file 19 RDM 19, 34, 39, 60 mapping file 19 Microsoft Cluster Service 19 removing datastores 88 removing LUNs 88 removing VMFS-2 drivers 111 requirements 50 boot from SAN 53 rescan 87 adding disk array 87 LUN creation 87 LUN display 32 LUN masking 87 reseating cables 87 when path is down 87 reservations reducing SCSI reservations 111 resignaturing options 118 resolving problems 105 resource utilization, optimizing 46 restrictions 50 resxtop 108 Round Robin path policy 98
S
SAN 36 arrays 31 backup considerations 114 hardware failover 62 installation considerations 53 preparing 75 preparing to install ESX Server 73 requirements 50 server failover 45
132
SAN fabric 26 SAN storage performance, optimizing 106 SAN storage, benefits 28 scanning, changing number 88 SCSI controllers, device driver options 103 SCSI reservations 31 SCSI reservations, reducing 111 SDK 20 selectable boot, enabling 77 server failover 45 server failure 44 server performance 107 service console 14, 21, 72, 104, 108 authentication 21 interfaces 21 setup steps 54 sharing diagnostic partitions 104 sharing VMFS across servers 31 size of zones 36 snapshot 118 snapshot software 115 SNMP server 22 software compatibility 15 solutions 82 SP visibility, LUN not visible 87 sparse LUN support 90 standby path status 96 storage arrays performance 106 storage choices 36 storage devices details 84 viewing 84 storage processors configuring sense data 65 port configuration 63 sense data 65
VMware, Inc.
Index
storage systems EMC CLARiiON 60 EMC Symmetrix 61 Hitachi 69 HP StorageWorks 66 Network Appliance 69 storageMonitor utility 124 supported devices 59
T
tape devices 52 testing 58 third-party backup package 115 third-party management applications 35 timeout 103, 104 TimeoutValue parameter 50 tips 38 troubleshooting 82, 105
U
use cases 29 utilities esxtop 124 storageMonitor 124
V
VI Client 14, 20 VI Web Access 14, 20 Virtual Infrastructure SDK 20 Virtual Machine File System 19 Virtual Machine Monitor 14 virtual machines assigning WWNs to 92 default configuration 33 equalizing disk access 110 I/O delay 43 locations 43 prioritizing 38 sharing devices 22
virtual ports (VPORTs) 91 virtual SCSI HBAs 17, 20 virtualization 16 at a glance 22 visibility issues 85 vmauthd 22 VMFS 19, 34, 36 1 volume per LUN 50 creating new volume 19 locking 19 minimum size 19 number of extents 19 SCSI reservations 31 sharing across ESX Servers 31 VMFS volume resignaturing 117 VMFS-2 drivers 111 vmhba 85 VMkernel 14 configuration 104 LUN discovery 33 VMM 14 VMotion 29, 46, 51 VMware Consolidated Backup proxy 50 VMware DRS 29, 47 VMware HA 29, 44 VMware Infrastructure Client 14 volume resignaturing 117
W
World Wide Names (WWNs) 91 assigning to virtual machines 92 World Wide Node Names (WWNNs) 91 World Wide Port Names (WWPNs) 27, 64, 91
X
XP (HP StorageWorks) 69
VMware, Inc.
133
Z
zones failover 36 recommended size 36 when to create 36 zoning and ESX Server 35 LUN not visible 86
134
VMware, Inc.
Updates for the Fibre Channel SAN Configuration Guide
LastUpdated:May14,2010 ThisdocumentprovidesupdatestotheESXServer3.5,ESXServer3iversion3.5,and VirtualCenter2.5versionoftheFibre Channel SAN Configuration Guide.Updated descriptionsareorganizedbypagenumbersoyoucaneasilylocatetheareasofthe guidethathavechanges. ThefollowingisalistofupdatestotheFibre Channel SAN Configuration Guide:

UpdatesfortheHPStorageWorksEVASectiononPage 68 UpdatestotheIssuesandSolutionsSectiononPage 82 UpdatesfortheRemovingDatastoresSectiononPage 88 UpdatesfortheTochangethevalueofDisk.MaskLUNsProcedureonPage 89 UpdatesfortheChangingSparseLUNSupportUsingDisk.SupportSparseLUN SectiononPage 90 UpdatesfortheMultipathingSetupRequirementsTableonPage 122
Updates for the HP StorageWorks EVA Section on Page 68

TheHPStorageWorksEVAsectioncontainsreferencestotheHPEVA3000/5000storage devices,whicharenotsupportedbyESX3.5. Thefollowingbulletsshouldberemovedfromthissection:
ForHPEVAgl3000/5000(active/passive),usethehostmodetype 000000002200282E. ForHPEVAglfirmware4.001(active/activefirmwareforGLseries)andabove,use thehostmodetypeVMware.
VMware, Inc.
Update1
Updates to the Issues and Solutions Section on Page 82

Table 61inIssuesandSolutionsshouldlistthefollowingadditionalissue:
Issue Astorageadministrator removesorreplacesaLUNthat yourESXServerhasaccessto. Solution IfthestorageadministratorremovesaLUNthatisnot activelyusedbyyourESXServersystemandthenlater createsanewLUNwiththesameLUNnumber,yourESX ServersystemcanaccessthenewLUNandformatitwitha VMFSdatastore.However,thesystemconsidersthenew LUNasnapshotandcannotmounttheVMFSdatastore deployedonthisLUN.Toresolvethisissue,enable resignaturing.SeeTomountoriginal,snapshot,orreplica VMFSvolumesonpage 118. IfyourESXServersystemusestheremovedLUN,the behavioroftheESX Serversystemanditsvirtualmachines isunpredictable,andtheESXServerlogfilereportsacritical error.Thereisnosolutioninthiscase.
Updates for the Removing Datastores Section on Page 88

TheRemovingDatastoressectionshouldcontainthefollowingcontent: WhenyouremoveadatastorefromtheESXhostbyusingtheVIClient,thedatastore isdeletedpermanentlyfromtheESXhost. To remove a datastore from an ESX host without deleting it permanently 1 2 3 Poweroffthevirtualmachinesinthedatastoreormigratethevirtualmachinesto anotherhostmachinebyusingvMotion. UnpresenttheLUNstothisESXhost. RescantheESXhost.
Updates for the To change the value of Disk.MaskLUNs Procedure on Page 89

AfteryouperformTochangethevalueofDisk.MaskLUNs,rescanstorageadapters.See UsingRescanonpage 87.
Update2
VMware, Inc.
Updates for the Changing Sparse LUN Support Using Disk.SupportSparseLUN Section on Page 90
IntheChangingSparseLUNSupportUsingDisk.SupportSparseLUNsection,the nameoftheDisk.SupportSparseLUNparameterisspelledincorrectlyas Disk.SupportSparesLUN.Thesentenceshouldreadasfollows: IfalltheLUNsarepresentintherange,youcandisabletheDisk.SupportSparseLUN parameter.
Updates for the Multipathing Setup Requirements Table on Page 122

TheMultipathingSetupRequirementstablecontainsreferencestotheHP EVA3000/5000storagedevices,whicharenotsupportedbyESX3.5.TheESXServer Configurationcomponentdetailsarealsoupdated. ThefollowingcellsreplacetheinformationgivenforHPEVA,andprovideupdatesfor theESXServerConfiguration:
HPEVA ForEVA4000/6000/8000firmware5.031andlater,setthehosttypeto VMware.Otherwise,setthehosttypetoCustom.Thevalueis: EVA4000/6000/8000:000000202200083E ESXServer Configuration ThefollowingAdvancedSettingsfortheESXServerhostare enabled,bydefault:

Disk.UseLunReset Disk.UseDeviceReset
AmultipathingpolicyofMostRecentlyUsedmustbesetforall LUNshostingclustereddisksforactivepassivearrays.A multipathingpolicyofMostRecentlyUsedorFixedmaybesetfor LUNsonactiveactivearrays. AllFCHBAsmustbeofthesamemodel.
VMware, Inc.
Update3
Update4
VMware, Inc.

Fibre Channel SAN Configure Guide

Hochgeladen von

Dokumentinformationen

Originalbeschreibung:

Copyright

Verfügbare Formate

Dieses Dokument teilen

Dokument teilen oder einbetten

Freigabeoptionen

Stufen Sie dieses Dokument als nützlich ein?

Sind diese Inhalte unangemessen?

Copyright:

Verfügbare Formate

Fibre Channel SAN Configure Guide

Hochgeladen von

Copyright:

Verfügbare Formate

Fibre Channel SAN Configuration Guide

ESX Server 3.5, ESX Server 3i version 3.5 VirtualCenter 2.5

Fibre Channel SAN Configuration Guide

VMware, Inc. 3401 Hillview Ave. Palo Alto, CA 94304 www.vmware.com

Fibre Channel SAN Configuration Guide

Fibre Channel SAN Configuration Guide

A MultipathingChecklist 121 B Utilities 123

Fibre Channel SAN Configuration Guide

About This Book

Thismanual,theFibreChannelSANConfigurationGuide,explainshowtousea VMwareESXServersystemwithastorageareanetwork(SAN).Themanualdiscusses conceptualbackground,installationrequirements,andmanagementinformationinthe followingmaintopics:

FortopicsspecifictoESXServer3.5,thisbookusesthetermESXServer3. FortopicsspecifictoESXServer3iversion3.5,thisbookusestheterm ESX Server 3i. Fortopicscommontobothproducts,thisbookusesthetermESXServer.

Fibre Channel SAN Configuration Guide

Whentheidentificationofaspecificreleaseisimportanttoadiscussion,thisbook referstotheproductbyitsfull,versionedname. WhenadiscussionappliestoallversionsofESXServerforVMwareInfrastructure 3,thisbookusesthetermESXServer3.x.

VMware Infrastructure Documentation

Abbreviations Used in Figures

About This Book

Technical Support and Education Resources

Online and Telephone Support

VMware Education Services

Fibre Channel SAN Configuration Guide

Overview of VMware ESX Server

IntroductiontoESXServeronpage 14 UnderstandingVirtualizationonpage 16 InteractingwithESXServerSystemsonpage 20 VirtualizationataGlanceonpage 22

ForindepthinformationonVMwareESXServer,includingdocumentation,hardware compatibilitylists,whitepapers,andmore,gototheVMwareWebsiteat http://www.vmware.com.

Fibre Channel SAN Configuration Guide

Introduction to ESX Server

WithESXServer3,theserviceconsolecommandlineinterface.See AppendixAintheESXServer3ConfigurationGuide. WithESXServer3i,theremotecommandlineinterfaces(RCLIs).See AppendixAintheESXServer3iConfigurationGuide.

Chapter 1 Overview of VMware ESX Server

Figure 11showshowthecomponentsinteract.TheESXServerhosthasfourvirtual machinesconfigured.Eachvirtualmachinerunsitsownguestoperatingsystemand applications.Administratorsmonitorthehostandthevirtualmachinesinthe followingways:

UsingaVIClienttoconnecttoanESXServerhostdirectly. UsingaVIClienttoconnecttoaVirtualCenterManagementServer.The VirtualCenterServercanmanageanumberofESXServerhosts.

Figure 1-1. Virtual Infrastructure Environment

Software and Hardware Compatibility

Fibre Channel SAN Configuration Guide

Mostapplicationsinteractonlywiththeguestoperatingsystem,notwiththe underlyinghardware.Asaresult,youcanrunapplicationsonthehardwareofyour choiceaslongasyouinstallavirtualmachinewiththeoperatingsystemthatthe applicationrequires.

CPU, Memory, and Network Virtualization

Chapter 1 Overview of VMware ESX Server

Fibre Channel SAN Configuration Guide

Figure 12givesanoverviewofstoragevirtualization.ItillustratesstorageusingVMFS andstorageusingrawdevicemapping(RDM). Figure 1-2. SAN Storage Virtualization

SCSI controller (Buslogic or LSI Logic) virtual disk 1 virtual disk 2

VMware virtualization layer HBA

VMFS LUN1 .vmdk LUN2 RDM LUN5

Disk Configuration Options

Virtualmachine.vmdkfilestoredonaVMFSvolume.SeeVirtualMachineFile Systemonpage 19. DevicemappingtoaSANLUN(logicalunitnumber).SeeRawDeviceMapping onpage 19.

Chapter 1 Overview of VMware ESX Server

Fromthestandpointofthevirtualmachine,eachvirtualdiskappearsasifitwerea SCSIdriveconnectedtoaSCSIadapter.Whethertheactualphysicaldiskdeviceis beingaccessedthroughSCSI,iSCSI,RAID,NFS,orFibreChannelcontrollersis transparenttotheguestoperatingsystemandtoapplicationsrunningonthevirtual machine.

Virtual Machine File System

Raw Device Mapping

Fibre Channel SAN Configuration Guide

Virtual SCSI Host Bus Adapters

Interacting with ESX Server Systems

VMware Virtual Center

Chapter 1 Overview of VMware ESX Server

ESX Server 3 Service Console

Fibre Channel SAN Configuration Guide

TheserviceconsoleisimplementedusingamodifiedLinuxdistribution.However,the serviceconsoledoesnotcorresponddirectlytoaLinuxcommandprompt. ThefollowingESXServer3managementprocessesandservicesrunintheservice console:

OneCPU Anallocationofmemoryandanetworkadapter(NIC) Twovirtualdisks

Chapter 1 Overview of VMware ESX Server

Figure 1-4. Virtual Machines Sharing Physical Resources

CPU memory physical resources

ThevirtualmachineseachuseoneoftheCPUsontheserverandaccessnoncontiguous pagesofmemory,withpartofthememoryofonevirtualmachinecurrentlyswapped todisk(notshown).Thetwovirtualnetworkadaptersareconnectedtotwophysical networkadapters. Thedisksaremappedasfollows:

Disk1ofvirtualmachine1ismappeddirectlytoarawdisk.Thisconfigurationcan beadvantageousundercertaincircumstances. Disk2ofvirtualmachine1andbothdisksofvirtualmachine2resideontheVMFS, whichislocatedonaSANstoragearray.VMFSmakessurethatappropriate lockingandsecurityisinplaceatalltimes.

Fibre Channel SAN Configuration Guide

Using ESX Server with Fibre Channel SAN

WhenyousetupESXServerhoststouseFCSANarraystorage,specialconsiderations arenecessary.Thischapterprovidesintroductoryinformationabouthowtouse ESX ServerwithaSANarrayanddiscussesthesetopics: