White Paper

NetApp Host Multipath Strategy

Amaresh T, Vijay Rao, NetApp May 2011 | WP-7135

ABSTRACT
In todays business environment, high availability is not just a benefit but a necessity. Multipath I/O is an important high-availability solution and an important component of SAN infrastructures. This paper discusses various distribution models of multipathing software and the host multipath strategy of NetApp.

TABLE OF CONTENTS 1 2 3 4 OVERVIEW ............................................................................................................................................ 3 HIGH AVAILABILITY ............................................................................................................................. 3 MULTIPATHING ..................................................................................................................................... 4 MULTIPATHING DISTRIBUTION MODEL ............................................................................................ 5
THIRD-PARTY MULTIPATHING SOLUTION ........................................................................................................................ 5 NATIVE MULTIPATHING SOLUTION ................................................................................................................................... 5

5 6 7 8 9

NETAPP HOST MULTIPATH STRATEGY............................................................................................ 7 SUMMARY OF NETAPP HOST MULTIPATH I/O SUPPORT .............................................................. 9 FEATURE COMPARISON OF SUPPORTED MULTIPATHING SOLUTIONS ..................................... 9 NETAPP HOST UTILITIES .................................................................................................................. 10 CONCLUSION ...................................................................................................................................... 11

NetApp Host Multipath Strategy

1 OVERVIEW
In todays business environment, high availability is a necessity. A growing number of organizations require that their data be available 24x7, all year round. To meet this requirement, centralized storage must be readily available and immune to outages at all times. Enterprise environments require 99.999% uptime for all key workloads, including file server, database, messaging, and other line of business applications. This level of availability can be difficult and costly to achieve as it requires redundancy to be built in at multiple levels: storage redundancy, backups to separate recovery servers, server clustering, and redundancy of the physical path components. This document provides an overview of the different methods of providing high availability with a primary focus on multipathing. This paper is intended for system and storage architects designing SAN infrastructure solutions using NetApp storage devices.

2 HIGH AVAILABILITY
Some techniques for keeping business-critical computer applications and digital data available to users despite hardware and software failures include: Application availability through server clustering. Clustering is the use of multiple servers, host bus adapters (HBAs), and storage devices that work together to provide users with high application availability. If a server experiences a hardware failure or is temporarily unavailable, end users are still able to transparently access data or applications on a redundant cluster node. Clustering can also be used as a tool to reduce the downtime required for patch management and hardware maintenance activities. Availability through RAID. High-availability solutions based on redundant array of independent disks (RAID) technology have been in use for several decades. While RAID 1 (mirroring) provides excellent fault tolerance, it only protects the disks. If there is only a single path from the server to the storage device, however, and one of the components in that path fails, no amount of disk redundancy can provide data access and availability.

Availability through multipathing. Multipathing is redundancy of the storage network components onlythe cabling, adapters, and switchesthat transfer data across the path between the server and storage. Multipathing solutions manage these redundant connections so that read/write requests can be instantaneously rerouted in case a path fails. Administrators have to deploy a good mix of these high-availability solutions for 24x7 availability. This white paper will focus on high availability using multipathing.

NetApp Host Multipath Strategy

3 MULTIPATHING
Multipathing solutions provide the host-side logic to transparently utilize the multiple paths of a redundant network to provide highly available and higher bandwidth connectivity between hosts and block level devices. The main objectives of multipath software are: Present the OS with a single virtualized path to the OS. The following figure includes two scenarios: OS with no multipath management software and OS with multipath management software. Without multipath management software, the OS believes it is connected to two different physical storage devices. With multipath management software, the OS correctly interprets that both HBAs are connected to the same storage device.

Figure 1) LUN visibility, with and without Multipath.

Server

Server

A1

A2

A1

A2

Without multipathing software, the host incorrectly interprets the two paths as leading to two storage units

With multipathing software, the server correctly interprets the two paths as leading to the same storage unit

Seamlessly recover from a path failure. Multipath software detects failed paths and recovers from the failure by routing traffic through another available path. The recovery is automatic, usually fast, and completely transparent to the IT organization. The data ideally remains available at all times. Enable load balancing. Load balancing is the use of multiple data paths between server and storage to provide greater throughput of data than could be achieved with only one connection. Multipathing software improves throughput by enabling load balancing across multiple paths between server and storage.

NetApp Host Multipath Strategy

4 MULTIPATHING DISTRIBUTION MODEL

Multipathing solutions are provided by: Third-party vendors: Storage vendors as support for their own storage arrays such as the NetApp Data ONTAP DSM for Windows. These solutions are generally specific to the particular vendors equipment. Independent third-party vendors offering heterogeneous host and storage support (Symantec and Veritas DMP).

Operating system vendors as part of the operating system (Windows MSDSM, Solaris MPxIO, AIX MPIO, Linux Device-Mapper Multipath, HP-UX PVLinks, VMware ESX Server NMP).

THIRD-PARTY MULTIPATHING SOLUTION

As mentioned previously, third-party multipathing solutions are provided either by storage vendors or by independent software vendors such as Symantec. The advantage of using multipathing solutions provided by storage vendors is that it provides a unified management interface for all operating systems; this makes life easier for the administrator in a heterogeneous host environment. Secondly, storage vendors know their array the best, and hence the multipathing solution provided by the storage array vendor can provide optimal performance. Conversely, the multipathing solutions provided by storage vendors have the following disadvantages: Most of these solutions come with fee-based software licenses and typically require ongoing license maintenance costs. The solutions provided by storage vendors lock the customer into a single storage platform. Some of these solutions do have support for other storage arrays, but there can be long qualification/support delays.

These solutions usually do not interoperate well with multipathing solutions from other storage vendors that must be installed on the same server. Multipathing solutions provided by Symantec also require fee-based software licenses. However, these solutions provide support for heterogeneous storage as well as heterogeneous host OS.

NATIVE MULTIPATHING SOLUTION

Native multipathing solutions are packaged as part of the operating system. As of the publication date of this document, Windows, ESX, Linux, HP-UX, Solaris, and AIX provide native multipathing solutions. The advantages of native multipathing solutions are: Native multipathing solutions are free of cost. Native multipathing reduces capital expense (can limit the number of redundant servers) and operating expense. The availability of multipath support in the server operating systems allows IT installations to adopt a more sensible server-led strategy that is independent of the storage array vendors. It doesnt lock a customer to a single storage array and provides freedom of choice and flexibility when selecting a storage vendor. Provides better interoperability among various vendor storage devices that connect to the same server(s). One driver stack and one set of HBAs can communicate with various heterogeneous storage devices simultaneously.

NetApp Host Multipath Strategy

With the advent of SCSI concepts such as asymmetric logical unit access (ALUA), native multipathing solutions have improved in functionality. For example, Microsoft provided Fibre Channel multipathing support only after ALUA became available for Windows. One disadvantage associated with native multipathing solutions is that there is no common management interface across the various operating systems. This limitation would be challenging in a heterogeneous host environment. To counter this, the Storage Networking Industry Association (SNIA) has come up with a standard interface for multipath management: the SNIA Multipath Management API (MMA). It opens up the possibility of a standard way of managing multipath facilities in a heterogeneous environment, both OS, as well as multipath software. Sun Solaris has already implemented SNIA MMA. Native multipathing was introduced by HP in 1995 with the release of PV-Links. This was followed by other UNIX variants, Solaris, and AIX. Linux released its native multipath solution Device-Mapper Multipath in August 2005. Microsoft provided an MPIO framework in 2003 and a full-fledged multipathing solution in 2008. Even though the first native multipathing solution was released in 1995, multipathing did not gain popularity and acceptance until early 2000. Prior to ALUA, there were major limitations in native multipathing. Windows did not have native multipathing solution for FC, and ESX native multipath would route I/O through unoptimized paths even if optimized paths were available, and so on. With the advent of ALUA, these problems were solved, and native multipathing solutions are much more functional and stable. As of today all the native multipath solutions have ALUA support.

Asymmetric Logical Unit Access Asymmetric logical unit access (ALUA) is an industry standard protocol that enables the communication of storage paths and path characteristics between an initiator port and a target port. ALUA occurs when the access characteristics of one port might differ from those of another port. A logical unit may be accessed from more than one target port. One target port might provide full performance access to a logical unit, while another target port, possibly on a different physical controller, either might provide lower performance access or might support a subset of the available SCSI commands to the same logical unit. In the following figure, we have an asymmetric array HA pair. There is a LUN mapped to the host from controller 1 with four paths to it. The two paths through controller 1 (controller 1, port 0a and controller 1, port 0b) marked in bold are optimized, whereas the two paths through controller 2 (controller 2, port 0a and controller 2, port 0b) marked in dotted lines are unoptimized paths. The

NetApp Host Multipath Strategy

multipathing layer should then use optimized paths as the primary paths and unoptimized paths as secondary (failover) paths.
Figure 2) HA configuration.

Prior to inclusion of ALUA in SCSI standards, multipath providers had to use vendor-specific SCSI commands to figure out the access characteristics of a target port. With the inclusion of ALUA in SCSI standards, the multipath vendor could use standard SCSI commands to determine the access characteristics. ALUA was implemented by NetApp in Data ONTAP 7.2. With iSCSI in NetApp controllers, there is no proxy path, and because link failover operates differently from Fibre Channel, ALUA is not supported on iSCSI connections.

5 NETAPP HOST MULTIPATH STRATEGY

NetApps strategy has been to enable cost-effective, reliable choice through native multipathing. NetApp provides a proprietary multipathing solution only on Microsoft Windows. NetApps primary multipathing approach is to support all OS native multipathing solutions, as well as support some third-party (for example, Symantec and VxDMP) solutions across all supported operating systems. Native multipathing solutions provide good functionality, reduce cost, and improve reliability. Native multipathing is a perfect choice for heterogeneous storage environments. Of late, native multipathing technologies in operating systems such as Windows, Linux, ESX, and HP-UX have advanced in function and stability. Since they are provided free of cost, they reduce the expenses associated with license and support. NetApp partners with Symantec (Veritas) for heterogeneous host environments where a unified management interface would be beneficial. Symantec (Veritas) provides a multipathing solution

NetApp Host Multipath Strategy

named Veritas DMP. Veritas DMP is supported on Solaris, Windows, Linux, AIX, and HPUX, and it provides identical functionality and user interfaces on all supported host platforms.

NetApp Host Multipath Strategy

6 SUMMARY OF NETAPP HOST MULTIPATH I/O SUPPORT

The following table lists the various multipathing solutions supported by NetApp in different operating systems. Operating System Native Multipathing Veritas DMP Multipathing Provided by NetApp

Windows

MSDSM

DMP DSM

Data ONTAP DSM

ESX4, PSA (vendor specific); native (NMP with SATP and PSP)

VMware ESX
Solaris 9 (MPXIO) Solaris 10 (STMS storage traffic manager system) MPIO

N/A

N/A

Solaris AIX

VXDMP VXDMP

N/A N/A

Red Hat, Oracle , LInux SUSE Linux HP-UX

DEVICE-MAPPER MULTIPATH

VXDMP

N/A

DEVICE-MAPPER MULTIPATH PV-LINKS (V1, V2, and V3) native MPIO (V3)

VXDMP

N/A

VXDMP

N/A

7 FEATURE COMPARISON OF SUPPORTED MULTIPATHING SOLUTIONS

The following table lists the features available in native multipathing solutions provided by different OS vendors and supported with NetApp arrays (check marks indicate supported, and Xs indicate not supported. For more information on what is supported, see the NetApp IMT: http://now.netapp.com/matrix. Feature Windows MPIO ESX Solaris MPIO Linux DM-MP AIX MPIO HP-UX MPIO Data ONTAP DSM Veritas

End-to-end channel failover Automatic path recovery Load balance algorithms

Dynamic least queue depth, round

NMP with SATP support for round robin,

Round robin

Round robin

Round robin

Round robin (+ cell, weighted),

Round robin, least queue depth,

Adaptive, adaptive min queue, balanced,

NetApp Host Multipath Strategy

robin, round robin with subset

MRU, and fixed, PSA vendor plug-in for additional algorithms

least command, preferred path

autoassigned, failover, round robin with subset, least weighted paths

single active, round robin, minimum queue

Load balance across alternate paths after failover Ability to disable load balancing Storageindependent management Serverindependent management

Native OS

Server operating environments supported

Native OS (April 2003)

Native OS

Native OS (February 2000)

Native OS (August 2005)

Native OS (October 2002)

PVLInks (1995) MPIO (CQ4 2007)

Windows

Windows, Linux, HPUX, Solaris, AIX

Cluster support Mixed mode (iSCSI and FC paths to the same LUN)

8 NETAPP HOST UTILITIES

NetApp provides a SAN host utilities kit for every supported OS. This is a set of data collection applications, configuration scripts, and documentation. On some operating systems such as Microsoft Windows and VMware ESX, the host utilities alter the SCSI and path timeout values and HBA parameters. These timeouts are modified to make sure of the best performance and to successfully handle storage system events. Host utilities make sure that hosts correctly handle the behavior of the NetApp storage system. On other operating systems such as those based on Linux and UNIX, timeout parameters will need to be modified manually, the details of which will be provided in the Host Utilities Setup Guide located on http://now.netapp.com. Host utilities also include a tool called sanlun, which is a host-based utility that helps configuring NetApp controllers and LUNs as simply as possible. Sanlun bridges the namespace between host and storage controller, collecting and reporting storage controller LUN information, and correlating it with the host

10

NetApp Host Multipath Strategy

device filename or equivalent entity. The sanlun tool assists with debugging SAN configuration issues. The sanlun utility is available in all operating systems except Windows.

9 CONCLUSION
In enterprise data centers, 24x7 data availability is a necessity. Multipathing is a high-availability solution that improves fault tolerance and helps improve throughput by using multiple paths from server to storage. Multipathing solutions are provided by either third-party vendors (array vendors for their arrays/independent solutions providers) or host operating system vendors, also known as the native multipathing solution. For many OSes, NetApp believes the native multipathing solution is robust enough and has not developed a proprietary solution. The native multipathing solution is provided as part of the OS and is free of cost. Over a period of time, native multipathing solutions have improved in functionality and stability with the advent of ALUA. For deployment in heterogeneous host environments, where native multipathing does not have a unified management interface, NetApp partners with Symantec to provide support for multipathing through the use of the Veritas DMP product. Symantec multipathing provides identical functionality and user interfaces on all supported host platforms.

NetApp provides no representations or warranties regarding the accuracy, reliability or serviceability of any information or recommendations provided in this publication, or with respect to any results that may be obtained by the use of the information or observance of any recommendations provided herein. The information in this document is distributed AS IS, and the use of this information or the implementation of any recommendations or techniques herein is a customers responsibility and depends on the customers ability to evaluate and integrate them into the customers operational environment. This document and the information contained herein may be used solely in connection with the NetApp products discussed in this document.

11

2011 NetApp, Inc. All rights reserved. No portions of this document may be reproduced without prior written consent of NetApp, Inc. Specifications are subject to change without notice. NetApp, the NetApp logo, Go further, faster, and Data ONTAP are trademarks or registered trademarks of NetApp, Inc. in the United States and/or other countries. Microsoft and Windows are registered trademarks of Microsoft Corporation. Linux is a registered trademark of Linus Torvalds. Oracle is a registered trademark of Oracle Corporation. Symantec and Veritas are trademarks of Symantec Corporation. VMware is a registered trademark of VMware, Inc. UNIX is a registered trademark of The Open Group. All other brands or products are NetApp Host Multipath Strategy registered trademarks of their respective holders and should be treated as such.WP-7135-0511 trademarks or