GPFS for AIX 5L Questions and Answers

Overview
The General Parallel File System (GPFS) for AIX 5L is a high-performance shared-disk file system that can provide data access from nodes in an AIX cluster environment. Parallel and serial applications can readily access shared files using standard UNIX file system interfaces, and the same file can be accessed concurrently from multiple nodes. GPFS is designed to provide high availability through logging and replication, and can be configured for failover from both disk and server malfunctions. GPFS has been widely accepted on the IBM RS/6000 SP system to scale file system I/O which can help meet the requirements of a wide range of applications including seismic data processing, digital library file serving, and data mining in business intelligence. GPFS for AIX 5L supports the IBM ERserver Cluster 1600 as well as clusters of IBM Eserver pSeries nodes or IBM Eserver p5 nodes. | It is suggested you always check GPFS common FAQ question What is the current service information for | GPFS? Updates to this FAQ include:
Date October 2004 Question What disk support guidelines must be followed when running GPFS in an sp cluster type? What disk support guidelines must be followed when running GPFS in an rpd cluster type? What are the disk support guidelines that must be followed when running GPFS in an hacmp cluster type? What disks support hardware fencing? What versions of GPFS support EtherChannel? IBM TotalStorage FAStT has been renamed IBM TotalStorage DS4000 series DS4100 formerly FAStT100 DS4300 formerly FAStT600 DS4300 Turbo formerly FAStT600 Turbo DS4400 formerly FAStT700 DS4500 formerly FAStT900

Questions & Answers

1. 2. 3. 4. 5. What is an sp cluster type? When running GPFS in an sp cluster type, what are the software requirements? What disk support guidelines must be followed when running GPFS in an sp cluster type? What is an rpd cluster type? When running GPFS in an rpd cluster type, what are the software requirements?

6. What disk support guidelines must be followed when running GPFS in an rpd cluster type? 7. What is an hacmp cluster type? 8. When running GPFS in an hacmp cluster type, what are the software requirements? 9. What are the disk support guidelines that must be followed when running GPFS in an hacmp cluster type? How can I implement disaster recovery for my GPFS cluster? What are the coexistence rules for GPFS clusters? What type of disk fencing mechanisms does GPFS support? How can the fencing technique in operation for a cluster be verified? What is AIX MPIO and does GPFS support it? What disks support hardware fencing? What should I do if commands such as mmcrlv, mmcrfs, mmadddisk, mmrpldisk, or mount fail to complete due to problems with non-IBM disks? 17. What cluster specific single-node quorum restrictions are there? 18. Why would I want to use EtherChannel in my GPFS cluster? 10. 11. 12. 13. 14. 15. 16. 19. 20. 21. 22. 23. 24. 25. 26. 27. 28. What versions of GPFS support EtherChannel? Whats the difference between using an ESS with or without SDD or SDDPCM installed on the host? Is a Fibre Channel Switch required for all Fibre Channel disk attachments which are used by GPFS? What Fibre Channel Switches are qualified for GPFS usage and is there a FC Switch support chart available? What performance tuning suggestions are there for AIX nodes? What configuration and performance tuning suggestions are there for GPFS when used primarily for Oracle databases? What is the largest file system size supported by GPFS? Are there any special concerns regarding the use of mksysb images? Are there any considerations when using NIS automount? How can I ask a more specific question about GPFS?

Q1: A1:

What is an sp cluster type? The GPFS cluster type sp is based on the IBM Parallel System Support Programs (PSSP) licensed product and the shared disk concept of the IBM Virtual Shared Disk component of PSSP. In the GPFS cluster type sp (a PSSP environment), the nodes which are members of the GPFS cluster depend on the network switch type being used. In a system with an SP Switch, the GPFS cluster is equal to all of the nodes in the corresponding SP partition which have GPFS installed. In a system with an SP Switch2, the GPFS cluster is equal to all of the nodes in the system which have GPFS installed. That is, the cluster definition is implicit and there is no need to run the GPFS cluster commands. Within the GPFS cluster, you define one or more nodesets within which your file systems operate. Disk requirements within your GPFS sp cluster require that all disks must be: 1. Attached to a virtual shared disk server within the nodeset. 2. Processed via either the mmcrvsd command or the directions in the Parallel System Support Programs for AIX: Managing Shared Disks book to create virtual shared disk useable by GPFS. 3. Conform to the disk support guidelines listed in this FAQ. When running GPFS in an sp cluster type, what are the software requirements? In an sp cluster type, GPFS requires the Parallel System Support Programs (PSSP) licensed product and its IBM Virtual Shared Disk and IBM Recoverable Virtual Shared disk components for uniform disk access and recovery.
AIX Level AIX 5L v5.2 AIX 5L v5.1 or AIX 5L v5.2 PSSP Level PSSP v3.5 PSSP v3.5 30April2006 GPFS End of Service

Q2: A2:

Table 1. Supported software levels for an sp cluster type GPFS Version GPFS v2.2.1 GPFS v2.1

Note: Ensure you are at the current service level for each software level listed. Q3: A3: What disk support guidelines must be followed when running GPFS in an sp cluster type? In an sp cluster type, a typical GPFS cluster configuration uses recoverable virtual shared disks (RVSD). Deviations from the typical case are GPFS clusters which use concurrent virtual shared disks (CVSD). Use of CVSD may have an impact on the types of disks supported. This distinction is noted in the Cluster Properties section in Table 2 on page 4, Table 3 on page 5, Table 4 on page 5, and Table 5 on page 6. There are no specific disk limitations in a two-node sp cluster running single-node quorum (see the question What considerations are there for two node nodesets?). For additional disk configuration information and examples, please see ftp.software.ibm.com/storage/fastt/fastt500/PSSP-GPFS_config_info.pdf

Table 2. GPFS v2.2.1 and PSSP v3.5 running AIX 5L v5.2 Cluster Properties: Typical GPFS cluster configuration IBM TotalStorage DS4100 with Serial Advanced Technology Attachment (SATA) disk drives (SM8.42 microcode and APAR IY62752) IBM TotalStorage FAStT500 (SM7.10, SM8.21, and SM8.3 microcode)

| |

Disks Supported:

| | | | | | | | | |

IBM DS4300 (SM8.33 microcode) IBM DS4300 Turbo (SM8.40 microcode) IBM DS4300 Turbo with IBM TotalStorage DS4000 EXP100 Storage Expansion Unit (1742-60U and 1742-60X ) with SATA disk drives (SM8.42 microcode and APAR IY62752) IBM DS4400 (SM8.21and SM8.3 microcode) IBM DS4500 (SM8.3 microcode) IBM DS4500 with IBM TotalStorage DS4000 EXP100 Storage Expansion Unit (1742-60U and 1742-60X ) with SATA disk drives (SM8.42 microcode and APAR IY62752) IBM TotalStorage Enterprise Storage Server (ESS) (2105-F20 or 2105-800 with Subsystem Device Driver (SDD) - disk leasing mode) IBM TotalStorage ESS (2105-F20 or 2105-800 using AIX Multiple Path I/O (MPIO) and Subsystem Device Driver Path Control Module (SDDPCM) - persistent reserve mode) IBM SAN Volume Controller (APARs IY47267 and IY42355) EMC Symmetrix Direct Matrix Architecture (DMX) Storage Subsystems (FC attach only) IBM Serial Storage Architecture (SSA) (all disk sizes) Hitachi Lightning 9900(9910, 9960, 9970V and 9980V (Hitachi Dynamic Link Manager) (HDLM) required) Restrictions: Fibre Channel disks must be FC switch attached. No disk can be larger than 1 TB.

Table 3. GPFS v2.2.1 and PSSP v3.5 running AIX 5L v5.2 Cluster Properties: CVSD required IBM TotalStorage DS4100 with Serial Advanced Technology Attachment (SATA) disk drives (SM8.42 microcode and APAR IY62752) IBM DS4300 Turbo with IBM TotalStorage DS4000 EXP100 Storage Expansion Unit (1742-60U and 1742-60X ) with SATA disk drives (SM8.42 microcode and APAR IY62752) IBM DS4500 with IBM TotalStorage DS4000 EXP100 Storage Expansion Unit (1742-60U and 1742-60X ) with SATA disk drives (SM8.42 microcode and APAR IY62752) IBM TotalStorage Enterprise Storage Server (ESS) (2105-F20 or 2105-800 using AIX MPIO and SDDPCM - persistent reserve mode) IBM SAN Volume Controller (APARs IY47267 and IY42355) EMC Symmetrix DMX Storage Subsystems (FC attach only) IBM SSA (all disk sizes) Restrictions: Fibre Channel disks must be FC switch attached. No disk can be larger than 1 TB. Table 4. GPFS v2.1 and PSSP v3.5 running AIX 5L v5.1 Cluster Properties: Typical GPFS cluster configuration IBM DS4100 with SATA disk drives (SM8.42 microcode and APAR IY61688) IBM FAStT500 (SM7.10, SM8.21, and SM8.3 microcode)

| | | | | | | |

Disks Supported:

Disks Supported:

| | | | | | | | | |

IBM DS4300 (SM8.33 microcode) IBM DS4300 Turbo (SM8.40 microcode) IBM DS4300 Turbo with IBM TotalStorage DS4000 EXP100 Storage Expansion Unit (1742-60U and 1742-60X ) with SATA disk drives (SM8.42 microcode and APAR IY61688) IBM DS4400 (SM8.21and SM8.3 microcode) IBM DS4500 (SM8.3 microcode) IBM DS4500 with IBM TotalStorage DS4000 EXP100 Storage Expansion Unit (1742-60U and 1742-60X ) with SATA disk drives (SM8.42 microcode and APAR IY61688) IBM TotalStorage ESS (2105-F20 or 2105-800 with SDD) IBM TotalStorage ESS (2105-F20 or 2105-800 without SDD - persistent reserve mode) IBM SAN Volume Controller (APARs IY47267 and IY42355) EMC Symmetrix DMX Storage Subsystems (FC attach only) IBM SSA (all disk sizes) Hitachi Lightning 9900 (9910, 9960, 9970V and 9980V ) (HDLM required) Restrictions: Fibre Channel disks must be FC switch attached. No disk can be larger than 1 TB.

Table 5. GPFS v2.1 and PSSP v3.5 running AIX 5L v5.1 Cluster Properties: CVSD required IBM TotalStorage DS4100 with Serial Advanced Technology Attachment (SATA) disk drives (SM8.42 microcode and APAR IY62752) IBM DS4300 Turbo with IBM TotalStorage DS4000 EXP100 Storage Expansion Unit (1742-60U and 1742-60X ) with SATA disk drives (SM8.42 microcode and APAR IY62752) IBM DS4500 with IBM TotalStorage DS4000 EXP100 Storage Expansion Unit (1742-60U and 1742-60X ) with SATA disk drives (SM8.42 microcode and APAR IY62752) IBM TotalStorage ESS (2105-F20 or 2105-800 without SDD - persistent reserve mode) IBM SAN Volume Controller (APARs IY47267 and IY42355) EMC Symmetrix DMX Storage Subsystems (FC attach only) IBM SSA (all disk sizes) Restrictions: Fibre Channel disks must be FC switch attached. No disk can be larger than 1 TB.

| | | | | | | |

Disks Supported:

Q4: A4:

What is an rpd cluster type? The GPFS cluster type rpd is based on the Reliable Scalable Cluster Technology (RSCT) subsystem of AIX 5L. The GPFS cluster is defined within an existing RSCT peer domain. The nodes which are members of the GPFS cluster are defined via the mmcrcluster, mmaddcluster, and mmdelcluster commands. With an RSCT peer domain, all nodes in the GPFS cluster have the same view of the domain and share the resources within the domain. Within the GPFS cluster, you define one or more nodesets within which your file systems operate. Disk requirements within your GPFS rpd cluster utilizing an IP network, require that all disks must be: 1. Logical volumes: v SAN-attached to all nodes in the cluster. v Processed via the mmcrlv command to create logical volumes useable by GPFS. 2. Virtual shared disks: v Accessed via the IBM Virtual Shared Disk component of RSCT. v Processed via the mmcrvsd command to create or register disks for use by GPFS. 3. Conform to the disk support guidelines listed in this FAQ. Disk requirements within your GPFS rpd cluster utilizing the IBM Eserver pSeries High Performance Switch (HPS), require that all disks must be: 1. Accessed via the IBM Virtual Shared Disk component of RSCT. 2. Processed via the mmcrvsd command to create or register disks for use by GPFS.

Q5: A5:

When running GPFS in an rpd cluster type, what are the software requirements? In an rpd cluster type, GPFS requires the RSCT component of AIX. The GPFS cluster is defined within an existing RSCT peer domain via the mmcrcluster command.
AIX Level AIX 5L Version 5 Release 2 For use of the Eserver p5, APAR IY58249, or later modifications For use of the pSeries HPS and IBM Virtual Shared Disks, AIX 5L Version 5 Release 2 with APAR IY56532, or later modifications. GPFS End of Service

Table 6. Supported levels of AIX in an rpd cluster type GPFS Version GPFS v2.2.1

GPFS v2.1

AIX 5L Version 5 Release 2

30April2006

For use of the pSeries HPS and IBM Virtual Shared Disks, 30April2006 AIX 5L Version 5 Release 2 with APAR IY47306, or later modifications. AIX 5L v5.1 5100-03 Recommended Maintenance package 30April2006 (APAR IY30258), or later

Note: Ensure you are at the current service level for each software level listed. Q6: A6: What disk support guidelines must be followed when running GPFS in an rpd cluster type? In an rpd cluster type, a typical GPFS cluster configuration uses multi-node quorum. Deviations from the typical case are two-node GPFS clusters which have single-node quorum enabled. Single-node quorum may have an impact on the types of disks supported. This distinction is noted in the Cluster Properties section in Table 7 on page 8, Table 8 on page 8, Table 9 on page 9, and Table 10 on page 9. For additional disk configuration information and examples, please see ftp.software.ibm.com/storage/fastt/fastt500/PSSP-GPFS_config_info.pdf

Table 7. GPFS v2.2.1 running AIX 5L v5.2 Cluster properties: Typical GPFS cluster configuration (2 - 32 nodes) IBM DS4100 with SATA disk drives (SM8.42 microcode and APAR IY62752) IBM FAStT500 (SM7.10, SM8.21, and SM8.3 microcode)

| |

Disks supported:

| | | | | | | | | |

IBM DS4300 (SM8.33 microcode) IBM DS4300 Turbo (SM8.40 microcode) IBM DS4300 Turbo with IBM TotalStorage DS4000 EXP100 Storage Expansion Unit (1742-60U and 1742-60X ) with SATA disk drives (SM8.42 microcode and APAR IY62752) IBM DS4400 (SM8.21 and SM8.3 microcode) IBM DS4500 (SM8.3 microcode) IBM DS4500 with IBM TotalStorage DS4000 EXP100 Storage Expansion Unit (1742-60U and 1742-60X ) with SATA disk drives (SM8.42 microcode and APAR IY62752) IBM TotalStorage ESS (2105-F20 or 2105-800 with SDD - disk leasing mode) IBM TotalStorage ESS (2105-F20 or 2105-800 using AIX MPIO and SDDPCM - persistent reserve mode) IBM SAN Volume Controller (APARs IY47267 and IY42355) EMC Symmetrix DMX Storage Subsystems (FC attach only) IBM SSA (all disk sizes) Hitachi Lightning 9900 (9910, 9960, 9970V and 9980V ) (HDLM required) Restrictions: Fibre Channel disks must be FC switch attached Maximum cluster size is 8 nodes if using SSA disks Table 8. GPFS v2.2.1 running AIX 5L v5.2 Cluster properties: Single-node quorum (two node cluster only) IBM DS4100 with SATA disk drives (SM8.42 microcode and APAR IY62752) IBM SSA (all disk sizes) IBM TotalStorage Enterprise Storage Server (ESS) (2105-F20 or 2105-800 using AIX MPIO and SDDPCM - persistent reserve mode)

| |

Disks supported:

| | | | |
Restrictions:

IBM DS4300 Turbo with IBM TotalStorage DS4000 EXP100 Storage Expansion Unit (1742-60U and 1742-60X ) (SM8.42 microcode and APAR IY62752) IBM DS4500 with IBM TotalStorage DS4000 EXP100 Storage Expansion Unit (1742-60U and 1742-60X ) (SM8.42 microcode and APAR IY62752) For single-node quorum - no disk leasing mode disks can be present. Fibre Channel disks must be FC switch-attached.

Table 9. GPFS v2.1 running AIX 5L v5.1 Cluster properties: Typical GPFS cluster configuration (2 - 32 nodes) IBM DS4100 with SATA disk drives (SM8.42 microcode and APAR IY61688) IBM FAStT500 (SM7.10, SM8.21, and SM8.3 microcode)

| |

Disks supported:

| | | | | | | | | |

IBM DS4300 (SM8.33 microcode) IBM DS4300 Turbo (SM8.40 microcode) IBM DS4300 Turbo with IBM TotalStorage DS4000 EXP100 Storage Expansion Unit (1742-60U and 1742-60X ) with SATA disk drives (SM8.42 microcode and APAR IY61688) IBM DS4400 (SM8.21 and SM8.3 microcode) IBM DS4500 (SM8.3 microcode) IBM DS4500 with IBM TotalStorage DS4000 EXP100 Storage Expansion Unit (1742-60U and 1742-60X ) with SATA disk drives (SM8.42 microcode and APAR IY61688) IBM TotalStorage ESS (2105-F20 or 2105-800 with SDD - disk leasing mode) IBM TotalStorage ESS (2105-F20 or 2105-800 without SDD - persistent reserve mode) IBM SAN Volume Controller (APARs IY47267 and IY42355) EMC Symmetrix/DMX Storage Subsystems (FC attach only) IBM SSA (all disk sizes) Hitachi Lightning 9900 (9910, 9960, 9970V and 9980V ) (HDLM required) Restrictions: Fibre Channel disks must be FC switch attached. Maximum cluster size is 8 nodes if using SSA disks Table 10. GPFS v2.1 running AIX 5L v5.1 Cluster properties: Disks supported: Single-node quorum (two node cluster only) IBM SSA (all disk sizes) IBM TotalStorage ESS (2105-F20 or 2105-800 without SDD) - persistent reserve mode

| | | | |
Restrictions:

IBM DS4300 Turbo with IBM TotalStorage DS4000 EXP100 Storage Expansion Unit (1742-60U and 1742-60X ) (SM8.42 microcode and APAR IY61688) IBM DS4500 with IBM TotalStorage DS4000 EXP100 Storage Expansion Unit (1742-60U and 1742-60X ) (SM8.42 microcode and APAR IY61688) For single-node quorum - no disk leasing mode disks can be present. Fibre Channel disks must be FC switch-attached.

Notes: 1. You may have up to 1024 external shared disks, disk arrays, or logical units (LUNs) with the adapters configured to allow each disk, connectivity to each node in the GPFS cluster. No disk can be larger than 1 TB. 2. You cannot protect your file system against disk failure by mirroring data at the LVM level. You must use GPFS replication or RAID devices to protect your data 1. Q7: A7: What is an hacmp cluster type? The GPFS cluster type hacmp is based on IBMs High Availability Cluster Multi-Processing for AIX (HACMP) licensed product. The GPFS cluster is defined over an existing HACMP cluster. The nodes which are members of the GPFS cluster are defined via the mmcrcluster, mmaddcluster, and mmdelcluster commands. Within the GPFS cluster, you define one or more nodesets within which your file systems operate. Disk requirements within your GPFS hacmp cluster require that all disks must be: 1. SAN-attached to all nodes in the cluster. 2. Processed via the mmcrlv command. 3. Follow the disk support guidelines listed in this FAQ. Q8: A8: When running GPFS in an hacmp cluster type, what are the software requirements? In an hacmp cluster type, GPFS requires IBMs HACMP licensed product over which the GPFS cluster is defined via the mmcrcluster command.
AIX Level AIX 5L v5.2 HACMP Level HACMP v5.1 GPFS End of Service

Table 11. Supported software levels in an hacmp cluster type GPFS Version GPFS v2.2.1

For use of the Eserver p5, APAR IY58249, or later modifications GPFS v2.1 GPFS v2.1 AIX 5L v5.2 AIX 5L v5.1 HACMP v4.5 / v5.1 HACMP/ES v4.4.1/4.5 30April2006 30April2006

Notes: 1. Ensure you are at the current service level for each software level listed 2. If running with HACMP/ES v4.5, the minimum APAR level required is IY32874 Q9: A9: What are the disk support guidelines that must be followed when running GPFS in an hacmp cluster type? In an hacmp cluster type, a typical GPFS cluster configuration uses multi-node quorum. Deviations from the typical case are two-node GPFS clusters which have single-node quorum enabled. Single-node quorum may have an impact on the types of disks supported. This distinction is noted in the Cluster Properties section in Table 12 on page 11, Table 13 on page 11, Table 14 on page 12, and Table 15 on page 12. For additional disk configuration information and examples, please see ftp://ftp.software.ibm.com/storage/fastt/fastt500/PSSPGPFS_config_info.pdf

1. IBM General Parallel File System for AIX 5L in an RSCT Peer Domain: Concepts, Planning, and Installation Guide Number GA22-7974 (July 2004), International Business Machines Corporation, 2455 South Road, Poughkeepsie, NY 12601-5400, USA.

10

Table 12. GPFS v2.2.1 running AIX 5L v5.2 Cluster properties: Typical GPFS cluster configuration (2- 32 nodes) IBM DS4100 with SATA disk drives (SM8.42 microcode and APAR IY62752) IBM FAStT500 (SM7.10, SM8.21, and SM8.3 microcode)

| |

Disks supported:

| | | | | | | | | |

IBM DS4300 (SM8.33 microcode) IBM DS4300 Turbo (SM8.40 microcode) IBM DS4300 Turbo with IBM TotalStorage DS4000 EXP100 Storage Expansion Unit (1742-60U and 1742-60X ) with SATA disk drives (SM8.42 microcode and APAR IY62752) IBM DS4400 (SM8.21 and SM8.3 microcode) IBM DS4500 (SM8.3 microcode) IBM DS4500 with IBM TotalStorage DS4000 EXP100 Storage Expansion Unit (1742-60U and 1742-60X ) with SATA disk drives (SM8.42 microcode and APAR IY62752) IBM TotalStorage ESS (2105-F20 or 2105-800 with SDD - disk leasing mode) IBM TotalStorage ESS (2105-F20 or 2105-800 using AIX MPIO and SDDPCM - persistent reserve mode) IBM SAN Volume Controller (APARs IY47267 and IY42355) EMC Symmetrix DMX Storage Subsystems (FC attach only) IBM SSA (all disk sizes) Hitachi Lightning 9900 (9910, 9960, 9970V and 9980V ) (HDLM required) Restrictions: Fibre Channel disks must be FC switch attached. Maximum cluster size is 8 nodes if using SSA disks Table 13. GPFS v2.2.1 running AIX 5L v5.2 Cluster properties: Single-node quorum (two node cluster only) IBM DS4100 with SATA disk drives (SM8.42 microcode and APAR IY62752) IBM SSA (all disk sizes) IBM TotalStorage Enterprise Storage Server (ESS) (2105-F20 or 2105-800 using AIX MPIO and SDDPCM - persistent reserve mode)

| |

Disks supported:

| | | | |
Restrictions:

IBM DS4300 Turbo with IBM TotalStorage DS4000 EXP100 Storage Expansion Unit (1742-60U and 1742-60X ) (SM8.42 microcode and APAR IY62752) IBM DS4500 with IBM TotalStorage DS4000 EXP100 Storage Expansion Unit (1742-60U and 1742-60X ) (SM8.42 microcode and APAR IY62752) For single-node quorum no disk leasing mode disks can be present. Fibre Channel disks must be FC switch attached.

11

Table 14. GPFS v2.1 running AIX 5L v5.1 Cluster properties: Typical GPFS cluster configuration (2- 32 nodes) IBM DS4100 with SATA disk drives (SM8.42 microcode and APAR IY61688) IBM FAStT500 (SM7.10, SM8.21, and SM8.3 microcode)

| |

Disks supported:

| | | | | | | | |

IBM DS4300 (SM8.33 microcode) IBM DS4300 Turbo with IBM TotalStorage DS4000 EXP100 Storage Expansion Unit (1742-60U and 1742-60X ) with SATA disk drives (SM8.42 microcode and APAR IY61688) IBM DS4400 (SM8.21 and SM8.3 microcode) IBM DS4500 (SM8.3 microcode) IBM DS4500 with IBM TotalStorage DS4000 EXP100 Storage Expansion Unit (1742-60U and 1742-60X ) with SATA disk drives (SM8.42 microcode and APAR IY61688) IBM TotalStorage ESS (2105-F20 or 2105-800 with SDD - disk leasing mode) IBM TotalStorage ESS (2105-F20 or 2105-800 without SDD - persistent reserve mode) IBM SAN Volume Controller (APARs IY47267 and IY42355) EMC Symmetrix DMX Storage Subsystems (FC attach only) IBM SSA (all disk sizes) Hitachi Lightning 9900 (9910, 9960, 9970V and 9980V ) (HDLM required) Restrictions: Fibre Channel disks must be FC switch attached. Maximum cluster size is 8 nodes if using SSA disks. Table 15. GPFS v2.1 running AIX 5L v5.1 Cluster properties: Single-node quorum (two node cluster only) IBM DS4100 with SATA disk drives (SM8.42 microcode and APAR IY61688) IBM SSA (all disk sizes) IBM TotalStorage ESS (2105-F20 or 2105-800 without SDD) - persistent reserve mode

| |

Disks supported:

| | | | |
Restrictions:

IBM DS4300 Turbo with IBM TotalStorage DS4000 EXP100 Storage Expansion Unit (1742-60U and 1742-60X ) (SM8.42 microcode and APAR IY61688) IBM DS4500 with IBM TotalStorage DS4000 EXP100 Storage Expansion Unit (1742-60U and 1742-60X ) (SM8.42 microcode and APAR IY61688) For single-node quorum no disk leasing mode disks can be present. Fibre Channel disks must be FC switch attached.

12

Notes: 1. You may have up to 1024 external shared disks, disk arrays, or LUNs with the adapters configured to allow each disk, connectivity to each node in the GPFS cluster. No disk can be larger than 1 TB. 2. You cannot protect your file system against disk failure by mirroring data at the LVM level. You must use GPFS replication or RAID devices to protect your data 2. Q10: A10: How can I implement disaster recovery for my GPFS cluster? Beginning with GPFS 2.2.1 and APAR IY59339, disaster recovery for your GPFS cluster can be implemented as described in Disaster Recovery with General Parallel File System located at http://www.ibm.com/servers/eserver/pseries/software/whitepapers/gpfs_recovery.html What are the coexistence rules for GPFS clusters? You must be concerned with the coexistence rules for each type of GPFS cluster. Updates to the coexistence rules for GPFS clusters listed in this FAQ supersede the product documentation. Table 16 and Table 17 identify release levels of additional clustering software which may coexist on the same node. This does not imply interaction with GPFS if the software is not required for the particular GPFS cluster type. Please note that PSSP and Cluster System Management (CSM) may not coexist and therefore the information is presented in two separate tables. There are also special considerations noted in Table 18 on page 14.
Table 16. Coexistence of clustering software with PSSP GPFS cluster type rpd (IP network) rpd (IP network) hacmp hacmp sp sp GPFS level 2.2.1 2.1 2.2.1 2.1 2.2.1 2.1 AIX level 5.2 5.2/5.1 5.2 5.1 5.2 5.1 PSSP level 3.5 3.5 3.5 3.5 3.5 3.5 HACMP level 5.1/4.5 5.1/4.5/4.4.1 5.1/4.5 4.5/4.4.1 5.1/4.5 4.5/4.4.1

Q11: A11:

Table 17. Coexistence of clustering software with CSM GPFS cluster type rpd (IP network) rpd pSeries HPS rpd (IP network) rpd pSeries HPS hacmp hacmp GPFS level 2.2.1 2.2.1 2.1 2.1 2.2.1 2.1 AIX level 5.2 5.2 5.2/5.1 5.2 5.2 5.2/5.1 CSM level 1.4 / 1.3.x.x 1.4 / 1.3x.x 1.1.x.x 1.4 / 1.3.x.x 1.4 / 1.3.x.x 1.1. x.x HACMP level 5.1/4.5 5.1/4.5 5.1/4.5/4.4.1 5.1/4.5/4.4.1 5.1/4.5 4.5/4.4.1

Notes: 1. The fact that GPFS and CSM may coexist does not imply that there is interaction between GPFS and CSM. 2. For the CSM levels, x.x refers to the current modification and fix levels. 3. PSSP does not support the pSeries HPS, therefore a GPFS rpd cluster utilizing the pSeries HPS may not coexist with PSSP

2. IBM General Parallel File System for AIX 5L in an HACMP Cluster: Concepts, Planning, and Installation Guide Number GA22-7971-00 (December 2003), International Business Machines Corporation, 2455 South Road, Poughkeepsie, NY 12601-5400, USA.

13

Table 18. Special considerations which must be adhered to when the products coexist GPFS Cluster Type rpd sp hacmp CSM 2 2 2 RSCT peer domain 2 2 2 PSSP 1,2 2 1,2 HACMP 2 2 2

1. When running in either a GPFS rpd or hacmp cluster type that contains nodes that are also within an sp cluster, you need to be concerned with the daemon to daemon authentication method for the common nodes. GPFS, as an SP trusted service, can have its daemon to daemon authentication method set by the system administrator. The authentication method for all nodes in your GPFS nodeset must be compatible. If nodes within a cluster are not configured for a compatible authentication method, an error condition will be written to/var/adm/ras/mmfs.log.latest when there is an attempt to make a connection between them. See the section on Security in the GPFS for AIX 5L in a PSSP Cluster: Clusters Concepts, Planning, and Installation Guide and search on Security. 2. The level of RSCT is determined by your AIX service level. Q12: A12: What type of disk fencing mechanisms does GPFS support? In order to preserve data integrity in the event of certain system failures, GPFS is designed to prevent a node that is down from accessing the file system until it returns to the available state. GPFS is said to fence the failing node from the disks. There are two types of fencing mechanisms: v Fencing that takes advantage of hardware assist v Fencing that is done in GPFS software without hardware assist Depending upon the types of disks that are being using, there are three possible ways for the fencing to occur: SSA fencing An SSA disk subsystem specific fencing mechanism. SSA fencing takes advantage of hardware assistance. persistent reserve SCSI-3 standard reserve mechanism. Persistent reserve takes advantage of hardware assistance. See the question Does GPFS support AIX MPIO? Disk leasing A GPFS specific fencing mechanism for disks which do not support either SSA fencing or persistent reserve. Disk leasing does not rely on any hardware assistance. GPFS can use different types of disks in the same cluster. GPFS will determine the disks capability and prepare the disks for proper use during the setup phase. This preparation is performed by the GPFS command mmcrlv and mmcrvsd, and can also be performed by the RVSD command createvsd. No options need to be provided to direct the GPFS codes to perform this function. GPFS will always attempt to use fencing with hardware assistance if possible. There can only be one fencing mechanism active in a cluster, hardware assisted or disk leasing. If all the disks in a configuration support fencing with hardware assistance, that configuration will be running in a useDiskLeasing no environment. In other words, disk leasing is not being used in this environment to perform fencing, the fencing is being done with the assistance of the disk subsystem hardware and firmware. If any disk in a GPFS configuration does not support hardware fencing, all disks in that same environment revert to using disk leasing. If any or all disks in a configuration do not support

14

hardware fencing, the cluster will be running in a useDiskLease yes environment. In other words, disk leasing is being used and none of the capabilities of the disk subsystem and firmware are being exploited for fencing. Single-node quorum is not supported when disk leasing is in effect. The only manual action that needs to be performed by the administrator is in the SSA disk types, the ODM attribute ssar needs to be setup on each node in the cluster. For directions on using the mmlsfenceid and mmsetfenceid commands: v For the GPFS cluster type rpd, please see the General Parallel File System for AIX 5L in an RSCT Peer Domain: Problem Determination Guide . v For a GPFS cluster type of hacmp, please the GPFS for AIX 5L in an HACMP Cluster General Parallel File System for AIX 5L in an HACMP Cluster: Problem Determination Guide . Q13: A13: How can the fencing technique in operation for a cluster be verified? The current fencing mode for a cluster is either useDiskLease yes or useDiskLease no. The current state of disk leasing in a cluster is determined by using the mmlsconfig command. What is AIX MPIO and does GPFS support it? Beginning with AIX 5.2, GPFS can use the Multiple Path I/O (MPIO) feature of AIX to define alternate paths to a device for failover purposes. Failover is a path-management algorithm that improves the reliability and availability of a device because the system automatically detects when one I/O path fails and re-routes I/O through an alternate path. All SCSI SCSD disk drives are automatically configured as MPIO devices. Other devices can be supported, providing the device driver is compatible with the MPIO implementation in AIX. For more information about MPIO, see the: v AIX 5L Version 5.2 System Management Concepts: Operating System and Devices book and search on Multi-path I/O. v AIX 5L Version 5.2 System Management Guide: Operating System and Devices book and search on Multi-path I/O. As GPFS supports disk subsystems exploiting the AIX MPIO function, this FAQ will be updated and the new support statements will be communicated to the IBM Sales representatives. Please see: v Q3: What disk support guidelines must be followed when running GPFS in an sp cluster type? v Q6: What disk support guidelines must be followed when running GPFS in an rpd cluster type? v Q9:What are the disk support guidelines that must be followed when running GPFS in an hacmp cluster type? Q15: A15: What disks support hardware fencing? The current set of disks that support hardware fencing are: v IBM SSA all models v IBM TotalStorage ESS using AIX MPIO and SDDPCM v Selected members of the IBM TotalStorage DS4000 family of subsystems with microcode at versions 8.40 or greater. See the individual cluster types (sp, rpd, hacmp) for details. What should I do if commands such as mmcrlv, mmcrfs, mmadddisk, mmrpldisk, or mount fail to complete due to problems with non-IBM disks? Certain GPFS disk commands issue multiple AIX varyonvg -u commands in parallel to the nodes in the cluster. For some non-IBM disks, when many varyonvg -u commands are issued in parallel, some of the AIX varyonvg -u invocations do not complete and the GPFS disk command hangs. This is recognized by the GPFS disk command not completing after a long period of time and the persistence of the varyonvg processes as shown by the output of the ps -ef command on some of the nodes of the cluster. In such cases, kill the varyonvg processes issued by the disk

Q14: A14:

| | Q16: A16:

15

command on the nodes of the cluster to allow the disk command to complete. Before mounting the file system in question on any node where a varyonvg process was killed, issue the varyonvg -u vgname command on the node to make the disk available to GPFS. Do this on each of the nodes in question one by one until all of the GPFS volume groups are varied on. Q17: A17: What cluster specific single-node quorum restrictions are there? Restrictions when using single-node quorum include: 1. Single-node quorum is not supported when: v The GPFS cluster type lc consists of only AIX nodes. v Disk leasing is in effect. Disk leasing is a GPFS-specific fencing mechanism for disks which do not support either SSA fencing or persistent reserve. Disk leasing is activated if any disk in any file system in the nodeset is not using SSA fencing or hardware assisted persistent reserve. See the question What type of disk fencing mechanisms does GPFS support? v SDD (the non-MPIO version) is installed. 2. If a disk is using persistent reserve, this needs to be both supported at both the disk subsystem level and the AIX device driver level. 3. When adding nodes to a nodeset: v The GPFS daemon must be stopped on all of the nodes. v If after adding the nodes, the number of nodes in the nodeset exceeds two, the quorum algorithm is automatically changed to the multi-node quorum algorithm. 4. When deleting nodes from a nodeset: v A node cannot be deleted from the nodeset without stopping the GPFS daemon on both nodes. v If the number of nodes remaining in the nodeset falls below three, you must run the mchconfig command if you want to change the quorum algorithm to single-node quorum. See the question, What considerations are there for two node nodesets? Q18: A18: Why would I want to use EtherChannel in my GPFS cluster? The main benefit of EtherChannel and IEEE 802.3ad Link Aggregation is increased bandwidth: the aggregated adapter has the network bandwidth of all its adapters in a single network presence. In addition, if an adapter fails, the packets are automatically sent on the next available adapter without disruption to existing user connections. The adapter is automatically returned to service on the EtherChannel or Link Aggregation when it recovers. EtherChannel and IEEE802.3ad each requires support within the Ethernet switch. Refer to the product documentation for your switch to detemine if EtherChannel is supported. To determine if your version of GPFS supports EtherChannel, see the question, What versions of GPFS support EtherChannel? For details on how to configure EtherChannel and IEEE 802.3ad Link Aggregation: 1. Go to
publib16.boulder.ibm.com/pseries/en_US/aixbman/commadmn/tcp_etherchannel.htm#yu528frokferg

2. Search on Configuring EtherChannel Hint: Make certain that the switch ports are configured for LACP (the default is PAGP). For details on how to verify whether the adapter and the switch are operating with the correct protocols For IEEE 802.3ad: 1. Go to
publib16.boulder.ibm.com/pseries/en_US/aixbman/commadmn/tcp_etherchannel.htm#yu528frokferg

2. Search on Troubleshooting IEEE 802.3ad. For additional service updates regarding the use of EtherChannel: 1. Go to www.ibm.com/support 2. Click Search technical support 3. Enter the search term EtherChannel

16

4. Click Submit Hint: A useful command for troubleshooting, where device is the Link Aggregation device, is:
entstat -d device

Q19: A19: | | | |

What versions of GPFS support EtherChannel? GPFS support for EtherChannel includes: v GPFS 2.2 on cluster types rpd or hacmp on AIX 5.2 and maintenance package 5.2.0.13 with APAR IY48170 and APAR IY44290 v GPFS 2.1 on cluster types rpd or hacmp on AIX 5.1 and maintenance package 5.1.0.50 with APAR IY48331 For the latest service level, go to www-912.ibm.com/eserver/support/fixes/fcgui.jsp 1. For your server type, select pSeries family. 2. For product or fix type select AIX OS, Java, compilers. 3. For your option, select the type of fixes you are searching for. 4. Select your OS level. 5. Click continue. 6. If choosing Latest fixes or Critical fixes, the GPFS fixes will be bundled with all of the file sets. 7. If you choose Specific fixes, enter mmfs as your search string

Q20: A20:

Whats the difference between using an ESS with or without SDD or SDDPCM installed on the host? The use of SDD or SDDPCM gives the AIX host the ability to access multiple paths to a single LUN within an ESS. This ability to access a single LUN on multiple paths allows for a higher degree of data availability in the event of a path failure. Data can continue to be accessed within the ESS as long as there is at least one available path. Without one of these installed, you will lose access to the LUN in the event of a path failure. However, your choice of whether to use SDD or SDDPCM impacts your ability to use single-node quourm: v Single-node quorum is not supported if SDD is installed. v Single-node quorum is support if SDDPCM is installed. To determine the GPFS disk support guidelines for SDD and SDDPCM for your cluster type, see v Q3: What disk support guidelines must be followed when running GPFS in an sp cluster type? v Q6: What disk support guidelines must be followed when running GPFS in an rpd cluster type? v Q9:What are the disk support guidelines that must be followed when running GPFS in an hacmp cluster type?

Q21: A21:

Is a Fibre Channel Switch required for all Fibre Channel disk attachments which are used by GPFS? Yes, all Fibre Channel disks require attachment to AIX hosts using a SAN FC switch when used by GPFS. What Fibre Channel Switches are qualified for GPFS usage and is there a FC Switch support chart available? There are no special requirements for FC switches used by GPFS other than the switch must be supported by AIX. For further information see www.storage.ibm.com/ibmsan/index.html What performance tuning suggestions are there for AIX nodes? Performance tuning suggestions for AIX nodes include: ipqmaxlen In a GPFS cluster type sp or rpd (utilizing the pSeries HPS), the ipqmaxlen network

Q22: A22:

Q23: A23:

17

option controls the number of incoming packets that can exist on the IP interrupt queue. Since both GPFS and the IBM Virtual Shared Disk use IP, the default value of 128 is often insufficient. This is especially important if your virtual shared disks are configured over IP rather than KLAPI. The recommended setting is 512:
no -o ipqmaxlen=512

Buddy buffers In a GPFS cluster type sp or rpd (utilizing the pSeries HPS), the buddy buffer is pinned kernel memory. The virtual shared disk server node uses the buddy buffer to temporarily store data for I/O operations originating at a client node. The data in a buddy buffer is purged immediately after the I/O operation completes. The values associated with the buddy buffer are: v Minimum buddy buffer size allocated to a single request v Maximum buddy buffer size allocated to a single request v Total number of maximum-size buffers Total number of maximum-size buddy buffers that the system will attempt to dynamically allocate IBM suggests using the following values: v Minimum buddy buffer size: 4096 (4 KB) v Maximum buddy buffer size: 262144 (256 KB) If your application uses the fastpath option of asynchronous I/O, the maximum buddy buffer size must be greater than or equal to 128 KB. Otherwise, you will receive EMSGSIZE Message too long errors. v Total number of maximum-size buffers: 2000 for a virtual shared disk server node One for a virtual shared disk client-only node You can either set these values using the vsdnode command, or update the values using the updatevsdnode command. IBM suggests, however, that you configure the recommended amount of buddy buffer space, unless your system is memory-constrained and you want to restrict the amount of buddy buffer space. When the device driver is configured, the total buddy buffer space is not pinned; instead, approximately one-quarter of the total space requested is pinned when the device driver is configured. This initial amount of spaced pinned is limited to a maximum of 64 MB for a 32-bit kernel, or 128 MB for a 64-bit kernel. After configuration, the device driver attempts to dynamically expand and contract additional buddy buffer space up to the maximum specified, or until AIX can no longer satisfy the memory request. If a buddy buffer cannot be obtained, then the request is queued at the virtual shared disk server until a buddy buffer is available. max_coalesce The max_coalesce (SSA) parameter defines the maximum number of bytes that the SSA disk device driver attempts to transfer to or from an SSA logical disk in one operation. This parameter is very important when dealing with writes on SSA RAID systems 3. The suggested setting is 256KB for a RAID-5 4+P configuration:
chdev -l hdiskx -a max_coalesce=0x40000

KLAPI In the GPFS cluster type sp, the default IBM Virtual Shared Disk communication protocol is IP. KLAPI is an alternate high-performance communication interface that can be used to avoid some data copies (which would have been in the IP path) in handling the I/O request. Avoiding the data copies reduces CPU utilization on both the client and server nodes. In order to switch from IP to KLAPI, the RVSD system has to be stopped and IBM
3. Dominique Heger, Gautam Shah: General Parallel File System (GPFS v1.4) for AIX Architecture and Performance, November 2001 4. Dominique Heger, Gautam Shah: General Parallel File System (GPFS v1.4) for AIX Architecture and Performance, November 2001

18

Virtual Shared Disk component has to be unconfigured. In the case that IBM Virtual Shared Disk over KLAPI encounters a communication problem, the system will switch to IBM Virtual Shared Disk over IP. Choosing KLAPI will not affect the number of user-space SP Switch2 windows that are available to user applications 4:
ctlvsd -l on

SP Switch, SP Switch2, and pSeries HPS subsystem parameters Depending upon which switch you are using, the communications subsystem parameters should be configured differently. These are especially important if you configure your virtual shared disks over IP. If you are using an SP Switch, the rpoolsize and spoolsize values should be set to 16777216:
chgcss -l css0 -a rpoolsize=16777216 -a spoolsize=16777216

If you are using an SP Switch2, the rpoolsize and spoolsize values should be set to 33554432:
chgcss -l css0 -a rpoolsize=33554432 -a spoolsize=33554432

If you are using the pSeries HPS, the rpoolsize and spoolsize values should be set to 67108864:
chgsni -l sn0 a rpoolsize=67108864 a spoolsize 67108864

For further performance information on the SP Switch or the SP Switch2, see the GPFS white papers at www.ibm.com/servers/eserver/clusters/library/wp_aix_lit.html Q24: A24: What configuration and performance tuning suggestions are there for GPFS when used primarily for Oracle databases? Oracle uses the Asynchronous I/O (AIO) and Direct I/O (DIO) features of AIX to do their own scheduling of I/O directly to disks, bypassing most of GPFS caching and prefetching facilities. Most of these settings are to make sure GPFS is out of the way once a database has been formatted, so Oracle can get the best performance: v When setting up your LUNS it is important to: 1. Create the logical volumes such that they map one to one with a volume group and a volume group be one to one with a LUN which is a single RAID device. 2. Not stripe logical volumes across multiple RAIDs for I/O parallelism when using raw logical volumes because: GPFS puts each block of a file on different LUNs spread across all LUNs Logical volume striping makes removing a bad RAID more difficult v For file systems holding large Oracle databases, set the GPFS file system block size via the mmcrfs command using the -B option, to a large value: 512 KB is generally suggested. 256 KB is suggested if there is activity other than Oracle using the filesystem and many small files exist which are not in the database. 1 MB is suggested for file systems 100 TB or larger. The large block size makes the allocation of space for the databases manageable and has no affect on performance when Oracle is using AIO/DIO. v Set the GPFS worker threads via the mmchconfig - worker1Threads command to allow the maximum parallelism of the Oracle AIO threads: On a 64-bit AIX kernel, the setting can be as large as 548. The GPFS prefetch threads must be adjusted accordingly via the mmchconfig prefetchThreads command as the sum of those two classes of threads must be 550 or less. On a 32-bit kernel, the setting can be as large as 162. The GPFS prefetch threads must be adjusted accordingly via the mmchconfig prefetchThreads command as the sum of those two classes of threads must be 164 or less.

19

When requiring GPFS sequential I/O, set the prefetch threads between 50 and 100 (the default is 64), and set the worker threads to have the remainder. Note: These changes via the mmchconfig command take effect upon restart of the GPFS daemon. v The number of AIX AIO kprocs to create should be approximately the same as the GPFS worker1Threads setting. For the AIO maxservers setting: On AIX 5.1 systems, it is the total number of AIO kprocs. On AIX 5.2 systems, it is the number of kprocs PER CPU. It is suggested to set is slightly larger than worker1Threads divided by the number of CPUs. For example if worker1Threads is set to 500 on a 32-way SMP: On an AIX 5.1 system, set maxservers to 640 On an AIX 5.2 system, set maxservers to 20 Setting the GPFS striping method to balanced random via the mmcrfs -s balancedRandom or the mmchfs -s balancedRandom command may prove to make disk access patterns slightly more random than the system default. This may avoid creating hotspots thus creating less work for the GPFS administrative operations that rebalance data across all LUNs such as mmdeldisk, mmadddisk, and mmrestripefs. Set the Oracle database block size equal to the LUN segment size or a multiple of the LUN pdisk segment size. Set the Oracle read-ahead value to prefetch one or two full GPFS blocks. For example, if your GPFS block size is 512 KB, set the Oracle blocks to either 32 or 64 16 KB blocks. Do not use the dio option on the mount command as this forces DIO when accessing all files. Oracle automatically uses DIO to open database files on GPFS. There are fixes in the GPFS PTF stream necessary for correct AIO/DIO operation for Oracle. If using GPFS release 2.1, the mmfs.base.rte fileset should be at least 3.5.0.11.

v v v v

Q25: A25:

What is the largest file system size supported by GPFS? On 32- bit AIX machines, the largest supported file system size depends on the blocksize of your file system: v If your file system has a 16 KB block size, you may have one or more file systems with a total size of 1 TB mounted. v If your file system has a 64 KB block size, you may have one or more file systems with a total size of 10 TB mounted v If your file system has a 256 KB or greater block size, you may have one or more file systems mounted with a total size of not greater than 200 TB where no single file system exceeds 100 TB mounted. The maximum supported file system size of 100 TB is a testing limitation rather than an architectural limitation of the GPFS licensed product.

Q26: A26:

Are there any special concerns regarding the use of mksysb images? Using a mksysb image to install or reinstall the IBM Virtual Shared Disk licensed product on a node presents some challenges to be aware of. In a GPFS cluster type sp: v The RVSD version number that the post-installation scripts would normally save in the System Data Repository can not be saved. This may prevent the RVSD subsystem from starting because it thinks that there are back-level nodes within the SP partition. This problem can be prevented by using the rvsdrestrict command to explicitly set the restriction level before starting RVSD. For example, on the control workstation issue:
rvsdrestrict -s RVSD3.5

v The RVSD subsystem expects that the underlying disks and volume groups are available to AIX. Therefore, if you are reinstalling an existing server node from a mksysb image, you may need to use the AIX importvg command to make AIX aware of the pre-existing volume groups.

20

Note: All virtual shared disk volume groups must have the auto varyon attribute turned off. To accomplished this, after the importvg, issue:
chvg -a -n volume_group_name

In a GPFS cluster type rpd, duplicate node ids may be created. The lsnodeid command (available in /usr/sbin/rsct/bin) has been provided for you to verify whether or not node ids are duplicated within the cluster. If a duplicate node id is found, see the General Parallel File System for AIX 5L in an RSCT Peer domain: Administration and Programming Reference and search on Changing IP addresses, node names, node numbers and reliable hostnames Q27: A27: Are there any considerations when using NIS automount? The default file system type when using the automounter daemon is NFS. When the -fstype option is not specified, and the server is the local node, a soft-mount of the local directory is done at the desired mount point. JFS is assumed as the only handler of local directories. A GPFS file system local soft-mount does not work implicitly, since the mount request is passed to JFS which then produces an error. When specifying -fstype mmfs the local soft-mount works because the mount is then passed to GPFS instead of JFS. A GPFS soft-mount does not automatically unmount. Setting -fstype nfs3 causes the local server mounts to always go through NFS. This allows you to have the same auto.map file on all nodes whether the server is local or not and the automatic unmount will occur. If you want local soft-mounts of GPFS file systems while other nodes perform NFS mounts, you should have different auto.map files on the different classes of nodes. This should improve performance on the GPFS nodes as they will not have to go through NFS. Q28: A28: How can I ask a more specific question about GPFS? If you have a question that can benefit other GPFS users, then please post it to the public GPFS community mailing list (gpfs-general@www-124.ibm.com). You can sign up for this list at oss.software.ibm.com/developerworks/mail/?group_id=91 You can also send a note directly to the GPFS development team at gpfs@us.ibm.com. However, this mailing list is informally monitored as time permits and should not be used for priority messages to the GPFS team. If your query concerns a potential software error in GPFS and you have an IBM software maintenance contract, then please contact 1-800-IBM-SERV in the United States or your local IBM Service Center in other countries. IBM Scholars Program users should notify the GPFS development team of potential software bugs via gpfs@us.ibm.com. Note: This FAQ is continually being enhanced. Please contribute questions or answers by sending them to gpfs@us.ibm.com

21

Notices
This information was developed for products and services offered in the U.S.A. IBM may not offer the products, services, or features discussed in this document in other countries. Consult your local IBM representative for information on the products and services currently available in your area. Any reference to an IBM product, program, or service is not intended to state or imply that only IBMs product, program, or service may be used. Any functionally equivalent product, program, or service that does not infringe any of IBMs intellectual property rights may be used instead. However, it is the users responsibility to evaluate and verify the operation of any non-IBM product, program, or service. IBM may have patents or pending patent applications covering subject matter described in this document. The furnishing of this document does not give you any license to these patents. You can send license inquiries, in writing, to: IBM Director of Licensing IBM Corporation North Castle Drive Armonk, NY 10594-1785 USA For license inquiries regarding double-byte (DBCS) information, contact the IBM Intellectual Property Department in your country or send inquiries, in writing, to: IBM World Trade Asia Corporation Licensing 2-31 Roppongi 3-chome, Minato-ku Tokyo 106, Japan The following paragraph does not apply to the United Kingdom or any other country where such provisions are inconsistent with local law: INTERNATIONAL BUSINESS MACHINES CORPORATION PROVIDES THIS PUBLICATION AS IS WITHOUT WARRANTY OF ANY KIND, EITHER EXPRESS OR IMPLIED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF NON-INFRINGEMENT, MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE. Some states do not allow disclaimer of express or implied warranties in certain transactions, therefore, this statement may not apply to you. This information could include technical inaccuracies or typographical errors. Changes are periodically made to the information herein; these changes will be incorporated in new editions of the publication. IBM may make improvements and/or changes in the product(s) and/or the program(s) described in this publication at any time without notice. Any references in this information to non-IBM Web sites are provided for convenience only and do not in any manner serve as an endorsement of those Web sites. The materials at those Web sites are not part of the materials for this IBM product and use of those Web sites is at your own risk. IBM may use or distribute any of the information you supply in any way it believes appropriate without incurring any obligation to you. Licensees of this program who wish to have information about it for the purpose of enabling: (i) the exchange of information between independently created programs and other programs (including this one) and (ii) the mutual use of the information which has been exchanged, should contact: IBM Corporation Intellectual Property Law 2455 South Road,P386 Poughkeepsie, NY 12601-5400 USA Such information may be available, subject to appropriate terms and conditions, including in some cases, payment or a fee. The licensed program described in this document and all licensed material available for it are provided by IBM under terms of the IBM Customer Agreement, IBM International Program License Agreement or any equivalent agreement between us. Any performance data contained herein was determined in a controlled environment. Therefore, the results obtained in other operating environments may vary significantly. Some

22

measurements may have been made on development-level systems and there is no guarantee that these measurements will be the same on generally available systems. Furthermore, some measurements may have been estimated through extrapolation. Actual results may vary. Users of this document should verify the applicable data for their specific environment. This information contains examples of data and reports used in daily business operations. To illustrate them as completely as possible, the examples include the names of individuals, companies, brands, and products. All of these names are fictitious and any similarity to the names and addresses used by an actual business enterprise is entirely coincidental. COPYRIGHT LICENSE: This information contains sample application programs in source language, which illustrates programming techniques on various operating platforms. You may copy, modify, and distribute these sample programs in any form without payment to IBM, for the purposes of developing, using, marketing or distributing application programs conforming to the application programming interface for the operating platform for which the sample programs are written. These examples have not been thoroughly tested under all conditions. IBM, therefore, cannot guarantee or imply reliability, serviceability, or function of these programs. You may copy, modify, and distribute these sample programs in any form without payment to IBM for the purposes of developing, using, marketing, or distributing application programs conforming to IBMs application programming interfaces. If you are viewing this information softcopy, the photographs and color illustrations may not appear.

v v v v v v v v v v v v v v v v v

HACMP IBM IBMLink LoadLeveler Netfinity pSeries POWER PowerPC RS/6000 Redbooks SANergy Scalable POWERParallel Systems ServerProven SP Tivoli TotalStorage xSeries

Dynamic Link Manager, Lightning 9900, Thunder 9200, and Thunder 9500 are trademarks of Hitachi Data Systems Corporation. Intel, Intel Inside (logos), MMX and Pentium are trademarks of Intel Corporation in the United States, other countries, or both. Java and all Java-based trademarks and logos are trademarks or registered trademarks of Sun Microsystems, Inc. in the United States, other countries, or both. Linux is a trademark of Linus Torvalds in the United States, other countries, or both. Microsoft, Windows, Windows NT, and the Windows logo are trademarks of Microsoft Corporation in the United States, other countries, or both. Red Hat, the Red Hat Shadow Man logo, and all Red Hat-based trademarks and logos are trademarks or registered trademarks of Red Hat, Inc., in the United States and other countries. UNIX is a registered trademark of the Open Group in the United States and other countries. Other company, product, and service names may be the trademarks or service marks of others.

Trademarks
The following terms are trademarks of the International Business Machines Corporation in the United States or other countries or both: v AFS v AIX v AIX 5L v Eserver v Enterprise Storage Server

23

October 2004 Copyright International Business Machines Corporation 2003, 2004. All rights reserved. US Government Users Restricted Rights Use, duplication or disclosure restricted by GSA ADP Schedule Contract with IBM Corp.