Installing and Configuring MC/Service Guard 1.

Install MC/Service Guard from the CD with the swinstall command on all the servers that will be in the cluster. 2. A directory called cmcluster is created under /etc on the servers during the installation. 3. Create a file /etc/cmcluster/cmclnodelist on each one of the servers and put there the names of all the servers that belong to the cluster and who can get permissions for them (you can use the /.rhosts file instead). 4. Create all the volume groups, logical volumes and file systems that belong to the cluster on the primary node(s). 5. In order for the adoptive nodes to recognize these volume groups do the following: on the primary node: vgexport -p -m vg01.mapfile -s -v /dev/vg01 (for every volume group) copy the map files to the adoptive nodes. on the adoptive node: mkdir /dev/vg01 mknod /dev/vg01/group c 64 0x010000 (it must be the same minor number as in the primary node) vgimport -s -v -m vg01.mapfile /dev/vg01 6. Create the cluster configuration file on the primary node: cd /etc/cmcluster cmquerycl C <cluster_name>.conf n <primary_node> n <adoptive_node> 7. Modify the file <cluster_name>.conf that was created. You can find an example for a cluster configuration file in appendix A. 8. Check if the configuration is correct cmcheckconf C /etc/cmcluster/<cluster name>.conf 9. Create and distribute the cluster binary file cmapplyconf C /etc/cmcluster/<cluster name>.conf 10. Bring up the cluster cmruncl v 11. Create a directory for each package on all the nodes mkdir /etc/cmcluster/<pkg_name> 12. Create a template of the package configuration file on the primary node: cd /etc/cmcluster/<pkg_name> cmmakepkg p <pkg_name>.conf 13. Modify the file <pkg_name>.conf that was created. You can find an example for a package configuration file in appendix B. 14. Create a template of the package control file on the primary node: cmmakepkg s <pkg_name>.cntl 15. Modify the file <pkg_name>.cntl that was created. You can find an example for a package control file in appendix C. 16. Distribute the control script <pkg_name>.cntl to all the nodes in the cluster using ftp/rcp

17. Check the package configuration file cmcheckconf C /etc/cmcluster/<cluster_name>.conf P /etc/cmcluster/<pkg_name>/<pkg_name>.conf 18. Apply the package configuration file cmapplyconf C /etc/cmcluster/<cluster_name>.conf P /etc/cmcluster/<pkg_name>/<pkg_name>.conf

* There is no need to use /etc/fstab file for file systems that are a part of the cluster. modify the /etc/lvmrc file: set AUTO_VG_ACTIVATE variable to 0 specify the local volume groups that are not associated with the MC/Service Guard and that needs to be mounted at boot time in the function custom_vg_activation.

Maintenance Add a new volume group to a package (package needs to be halted): on the primary node: 1. pvcreate <new disks> 2. mkdir /dev/vg_name 3. mknod /dev/vg_name/group c 64 0x0#0000 4. vgcreate /dev/vg_name <new disks> 5. Create all the logical volumes and file systems. 6. Update the package control file and distribute it to the other nodes. 7. vgexport p m <map_file> -s v /dev/vg_name 8. Copy the map file to the adoptive nodes. 9. vgchange c y /dev/vg_name 10. Umount all the file systems on this volume group. 11. vgchange a n /dev/vg_name On the adoptive node: 1. mkdir /dev/vg_name 2. mknod /dev/vg_name/group c 64 0x0#0000 (use the same minor number!!) 3. vgimport m <map_file> -s v /dev/vg_name On the primary node: 12. cmhaltpkg <pkg_name> 13. cmrunpkg <pkg_name> 14. cmmodpkg e <pkg_name> Add logical volume to an existing volume group of a package (package needs to be halted): On the primary node: 1. lvcreate L <size> /dev/vg_name 2. newfs F vxfs o largefiles /dev/vg_name/lv_name 3. mkdir /mount_point 4. vgexport p m <map file> -s -v /dev/vg_name. 5. Copy the map file to the adoptive nodes. On the adoptive node: 1. vgexport /dev/vg_name 2. mkdir /mount_point 3. mkdir /dev/vg_name 4. mknod /dev/vg_name/group c 64 0x0#0000 (use the same minor number!!) 5. vgimport m <map_file> -s v /dev/vg_name On the primary node: 6. cmhaltpkg <pkg_name> 7. Update the package control file and distribute it to the adoptive nodes. 7. cmrunpkg <pkg_name> 8. cmmodpkg e <pkg_name> * There is no need to update the /etc/fstab file.

Add disk to a volume group of a package (on line): on the primary node: 1. pvcreate /dev/rdsk/c#t#d# 2. vgextend /dev/vg_name /dev/dsk/c#t#d# 3. vgexport p m <map file> -s v /dev/vg_name 4. Copy the new map file to the adoptive nodes. on the adoptive node: 1. vgexport /dev/vg_name 2. mkdir /dev/vg_name 3. mknod /dev/vg_name/group c 64 0x0#0000 (use the same minor number!!) 4. vgimport m <map_file> -s v /dev/vg_name Add a new package to an existing cluster (on line): Follow steps 11-18 from the section Installing and Configuring MC/Service Guard. On section 17,18 dont use the `C /etc/cmcluster/<cluster_name>.conf ` flag so you can add the package on-line. Remove a package from an existing cluster (on line): 1. Halt the package cmhaltpkg <pkg_name> 2. Delete the package configuration: cmdeleteconf v p <pkg_name> Add a new node to an existing cluster (on line): 1. Gather current cluster configuration: cmgetconf v c <cluster_name> temp.conf 2. Gather information about the new node: cmquerycl n node1 n node2 n new_node C <cluster_name>.conf. 3. Compare between these two files and update the cluster configuration file. 4. Verify the new configuration: cmcheckconf C <cluster_name>.conf 5. Apply the new configuration: cmapplyconf C <cluster_name>.conf 6. Start the new node: cmrunnode <new_node> Remove a node from an existing cluster (on line): 1. Make sure to reconfigure the packages so they will not fail over to this node. 2. Halt the node: cmhaltnode <node_name> 3. Edit the cluster configuration file so it will not include this node. 4. Verify the new configuration: cmcheckconf C <cluster_name>.conf 5. Apply the new configuration: cmapplyconf C <cluster_name>.conf

Monitor the cluster state: 1. cmviewcl -v: information about the cluster confuiguration, where the packages currently run, whether the application is up and running, etc. 2. /etc/cmcluster/<pkg_name>/<pkg_name>.cntl.log: a log file for each package that shows status about starting up and shutting down the package. 3. /var/adm/syslog/syslog.log: the system log file. 4. /var/symapi/log/: error messages from the symAPI. Manual operations after failover situation: 1. Make sure the primary node is up and running again. 2. Join the node to the cluster: cmrunnode <primary_node> 3. Halt the package on the adoptive node: cmhaltpkg n <adoptive_node> <pkg_name> 4. Run the package on the primary node: cmrunpkg n <primary_node> <pkg_name> 5. Enable the package: cmmodpkg e <pkg_name>