Installing and Maintaining An H CP System

Hitachi Content Platform
Installing and Maintaining an HCP System
FASTFIND LINKS Document Organization Product Version Getting Help Table of Contents
FE-98ARC018-07
Copyright 20082011 Hitachi Data Systems Corporation, ALL RIGHTS RESERVED No part of this publication may be reproduced or transmitted in any form or by any means, electronic or mechanical, including photocopying and recording, or stored in a database or retrieval system for any purpose without the express written permission of Hitachi Data Systems Corporation (hereinafter referred to as Hitachi Data Systems). Hitachi Data Systems reserves the right to make changes to this document at any time without notice and assumes no responsibility for its use. This document contains the most current information available at the time of publication. When new and/or revised information becomes available, this entire document will be updated and distributed to all registered users. Some of the features described in this document may not be currently available. Refer to the most recent product announcement or contact your local Hitachi Data Systems sales office for information about feature and product availability. Notice: Hitachi Data Systems products and services can be ordered only under the terms and conditions of the applicable Hitachi Data Systems agreement(s). The use of Hitachi Data Systems products is governed by the terms of your agreement(s) with Hitachi Data Systems. By using this software, you agree that you are responsible for: a) Acquiring the relevant consents as may be required under local privacy laws or otherwise from employees and other individuals to access relevant data; and b) Ensuring that data continues to be held, retrieved, deleted, or otherwise processed in accordance with relevant laws. Hitachi is a registered trademark of Hitachi, Ltd. in the United States and other countries. Hitachi Data Systems is a registered trademark and service mark of Hitachi, Ltd. in the United States and other countries. All other trademarks, service marks, and company names are properties of their respective owners.
Contents
Preface........................................................................................................ xi
Intended audience . . . . Product version . . . . . . Document organization . Syntax notation . . . . . . Related documents. . . . Getting help. . . . . . . . . Comments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ........... ........... ........... ........... ........... ........... ........... . . . . . . . . . . . . . . . . . . . . . .... .... .... .... .... .... .... . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .xi . .xi . xii . xiii . xiv xvii xvii
HCP overview.....................................................................................1-1
Introduction to Hitachi Content Platform. . . . . . . . . . . . . . . HCP hardware . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Nodes and storage . . . . . . . . . . . . . . . . . . . . . . . . . . . Storage nodes . . . . . . . . . . . . . . . . . . . . . . . . . . . . Search nodes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Networking . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . System architecture . . . . . . . . . . . . . . . . . . . . . . . . . . HCP features that depend on hardware configuration. . . HCP software . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . HCP System Management Console . . . . . . . . . . . . . . . . . . . HCP service activities . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1-2 . 1-4 . 1-5 . 1-5 . 1-5 . 1-6 . 1-6 . 1-9 .1-10 .1-11 .1-11
Gathering installation information .......................................................2-1

Information for an OS installation . . . . . . . . . . . . . . . . . Information for an HCP software installation . . . . . . . . . . Distributor key . . . . . . . . . . . . . . . . . . . . . . . . . . . . Networking . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Domain name system . . . . . . . . . . . . . . . . . . . . . . . Time . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. .. .. .. .. .. . . . . . . .... .... .... .... .... .... . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2-2 2-3 2-3 2-4 2-4 2-5
Contents Installing and Maintaining an HCP System
iii
Internals . . . . . . . . . . . . . . . . . . . . . Security . . . . . . . . . . . . . . . . . . . . . . HCP nodes . . . . . . . . . . . . . . . . . . . . Post-installation configuration information
. . . .
. . . .
. . . .
. . . .
.... .... .... ....
. . . .
.. .. .. ..
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
2-6 2-8 2-9 2-9
Installing an HCP system ...................................................................3-1

Installing the Appliance Operating System . . . . . . . . . . . . . . Resources you need for an OS installation . . . . . . . . . . . Before you begin the OS installation . . . . . . . . . . . . . . . . Optionally reviewing the storage configuration . . . . . . . . Performing the OS installation . . . . . . . . . . . . . . . . . . . . Installing the HCP software . . . . . . . . . . . . . . . . . . . . . . . . . HCP Setup program . . . . . . . . . . . . . . . . . . . . . . . . . . . Using the HCP Setup wizard menus . . . . . . . . . . . . . . Specifying configuration options. . . . . . . . . . . . . . . . . Saving the system configuration. . . . . . . . . . . . . . . . . Resetting configuration options to their default values . Running HCP Setup from a service shell . . . . . . . . . . . Resources you need for an HCP software installation . . . . Before you begin the HCP software installation . . . . . . . . Performing the HCP software installation . . . . . . . . . . . . Step 1: Log in for the HCP software installation . . . . . Step 2: Retrieve the HCP software installation files . . . Step 3: Configure the HCP system. . . . . . . . . . . . . . . Step 4: Execute the installation. . . . . . . . . . . . . . . . . Verifying the HCP installation. . . . . . . . . . . . . . . . . . . . . Verifying the installation from the System Management Verifying the installation from the HCP Service menu . . Setting additional configuration options . . . . . . . . . . . . . Installing the HCP software advanced options . . . . . . . . . . Using a node-list file . . . . . . . . . . . . . . . . . . . . . . . . . . . Step 1: Create the node-list file. . . . . . . . . . . . . . . . . Step 2: Start the installation procedure . . . . . . . . . . . Step 3: Identify the node-list file . . . . . . . . . . . . . . . . Using a modified HCP configuration file . . . . . . . . . . . . . Step 1: Do the basic configuration of the HCP system . Step 2: Modify the configuration file . . . . . . . . . . . . . Step 3: Restart the installation procedure . . . . . . . . . Step 4: Load the modified configuration file . . . . . . . . ....... ....... ....... ....... ....... ....... ....... ....... ....... ....... ....... ....... ....... ....... ....... ....... ....... ....... ....... ....... Console . ....... ....... ....... ....... ....... ....... ....... ....... ....... ....... ....... ....... . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3-2 . 3-2 . 3-2 . 3-3 . 3-4 . 3-6 . 3-7 . 3-7 . 3-8 . 3-9 . 3-9 . 3-9 .3-11 .3-11 .3-12 .3-12 .3-15 .3-16 .3-19 .3-20 .3-21 .3-23 .3-26 .3-26 .3-27 .3-27 .3-28 .3-30 .3-32 .3-32 .3-33 .3-33 .3-35
Upgrading an HCP system .................................................................4-1

About upgrading . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4-2
iv
Online and offline upgrades. . . . . . . . . . . . . . . . . . . . . . . . Considerations for online upgrades. . . . . . . . . . . . . . . . Monitoring HCP during an online upgrade . . . . . . . . . . . Before you begin an HCP upgrade . . . . . . . . . . . . . . . . . . . Information you need for an upgrade . . . . . . . . . . . . . . Resources you need for an upgrade . . . . . . . . . . . . . . . Running upgrade prechecks . . . . . . . . . . . . . . . . . . . . Freeing space on the HCP nodes . . . . . . . . . . . . . . . . . Preparing the HCP system for an upgrade. . . . . . . . . . . . . . Performing the upgrade . . . . . . . . . . . . . . . . . . . . . . . . . . Step 1: Log in for the upgrade . . . . . . . . . . . . . . . . . . Step 2: Retrieve the files for the upgrade. . . . . . . . . . . Step 3: Set up the upgrade . . . . . . . . . . . . . . . . . . . . Step 4: Execute the upgrade . . . . . . . . . . . . . . . . . . . Step 5: Verify the upgrade . . . . . . . . . . . . . . . . . . . . . Step 6: Complete the upgrade . . . . . . . . . . . . . . . . . . Step 7 (conditional): Resume replication . . . . . . . . . . . Restarting an upgrade . . . . . . . . . . . . . . . . . . . . . . . . . . . Rolling back an offline upgrade . . . . . . . . . . . . . . . . . . . . . Resources you need for an upgrade rollback . . . . . . . . . Performing the upgrade rollback . . . . . . . . . . . . . . . . . Step 1: Log in for the upgrade rollback . . . . . . . . . . Step 2: Execute the upgrade rollback . . . . . . . . . . . Step 3: Verify the upgrade rollback . . . . . . . . . . . . . Installing a hotfix . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Resources you need for a hotfix installation . . . . . . . . . Performing the hotfix installation . . . . . . . . . . . . . . . . . Step 1: Log in for the hotfix installation . . . . . . . . . . Step 2: Execute the hotfix installation . . . . . . . . . . . Step 3: Verify the hotfix installation. . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. 4-3 . 4-3 . 4-5 . 4-5 . 4-6 . 4-6 . 4-7 .4-14 .4-16 .4-17 .4-17 .4-19 .4-22 .4-23 .4-24 .4-25 .4-27 .4-27 .4-29 .4-30 .4-30 .4-30 .4-31 .4-32 .4-32 .4-33 .4-33 .4-34 .4-34 .4-36
Adding nodes to an HCP system........................................................5-1

About adding nodes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Adding search nodes. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Adding nodes in SAIN systems with CB 320 servers . . . . . . . . . . Before you begin a node addition. . . . . . . . . . . . . . . . . . . . . . . . . . Information you need for a node addition . . . . . . . . . . . . . . . . . Resources you need for a node addition . . . . . . . . . . . . . . . . . . IP addresses for new nodes. . . . . . . . . . . . . . . . . . . . . . . . . . . Performing the node addition . . . . . . . . . . . . . . . . . . . . . . . . . . . . Step 1: Install the OS on the new nodes . . . . . . . . . . . . . . . . . Step 2: Log in for the node addition . . . . . . . . . . . . . . . . . . . . Step 3 (conditional): Retrieve the HCP software installation files . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5-2 5-3 5-4 5-4 5-5 5-6 5-7 5-7 5-8 5-8 5-9
Step 4: Set up the node addition. . . . . . . . . . . . . . . . . . . . . . . . . . . . . .5-11 Step 5: Execute the node addition. . . . . . . . . . . . . . . . . . . . . . . . . . . . .5-12 Verifying the node addition . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .5-13
Maintaining HCP nodes......................................................................6-1

Recovering an existing storage node . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6-2 Before you begin a storage node recovery . . . . . . . . . . . . . . . . . . . . . . . 6-3 Information you need for a storage node recovery. . . . . . . . . . . . . . . . 6-3 Resources you need for a storage node recovery . . . . . . . . . . . . . . . . . 6-4 Recovering the OS on a storage node. . . . . . . . . . . . . . . . . . . . . . . . . . . 6-5 Step 1: Reinstall the Appliance OS. . . . . . . . . . . . . . . . . . . . . . . . . . . 6-6 Step 2: Log in for the OS recovery . . . . . . . . . . . . . . . . . . . . . . . . . . 6-6 Step 3 (conditional): Retrieve the installation files for the OS recovery . 6-7 Step 4: Execute the OS recovery . . . . . . . . . . . . . . . . . . . . . . . . . . . .6-10 Recovering all the logical volumes on a storage node. . . . . . . . . . . . . . . .6-12 Step 1: Prepare for the logical-volume recovery . . . . . . . . . . . . . . . . .6-12 Step 2: Log in for the logical-volume recovery . . . . . . . . . . . . . . . . . .6-13 Step 3 (conditional): Retrieve the installation files for the logical-volume recovery . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .6-14 Step 4: Execute the logical-volume recovery . . . . . . . . . . . . . . . . . . .6-16 Verifying a storage node recovery . . . . . . . . . . . . . . . . . . . . . . . . . . . . .6-18 Recovering an existing search node . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .6-19 Before you begin a search node recovery . . . . . . . . . . . . . . . . . . . . . . . .6-19 Resources you need for a search node recovery . . . . . . . . . . . . . . . . . . .6-20 Performing the search node recovery . . . . . . . . . . . . . . . . . . . . . . . . . . .6-21 Step 1: Reinstall the OS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .6-21 Step 2: Log in for the search node recovery . . . . . . . . . . . . . . . . . . . .6-21 Step 3 (conditional): Retrieve the HCP software installation files . . . . .6-22 Step 4: Execute the search node recovery . . . . . . . . . . . . . . . . . . . . .6-23 Verifying the search node recovery . . . . . . . . . . . . . . . . . . . . . . . . . . . .6-25
Managing storage in an HCP SAIN system........................................7-1

Adding storage to an HCP SAIN system . . . . . . . . . . . . . . . . . . . . . . . Before you begin a storage addition . . . . . . . . . . . . . . . . . . . . . . . Information you need for a storage addition . . . . . . . . . . . . . . . Resources you need for a storage addition . . . . . . . . . . . . . . . . Performing the storage addition . . . . . . . . . . . . . . . . . . . . . . . . . . Step 1: Configure the logical volumes at the storage tier . . . . . . Step 2: Prepare the HCP system for the storage addition. . . . . . Step 3: Log in for the storage addition. . . . . . . . . . . . . . . . . . . Step 4 (conditional): Retrieve the HCP software installation files Step 5: Make the new logical volumes known to HCP . . . . . . . . Step 6: Complete the storage addition . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7-2 . 7-2 . 7-3 . 7-3 . 7-3 . 7-4 . 7-4 . 7-4 . 7-5 . 7-8 .7-10
vi
Verifying the storage addition . . . . . . . . . . . . . . . . . . . . . . . . . Reconfiguring HCP to use a new or different storage array . . . . . . . . Using TrueCopy to migrate data. . . . . . . . . . . . . . . . . . . . . . . . Step 1: Prepare the new array . . . . . . . . . . . . . . . . . . . . . . Step 2: Use TrueCopy to migrate the data . . . . . . . . . . . . . . Step 3: Reconfigure the SAN fabrics . . . . . . . . . . . . . . . . . . Step 4: Update the HCP nodes to recognize the new array . . Step 5: Power on the nodes . . . . . . . . . . . . . . . . . . . . . . . . Step 6: Reenable zero-copy failover . . . . . . . . . . . . . . . . . . Virtualizing an existing storage array behind an enterprise array . Step 1: Prepare the storage arrays for the virtualization . . . . Step 2: Prepare the HCP system for the virtualization . . . . . . Using Tiered Storage Manager to migrate data . . . . . . . . . . . . .
. . . . . . . . . . . . .
. . . . . . . . . . . . .
. . . . . . . . . . . . .
. . . . . . . . . . . . .
. . . . . . . . . . . . .
. . . . . . . . . . . . .
.7-10 .7-11 .7-12 .7-13 .7-13 .7-13 .7-14 .7-15 .7-15 .7-15 .7-16 .7-17 .7-18
Retiring devices ..................................................................................8-1

Finalizing a data migration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8-2 Before you begin finalizing a migration . . . . . . . . . . . . . . . . . . . . . . . . . . 8-3 Resources you need for finalizing a migration . . . . . . . . . . . . . . . . . . . . . 8-4 Finalizing the migration and removing the retired devices. . . . . . . . . . . . . 8-4 Step 1: Log in for the migration finalization . . . . . . . . . . . . . . . . . . . . 8-5 Step 2: Execute the migration finalization . . . . . . . . . . . . . . . . . . . . . 8-6 Step 3: Remove the retired devices from the HCP system . . . . . . . . . . 8-7 Step 4 (SAIN systems only; conditional): Reinstall the Appliance OS on the affected search nodes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8-8 Step 5 (SAIN systems only; conditional): Update the fibre channel HBA firmware and reboot the nodes . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8-8 Step 6 (SAIN systems only; conditional): Rebuild the search index . . . . 8-8 Removing nodes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8-9
Reconfiguring HCP system software..................................................9-1

Changing DNS settings . . . . . . . . . . . . . . . . . . . . . . . . . . Changing network settings . . . . . . . . . . . . . . . . . . . . . . . Adding and removing chassis IP addresses . . . . . . . . . . . . Changing time settings . . . . . . . . . . . . . . . . . . . . . . . . . . Changing the HCP serial number . . . . . . . . . . . . . . . . . . . Changing the HTTP persistent connection timeout interval . Enabling or disabling zero-copy failover . . . . . . . . . . . . . . Enabling or disabling DPL one in RAIN systems . . . . . . . . . Changing the region count . . . . . . . . . . . . . . . . . . . . . . . About the region count . . . . . . . . . . . . . . . . . . . . . . . Changing or canceling a change to the region count . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9-2 . 9-4 . 9-7 . 9-8 . 9-9 .9-10 .9-10 .9-11 .9-12 .9-12 .9-13
vii
Enabling or disabling support for replication . . . . . . . . . . . . . . . . . . . . . . . . .9-16 Enabling support for replication . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .9-16 Disabling support for replication. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .9-17
10 Managing an HCP system................................................................10-1

Managing HCP services . . . . . . . . . . . . . . . . . . . . . . . . . . . Service management . . . . . . . . . . . . . . . . . . . . . . . . . Starting, stopping, enabling, and disabling services . . . . Resetting the checkpoint for a replication link . . . . . . . . . . . Disabling chained writes . . . . . . . . . . . . . . . . . . . . . . . . . . Accessing the HCP search facility administrative interface. . . . . . . . . . . . . . . . . . . . . .. .. .. .. .. .. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .10-2 .10-2 .10-3 .10-4 .10-5 .10-6
11 Troubleshooting an HCP system......................................................11-1

Understanding advanced alerts . . . . . . . . . . . . . . . . . . . . . Retrieving the HCP Setup logs . . . . . . . . . . . . . . . . . . . . . . Resetting the install user password . . . . . . . . . . . . . . . . . . Resetting a security user password . . . . . . . . . . . . . . . . . . Managing the HCP internal logs . . . . . . . . . . . . . . . . . . . . . HCP component log files . . . . . . . . . . . . . . . . . . . . . . . Server system event log . . . . . . . . . . . . . . . . . . . . . . . Changing logging levels . . . . . . . . . . . . . . . . . . . . . . . Logging levels . . . . . . . . . . . . . . . . . . . . . . . . . . . . Changing the logging level for C components . . . . . . Changing the logging level for the JVM . . . . . . . . . . . Downloading the logs for a single node . . . . . . . . . . . . Managing encryption keys for log downloads . . . . . . . . Creating a snapshot of the JVM . . . . . . . . . . . . . . . . . . . . . Using the system service tools . . . . . . . . . . . . . . . . . . . . . . cluster_ssh . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . cluster_get. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . cluster_put . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Viewing the system status pages . . . . . . . . . . . . . . . . . . . . Ejecting the CD tray from a node . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .11-2 . .11-3 . .11-3 . .11-4 . .11-5 . .11-6 . .11-8 . .11-8 . .11-8 . .11-9 . 11-10 . 11-12 . 11-13 . 11-14 . 11-15 . 11-17 . 11-24 . 11-29 . 11-32 . 11-33
Forms ................................................................................................ A-1

Appliance OS Installation Information. HCP Software Installation Information Encryption Key. . . . . . . . . . . . . . . . . HCP Node Addition Information . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. .. .. .. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . A-2 A-3 A-5 A-6
viii
HCP Setup wizard menus ................................................................. B-1

Installation menus . . . . . . Upgrade menu . . . . . . . . . Node addition menu . . . . . Storage addition menu . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. .. .. .. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . B-2 B-4 B-4 B-5
C D
HCP configuration file........................................................................ C-1 Setting up SSH key authentication .................................................... D-1
Adding and forwarding an SSH key in Windows. . . . . . . . . . . . . . . . . . . . . . . D-2 Adding and forwarding an SSH key in Linux . . . . . . . . . . . . . . . . . . . . . . . . . D-3
Accessing HCP interfaces through the back-end network ................ E-1

Connecting to the back-end network . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . E-2 Accessing the interfaces . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . E-2
Glossary Index
ix
Preface
This book is the service guide for Hitachi Content Platform (HCP). It contains complete information for installing and upgrading HCP system software. It also provides instructions for system reconfiguration, management, and troubleshooting procedures not available to customers. This book focuses on the HCP software. For information on maintaining the hardware for an HCP system, see the applicable HCP hardware maintenance documentation. Note: Throughout this book, the word Unix is used to represent all UNIXlike operating systems (such as UNIX itself or Linux), except where Linux is specifically required.
Intended audience
This book is for Hitachi Data Systems (HDS) service and support personnel and for HDS partners and authorized resellers who are responsible for installing and upgrading the HCP software and providing the first tier of customer support. It assumes you are familiar with basic HCP concepts and administrative procedures. It also assumes you have at least a couple of years of experience working with computer systems, including using command-line interfaces.
Product version
This book applies to release 4.1 of HCP.
Preface Installing and Maintaining an HCP System
xi
Document organization
Document organization
This book contains eleven chapters and five appendixes.
Chapter/Appendix
Chapter 1, HCP overview
Description
Contains an overview of HCP hardware and software, as well as an outline of the HCP service provider responsibilities described in this book Describes the information you need to perform a fresh installation of HCP Contains complete instructions for installing an HCP system Contains complete instructions for upgrading an HCP system Contains instructions for the software portion of the procedure for adding nodes to an HCP system Contains instructions for recovering nodes with corrupted file systems Contains instructions for the software portion of the procedure for adding storage to an HCP SAIN system Provides instructions for retiring nodes and storage arrays from HCP systems Provides instructions for various system reconfiguration procedures, most of which you perform through the HCP System Management Console Provides instructions for various system management procedures, most of which you perform through the HCP System Management Console Explains procedures that can help identify or repair problems with an HCP system Contains forms you can use to record the information you gather for an OS or HCP software installation, system upgrade, or node addition; also contains a form for recording the encryption key displayed during a core software installation Shows the menus you use when installing, upgrading, or adding nodes to an HCP system
Chapter 2, Gathering installation information Chapter 3, Installing an HCP system Chapter 4, Upgrading an HCP system Chapter 5, Adding nodes to an HCP system Chapter 6, Maintaining HCP nodes Chapter 7, Managing storage in an HCP SAIN system Chapter 8, Retiring devices Chapter 9, Reconfiguring HCP system software
Chapter 10, Managing an HCP system
Chapter 11, Troubleshooting an HCP system Appendix A, Forms
Appendix B, HCP Setup wizard menus
xii
Syntax notation
(Continued)
Chapter/Appendix
Appendix C, HCP configuration file
Description
Describes all the configuration parameters used in the installation or upgrade of an HCP system Explains how to add SSH keys to a keychain Explains how to access the HCP Configuration menu, the HCP System and Tenant Management Consoles, and the Namespace Browser through the backend network
Appendix D, Setting up SSH key authentication Appendix E, Accessing HCP interfaces through the back-end network
Syntax notation
The table below describes the conventions used for the syntax of commands, expressions, URLs, and object names in this book.
Notation boldface Meaning
Type exactly as it appears in the syntax (if the context is case insensitive, you can vary the case of the letters you type) Replace with a value of the indicated type Vertical bar Choose one of the elements on either side of the bar, but not both Square brackets Include none, one, or more of the elements between the brackets Parentheses Include exactly one of the elements between the parentheses Replace with the combination of the directory path and name of a file
Example
This book shows: --version You enter: --version
italics |
This book shows: node-ip-address You enter: 192.168.210.16 This book shows: -h|--help You enter: -h or: --help This book shows: [--unique-ify] You enter: --unique-ify or nothing This book shows: (-u|--user) username You enter: -u root or: --user root This book shows: source-file-spec You enter: /opt/arc/build.version
[ ]
( )
-file-spec
xiii
Related documents
(Continued)
Notation -path
Meaning
Replace with a directory path with no file or object name
Example
This book shows: destination-path You enter: /opt/arc/tools/cluster/temp
Related documents
The following documents contain additional information about Hitachi Content Platform:
Administering HCP This book explains how to use an HCP system to

monitor and manage a digital object repository. It discusses the capabilities of the system, as well as its hardware and software components. The book presents both the concepts and instructions you need to configure the system, including creating the tenants that administer access to the repository. It also covers the processes that maintain the integrity and security of the repository contents.
Managing a Tenant and Its Namespaces This book contains complete

information for managing the HCP tenants and namespaces created in an HCP system. It provides instructions for setting up both administrative user accounts and data access accounts, configuring the HTTP protocol, which allows access to namespaces, managing search, and downloading installation files for HCP Data Migrator and the HCP client tools. It also explains how to work with retention classes and the privileged delete functionality.
Managing the Default Tenant and Namespace This book contains

complete information for managing the default tenant and namespace in an HCP system. It provides instructions for changing tenant and namespace settings, configuring the protocols that allow access to the namespace, managing search, and downloading installation files for HCP Data Migrator and the HCP client tools. It also explains how to work with retention classes and the privileged delete functionality.
Replicating Tenants and Namespaces This book covers all aspects of

tenant and namespace replication. Replication is the process of copying tenants and namespaces from one HCP system to another to ensure data availability and enable disaster recovery. The book describes how replication works, contains instructions for working with replication links, and explains how to monitor the replication process.
xiv
Related documents
HCP Management API Reference This book contains the information

you need to use the HCP management API. This REST API enables you to create and manage tenants and namespaces programmatically. The book explains how to use the API to access an HCP system, specify resources, and update and retrieve resource properties.
Using a Namespace This book describes the properties of objects in

HCP namespaces. It provides instructions for accessing namespaces by using the HTTP protocol for the purpose of storing, retrieving, and deleting objects, as well as changing object metadata such as retention and shred settings. It also explains how to manage namespace content and view namespace information in a web browser.
Using the Default Namespace This book describes the file system
HCP uses to present the contents of the default namespace. It provides instructions for accessing the namespace by using the HCP-supported protocols for the purpose of storing, retrieving, and deleting objects, as well as changing object metadata such as retention and permissions.
Searching Namespaces This book describes the HCP Search Console.

It explains how to search namespaces for objects that satisfy criteria you specify. It also explains how to manage and manipulate queries and search results. The book contains many examples, which you can use as models for your own searches.
Using HCP Data Migrator This book contains the information you
need to install and use the HCP Data Migrator (HCP-DM) utility distributed with HCP. This utility enables you to copy data between local file systems, HCP namespaces, and earlier HCAP archives. It also supports bulk delete operations. The book describes both the interactive window-based interface and the set of command-line tools included in HCP-DM.
Using the HCP Client Tools This book contains the information you
need to install and use the legacy set of client command-line tools distributed with HCP. These tools enable you to find files and to copy and move files to and from namespaces. The book contains many examples that show command-line details and the overall workflow. Note: For most purposes, the HCP client tools have been superseded by HCP Data Migrator. However, they have some features, such as finding files, that are not available in HCP-DM.
xv
Related documents
Installing an HCP System This book provides the information you

need to install the software for a new HCP system. It explains what you need to know to successfully configure the system and contains step-by-step instructions for the installation procedure.
Third-Party Licenses and Copyrights This book contains copyright

and license information for third-party software distributed with or embedded in HCP.
HCP-DM Third-Party Licenses and Copyrights This book contains

copyright and license information for third-party software distributed with or embedded in HCP Data Migrator.
Installing an HCP 500 System Final On-site Setup This book

contains instructions for deploying an assembled and configured HCP 500 system at a customer site. It explains how to make the necessary physical connections and reconfigure the system for the customer computing environment.
Assembling and Configuring an HCP 300 System This book contains

complete instructions for building an HCP 300 system, from assembling and configuring the hardware to installing the HCP software.
Installing an HCP 300 System Final On-site Setup This book

contains instructions for deploying an assembled and configured HCP 300 system at a customer site. It explains how to make the necessary physical connections and reconfigure the system for the customer computing environment.
Maintaining HCP 300 System Hardware This book is the hardware

maintenance guide for HCP 300 systems. It explains how to replace failed components of an HCP 300 system, as well as how to add new nodes to a system.
xvi
Getting help
Getting help
The Hitachi Data Systems customer support staff is available 24 hours a day, seven days a week. If you need technical support, please call:
United States: (800) 446-0744 Outside the United States: (858) 547-4526
Note: If the customer purchased HCP from a third party, please contact the applicable HCP support center.
Comments
Please send us your comments on this document: hcp.documentation.feedback@hds.com Include the document title, number, and revision, and refer to specific sections and paragraphs whenever possible. Thank you! (All comments become the property of Hitachi Data Systems.)
xvii
Comments
xviii
1
HCP overview
Hitachi Content Platform (HCP) is the distributed, fixed-content, data storage system from Hitachi Data Systems. An HCP system consists of both hardware and software. Once the HCP hardware is set up, the software must be installed, sometimes upgraded or reconfigured, and always managed. The customer is responsible for some management activities. As a service provider, you are responsible for installing, upgrading, and reconfiguring the software, as well as for system management activities that are unavailable to customers. You are also responsible for adding, maintaining, and removing nodes and storage arrays. This chapter reviews some basic HCP concepts and describes the HCP hardware architecture. It also contains an overview of your responsibilities as a service provider.
HCP overview Installing and Maintaining an HCP System
11
Introduction to Hitachi Content Platform

Hitachi Content Platform is a distributed storage system designed to support large, growing repositories of fixed-content data. It provides a cost-effective, scalable, easy-to-use repository that can accommodate all types of data, from simple text files to medical images to multigigabyte database images. A fixed-content storage system is one in which the data cannot be modified. HCP uses write-once, read-many (WORM) storage technology and a variety of policies and services to ensure the integrity of the stored data and the efficient use of storage capacity. What follows is a brief overview of HCP basics. For a more in-depth introduction to HCP, see Administering HCP. Object-based storage HCP stores objects in a repository. Each object permanently associates data HCP receives with information about that data; that is, each object encapsulates both object data and metadata. HCP distributes objects across its storage space but still presents them as files in a standard directory structure. Namespaces An HCP repository is partitioned into namespaces. A namespace is a logical grouping of objects such that the objects in one namespace are not visible in any other namespace. Namespaces are not associated with any preallocated storage. An HCP system can have up to 100 namespaces, including one special one called the default namespace. The default namespace is typically used for legacy applications. New applications are typically written against other namespaces, which are called HCP namespaces. Tenants Namespaces are owned and managed by administrative entities called tenants. A tenant typically corresponds to an organization such as a company or a division or department within a company. A tenant can also correspond to an individual person. An HCP system can have at most 50 tenants. Each tenant, except one, can own multiple namespaces. The exception, called the default tenant, owns the default namespace and only that namespace.
12
Namespace access protocols HCP provides access to namespaces through a variety of industry-standard protocols. For actions such as adding objects to a namespace, viewing and retrieving objects, changing object metadata, and deleting objects, HCP supports the HTTP, WebDAV, CIFS, and NFS protocols. The latter three protocols are supported only for the default namespace. HCP allows special-purpose access to the default namespace through two additional protocols: SMTP (for storing email) and NDMP (for backing up and restoring data). Data access accounts To access the data in an HCP namespace, users and applications must present valid credentials. These credentials are defined by a data access account. The account specifies a username and password. It also specifies which namespaces the user or application can access and which operations the user or application can perform in each of those namespaces. HCP Search Console HCP includes a web application called the Search Console that lets users search for objects based on specified criteria. This Console works with either of two implementations: The HDDS search facility This facility interacts with Hitachi Data Discovery Suite (HDDS), which performs searches and returns results to the HCP Search Console. To use the HDDS search facility, you need to first install and configure HDDS, which is a separate product from HCP. The HCP search facility This facility is integrated with HCP and works internally to perform searches and return results to the Search Console. The HCP search facility is available only in HCP systems that included it before being upgraded to release 4.1.
Only one of the search facilities can be enabled at any given time. If neither one is enabled, the HCP system does not support searching namespaces. The system associated with the enabled search facility is called the active search system. The active search system (that is, HDDS or HCP) maintains an index of data objects in each search-enabled namespace. This index is based on object content and metadata. The active search system uses the index for fast retrieval of search results. When data objects are added to or removed from the namespace or when object metadata changes, the active search system automatically updates the index to keep it current.
13
HCP hardware
For more information on the HDDS and HCP search facilities, see Administering HCP. For information on using the HCP Search Console, see Searching Namespaces. HCP services HCP services are background processes that each perform a specific function that contributes to the continuous tuning of the HCP system. In particular, services are responsible for optimizing the use of system resources and maintaining the integrity and availability of the stored data. Replication An HCP system can be replicated to another HCP system. Replication, an HCP service, is the process of keeping selected tenants and namespaces in two HCP systems in sync with each other. This entails copying object creations, deletions, and metadata changes, as well as tenant and namespace configurations, from one system to the other. For more information on replication, see Replicating Tenants and Namespaces. Note: Replication is an add-on feature to HCP, with a separate purchase price.
HCP hardware
HCP runs on a networked redundant array of independent nodes (RAIN) or a SAN-attached array of independent nodes (SAIN). SAN stands for storage area network. HCP 300 systems use a RAIN configuration. HCP 500 systems use a SAIN configuration. HCP hardware consists of:
Servers Internal, direct-attached, or SAN-attached storage Networking components such as cables and switches Additional infrastructure items such as racks and power distribution
units (PDUs)
14
HCP hardware
Nodes and storage

An HCP system includes multiple servers, called nodes, that are networked together. Each node can have multiple physical drives or connect to external fibre channel storage. (In a SAIN system, the nodes may be blades in one or more Hitachi Compute Blade 320 (CB 320) servers.) At least four of the nodes must be storage nodes. Systems upgraded from an HCP release earlier than 4.1 can also have one or more search nodes.
Storage nodes
Storage nodes are the essential part of HCP. They manage the objects that reside in the internal or external storage. To ensure data integrity and continuous availability in case of a hardware or software failure, HCP uses RAID technology and can also store the data and metadata for each object in multiple locations. Each storage node runs the complete HCP software, with the exception of the HCP search facility software, if present. These nodes, therefore, act both as a repository manager and a gateway that enables access to the data in the repository. Runtime operations are distributed among the storage nodes, thereby ensuring reliability and performance as capacity grows. If a node fails, the HCP system adapts by redirecting processing to other nodes, so the stored data remains available to users.
Search nodes
To support the HCP search facility, which is available only in systems that already included it before being upgraded to release 4.1, an HCP system must include one or more search nodes. Search nodes run the HCP search facility software and store the HCP search index in a distributed manner. The portion of the index on each node is protected by RAID storage technology. (The HDDS search facility index is stored in HDDS.) All runtime HCP search facility processing is distributed among the search nodes, thereby ensuring reliability and performance as indexing and querying needs grow. The HDDS search facility does not uses search nodes.
15
HCP hardware
Networking
HCP has both back-end and front-end networks. The isolated back-end network connects the HCP nodes to each other through back-end Ethernet switches. Caution: The back-end network must remain separate from the front-end network. Interfering with the operation of the back-end network or backend switches can cause the HCP system to become inoperable and can result in data loss. Each storage node is configured with two pairs of bonded Ethernet ports that allow external applications to access the system. The recommended setup includes two independent Ethernet switches that connect these ports to the front-end network.
System architecture
The following figures show the architecture of HCP RAIN and SAIN systems and the relationship between two systems involved in replication. RAIN system architecture The figure below shows the architecture of an HCP system that uses internal storage. This system has four storage nodes.
16
HCP hardware
The table below describes the cables in this figure:

Cable
Red and blue Ethernet Green and yellow Ethernet Purple Ethernet Black power
Connects from
Back-end network interface cards (NICs) in each node Front-end NICs in each node Back-end switches Each node Each back-end switch
Connects to
Back-end switches Front-end switches Each other Two PDUs One PDU
SAIN system architecture The figure below shows the architecture of an HCP system that uses external fibre channel storage. This system has four storage nodes and two modular storage trays.
17
HCP hardware
The table below describes the cables in this figure.

Cable
Red and blue Ethernet Green and yellow Ethernet Purple Ethernet Orange fibre channel Black power
Connects from
Back-end NICs in each node Front-end NICs in each node Back-end switches Each node Each node Each back-end switch Each storage tray
Connects to
Back-end switches Front-end switches Each other External storage Two PDUs One PDU Two PDUs
Notes:
Nodes in HCP SAIN systems are either individual servers or individual

blades in Hitachi Compute Blade 320 (CB 320) servers. The figure above shows the nodes as individual servers.
Some HCP SAIN systems include fibre channel switches between the
nodes and the external storage. Network connections for HCP with replication Replication requires a primary HCP system and a secondary HCP system called a replica. The two systems are connected through the front-end network infrastructure.
18
HCP hardware
For more information on replication, see Replicating Tenants and Namespaces.
HCP features that depend on hardware configuration

Certain features of HCP SAIN systems require specific hardware configurations. Multipathing In an HCP system that uses SAN-attached storage, a node can connect to a storage array through multiple physical paths, effectively creating multiple paths between the node and any given logical volume that maps to it. With this setup, if one component of a physical path connecting such a node to the array fails, the node still has access to the logical volume through another physical path. Multiple means of access to a logical volume from a single node is called multipathing. In the picture below, the red lines represent the physical paths between the node (bottom) and the storage array (top). The broken green lines represent two mappings between one logical volume and the node.
Logical volume
Node
Multipathing is configured at the storage tier. Zero-copy failover Zero-copy failover is the process of one node in an HCP SAIN system automatically taking over management of storage previously managed by another node that has failed. To support zero-copy failover, each logical volume that stores objects must map to two different storage nodes. The pair of nodes forms a set such that the volumes that map to one of the nodes also map to the other. This is called cross-mapping.
19
HCP software
In the picture below, the red lines represent the physical paths between the two nodes (bottom) and the storage array (top). The broken green lines represent the mappings between each of two sets of logical volumes and each of the two nodes.
Logical volume
Logical volume
Node
Node
Cross-mapping is defined at the storage tier. Zero-copy failover is automatically enabled for an HCP SAIN system during installation.
HCP software
HCP system software consists of an operating system (the Appliance Operating System) and HCP software. For maximum reliability, each storage node in an HCP system runs all the HCP software (except the HCP search facility software, which runs on search nodes). This software includes components that:
Handle object data and metadata Ensure that services run as expected Enable you to configure, monitor, and manage the HCP system and the
tenants and namespaces defined in the system
Provide a human-readable interface to HCP system configuration,

activity, and status
Enable access to namespaces using the industry-standard HTTP,

WebDAV, CIFS, NFS, SMTP, and NDMP protocols
Support data transfer to and from the HCP system through the HCP
Data Migrator utility
110
HCP System Management Console
Support searching namespaces through the metadata query API and

the HCP Search Console
HCP System Management Console

The System Management Console is a secure web application that provides interactive access to the HCP system. To use the System Management Console, you need a user account. This account determines what you have permission to do in the Console. A role is a named collection of permissions that can be associated with a user account. Some of the activities in this book require you to have a user account with a specific role. This is noted with the applicable procedures. HCP is installed with one predefined user account. The username and default password for this account are: Username: security Password: Chang3Me! This account includes only the security role. When you first install HCP, you need to either create a new user account with the system role or modify the security account to include the administrator role so you can change the system configuration as needed for the customer site. Note: The password for the security account must be changed the first time someone logs in as security. For more information on the System Management Console, user accounts, and roles, see Administering HCP. Note: When administering a tenant, you use a different web application, called the Tenant Management Console. For information on this Console, see Managing a Tenants and Its Namespaces or Managing the Default Tenant and Namespace.
HCP service activities

As an HCP service provider, youre responsible for seven types of activities, as described below. You are also expected to help customers and your HCP support center with troubleshooting and disaster recovery.
111
Software installation activities To install a new HCP system, you need to: 1. Gather the information you need to perform the installation 2. Install the Appliance Operating System (OS) 3. Learn about the HCP software installation program 4. Install the HCP software 5. Verify the installation 6. Perform additional system configuration activities, as needed For more information on these activities, see Chapter 2, Gathering installation information, and Chapter 3, Installing an HCP system. Software upgrade activities To upgrade an existing HCP system from one major or minor release to another, you need to: 1. Gather the information you need to perform the upgrade 2. Upgrade the HCP software (depending on release requirements, this may include the OS) 3. Verify the upgrade If you run into trouble with an upgrade, you may need to restart it. Or, for an offline upgrade, you may need to roll it back. Additionally, you may occasionally need to apply hotfixes to resolve particular customer needs. For more information on these activities, see Chapter 4, Upgrading an HCP system. Node maintenance activities If a node fails, you may need to recover it. The procedure for this differs for storage and search nodes. For information on these activities, see Chapter 6, Maintaining HCP nodes.
112
System hardware reconfiguration activities To accommodate growing and changing HCP systems, you may need to:
Add nodes to the system to support a greater processing load and, for
RAIN systems, increase storage capacity. For information on this activity, see Chapter 5, Adding nodes to an HCP system.
Add logical volumes to a SAIN system to increase storage capacity. For

information on this activity, see Adding storage to an HCP SAIN system on page 7-2.
Reconfigure a SAIN system to use a new or different storage array. For

information on this activity, see Reconfiguring HCP to use a new or different storage array on page 7-11.
Remove nodes and, for SAIN systems, storage arrays. For information
on this activity, see Chapter 8, Retiring devices. System software reconfiguration activities Occasionally, changes to the HCP configuration become necessary. You may need to:
Change domain name system (DNS) information Change the network configuration Change time settings Change the HCP system serial number Change the HTTP persistent connection timeout interval Enable or disable zero-copy failover Enable or disable DPL one in RAIN systems Change the region count Enable or disable support for the replication feature
For more information on these activities, see Chapter 9, Reconfiguring HCP system software.
113
Service-level system management activities Some HCP management activities are performed by the customer. Others are reserved for service providers. As a service provider, you may need to:
Start, stop, enable, or disable services Reset the checkpoint for a replication link Disable chained writes Access the HCP search facility administrative interface
For more information on these activities, see Chapter 10, Managing an HCP system. Troubleshooting activities Customers can perform some troubleshooting and disaster recovery activities on their own. The ones listed here are reserved for service providers:
Understanding advanced alerts Retrieving the HCP Setup logs Resetting the install user password Resetting a security user password Setting the logging level for the HCP internal logs Downloading the HCP internal logs for a single node Managing encryption keys for log downloads Creating a snapshot of the JVM Using the system service tools Viewing the system status pages Ejecting the CD tray from a node
114
For more information on these activities, see Chapter 11, Troubleshooting an HCP system. Note: As a service provider, you are also responsible for carrying out instructions in service alerts released by the HCP support center.
115
116
2
Gathering installation information
For a new HCP system, the software to be installed consists of an operating system (the Appliance Operating System) and the HCP core software. For a reinstallation of a system that includes search nodes, the software also includes the HCP search facility software. Before you can install the OS and HCP software, you need to know certain information about the system youre creating. For example, you need to know the IP addresses of the HCP nodes, the serial number for the system, and which features the customer wants enabled. The installation information you need comes from both the customer and the HCP distributor. Some of it, such as node IP addresses and serial number, is straightforward. Other information, such as whether to use DNS, requires a customer decision. Most customer decisions are made before the hardware is set up for the HCP system. Once the HCP system is up and running, you may need to do some postinstallation setup. For example, you may need to enable syslog logging or disable the ability to ping the HCP nodes. To do this setup, you need to know what the customer wants. This chapter describes the information you need for installing both the OS and HCP software. It also outlines the information required for any postinstallation setup you may need to do. The descriptions in this chapter are intended to help you with the specifications required by the HCP configuration tool supplied by the HCP distributor. If you dont have such a tool, you can use the forms in Appendix A, Forms, to record the information you collect. You can then keep the completed forms available to facilitate the installation procedure.
Gathering installation information Installing and Maintaining an HCP System
21
Information for an OS installation
Information for an OS installation

To install the Appliance OS for a new HCP system, you need to know about the nodes and the front- and back-end networks for the system. Tip: You can use the Appliance OS Installation Information form on page A-2 to record this information. HCP nodes For a fresh installation of the OS, you need to know which servers at the customer site are supposed to be nodes in the HCP system. For each server, you need to know:
The physical location. The front-end IP address. The subnet mask for the front-end network. The IP address of the default gateway for the front-end network. This
is the same for all nodes.
The back-end IP address.

Once the HCP system is running, you can use the System Management Console to change the node IP addresses and front-end subnet mask you specify during the OS installation. For more information on this, see Changing network settings on page 9-4. Back-end IP addresses All the nodes in an HCP system must be on the same system-specific private back-end IP subnet. This subnet is used only for internode communication. It must not be connected to the corporate network, which is used for other types of data communication at the customer site. For the back-end IP subnet, use the installation default of 10.1.1.x (with the required subnet mask of 255.255.255.0) unless its already in use in the corporate network. If 10.1.1.x is unavailable, choose another private IP subnet. For the fourth octet of the back-end IP addresses, use sequential numbers such as 101, 102, 103, and so on. These octets can differ from the fourth octets in the front-end IP addresses.
22
Information for an HCP software installation
After the HCP core software is installed, the fourth octet of the back-end IP address cannot change, even if the fourth octet of the front-end IP address changes. Note: Several procedures in this book refer to the highest-numbered storage node. The number assigned to a node is the fourth octet of its back-end IP address. Caution: Once HCP is installed, interfering with the operation of the backend network or back-end switches can cause the system to become inoperable and/or can result in data loss. Front-end network gateway For an OS installation, you need to know the IP address of the default gateway for the front-end network. Ask the customer what IP address to use for this purpose. Once the HCP system is running, you can use the System Management Console to change the gateway IP address you specify during the OS installation. For more information on this, see Changing network settings on page 9-4.

The software installation program prompts for a variety of configuration information. This information is described in the following sections. Tip: You can use the HCP Software Installation Information form on page A-3 to record this information.
Distributor key
For an HCP core software installation, you need to know the name of the distributor key. This key enables access to the HCP nodes for troubleshooting purposes. For the name of the distributor key, contact the HCP distributor.
23
Networking
When installing the HCP core software, you need this networking information:
The IP address of the gateway for routing requests from the HCP
system to the front-end network. This is the same as the front-end gateway IP address you specify during the OS installation.
The system-specific IP address the HCP nodes use to multicast

messages to the other nodes in the system over the back end. This address must begin with 238. Normally, you accept the default multicast address proposed by the installation program. This default is 238.177.1.1. Once the HCP system is running, you can use the System Management Console to change the gateway and multicast IP addresses. For more information on this, see Changing network settings on page 9-4.
Domain name system

If the customer uses DNS at the corporate level, the HCP system should be a subdomain of the corporate domain. To set this up, the customer needs to configure the HCP subdomain in the corporate DNS setup. The recommended procedure is to do this before the HCP core software is installed. With HCP configured for DNS, clients can direct requests to the HCP system by using the system name, and the system can distribute those requests among the nodes. Without DNS, clients must direct requests to specific nodes by using the node IP addresses, which can create an imbalance in node activity. The DNS configuration of the subdomain specifies a DNS name for the HCP system. This name consists of a name for the system together with the name of the corporate domain (for example, hcp-ma.example.com, where hcpma is the system name and example.com is the corporate domain name). The DNS name can contain only lowercase letters, numbers, and hyphens (-). It must consist of at least three segments, separated by periods (.). Each segment must be one through 63 characters long. The entire DNS name, including the periods between segments, must be less than 128 characters long.
24
When installing the core software for an HCP system set up for DNS, you need to know the DNS name of the system and the IP addresses of all the corporate DNS servers. Notes:
You cannot use the DNS name to access an HCP system until the HCP
subdomain is included in the corporate DNS setup.
If the customer doesnt use DNS, you still need provide a DNS name
for the HCP system during the core software installation. This dummy DNS name is used for access to both the Tenant Management Console and the stored data. Once the HCP system is running, you can use the System Management Console to change the DNS name of the system and specify different DNS servers. For more information on this, see Changing DNS settings on page 9-2. For instructions on configuring the HCP subdomain for DNS, see Administering HCP.
Time
The customer needs to decide whether the HCP system should use itself as a time source or should use one or more external time servers. When the time source is internal, the nodes synchronize time among themselves. Because HCP is a closed system, making the time source internal ensures compliance with the applicable regulatory requirements. On the other hand, using internal time can result in clock drift, thereby causing HCP system time to differ from the time settings of other applications in the corporate environment. Resetting the HCP system time to compensate for this drift can affect object retention, thereby violating compliance. External time servers can be corporate or Internet time servers. Using a corporate time server keeps the HCP system time synchronized with other applications in the corporate environment. However, if the corporate time server is not completely secure, compliance is not guaranteed. The same concern about compliance also applies to Internet time servers. Using multiple external time servers helps ensure the integrity of the time source.
25
Tip: Several companies offer time server appliances that enable a system to manage time independently of the concerns mentioned above. Such appliances plug into the HCP network and use a GPS device to keep time. Notes:
When HCP is using an external time server, if either of the following

happens, the HCP system automatically restarts itself:
The time on the time server is changed by more than 1,000 seconds. The HCP system switches to a different time server whose time differs from the original time server by more than 1,000 seconds.
If you plan to use Active Directory authentication for default

namespace access with the CIFS protocol, the HCP system time must match the Active Directory server time. To set the time source during a core software installation, you specify either internal or the IP addresses or hostnames of one or more external time servers. If you specify internal, you also need to supply the initial time to which the nodes should synchronize. If you start the installation itself within the same installation program session in which you specify the initial time, the program adjusts that time to account for the time that elapses between when you specify the time and when the software is actually installed. Regardless of the time source, you also need to specify a time zone for the HCP system. Normally, this is the time zone in which the system is located. The installation program can present a list of time zones in which you can find the exact text for the one you want. You can also find lists of time zones on the Internet. Once the HCP system is running, you can use the System Management Console to change the time settings. For more information on this, see Changing time settings on page 9-8.
Internals
For a core software installation, you need to know what the customer wants for several internal settings.
26
Storage configuration During the installation of the HCP core software, you need to specify the type of storage the system uses. The options are internal and external. For SAN-attached storage, you need to specify that the storage is external. For all other types of storage, you need to specify that its internal. Additionally, for SAN-attached storage, you need to know:
Whether any of the storage arrays used by the HCP system are shared
with other applications or systems
Whether any of the storage arrays used by the HCP system are model
WMS-100 or AMS-200
Whether the HCP nodes are blades in one or more Hitachi CB 320
servers and, if so, the IP address of the management module (also called the chassis IP address) for each server Note: If the HCP system is using CB 320 servers, for each server, you need to add the lowest-numbered storage node in the HCP system to the list of SNMP managers in the server configuration. To identify the node, use its front-end IP address. For the community name, use public. If the lowest-numbered node changes at any time (for example, due to prolonged node unavailability or the addition of a node with a lower number), you need to update the configuration of each CB 320 server accordingly. For information on configuring CB 320 servers, see the applicable Hitachi documentation. Serial number When you install HCP, you need to specify the unique five-digit serial number for the system. This number is on a label on the rack that houses the HCP nodes. This label is attached to the side of the rack at the bottom, just inside the left rear door. Once the HCP system is running, you can use the System Management Console to change the serial number (for example, if you mistyped it during installation). For more information on this, see Changing the HCP serial number on page 9-9.
27
Replication If the customer purchased the HCP replication feature, you need to enable the feature when you install the core software. Enabling replication allows the feature to be implemented; it does not create the replication link or start replication activity. Important: Do not enable replication if the customer has not purchased this feature. Doing so makes the system violate the customers license agreement. For information on replication, see Replicating Tenants and Namespaces. Customer support contact information When installing the HCP core software, you have the option of specifying HCP customer support contact information. You specify this information as a text string. After installation, the customer can use the syscontact field in the HCP MIB file (HCP-MIB.txt) to retrieve the contact information through SNMP.
Security
During a core software installation, you can specify that HCP should encrypt all data and metadata stored in the repository. This option, which can be enabled only when the system is installed, ensures the privacy of data and may be required for compliance with certain government regulations. With the repository encrypted, lost or stolen storage devices are useless to parties without the correct encryption key. HCP handles data encryption and decryption automatically, so no human interaction is required. However, once the repository is encrypted, it remains encrypted throughout its lifetime. Encryption cannot be disabled. If you enable encryption for an HCP system, the installation program displays the encryption key while its installing the software and then prompts you to enter the key for confirmation. Before entering the key, write it down on paper. Then give the paper to the customers security administrator for storage in a secure location. Alternatively, if the security administrator is present for the installation, have that person write down and enter the key. This provides the customer with a high level of confidence in the security of their data. Tip: You can use the Encryption Key form on page A-2 to record the encryption key.
28
Post-installation configuration information
Important: The encryption key is displayed only once during installation and is not accessible through the HCP System Management Console. Caution: Loss of the encryption key will most likely result in unrecoverable data in the case of catastrophic system failure.
HCP nodes
For an HCP software installation, you need to know the back-end IP address of each HCP node. If youre reinstalling a system that already included search nodes, you also need to know which nodes are storage nodes and which are search nodes. During the installation, you can specify node IP addresses individually or as a single range for each type of node. You can also use a file to supply a list of IP addresses. For information on this last option, see Using a node-list file on page 3-27.

Once the HCP system is running, some additional configuration may be required, based on the customer needs. You perform this configuration through the HCP System Management Console. For some configuration activities, you need a user account with the administrator role; for others, a user account with the security role; and for one, a user account with the service role. Configuration that requires the administrator role For post-installation configuration activities that require the administrator role, you need to know:
Whether to enable integration with Hitachi Device Manager. If you

enable this option, you need to know the Device Manager URL, username, and password.
Whether to allow HCP nodes to respond to ping requests. Disabling this

option provides extra security for the HCP system.
29
Whether to disable the ability for HCP service providers and support
personnel to use SSH to log into the HCP nodes. Important: While disabling SSH provides extra security for the HCP system, it also severely limits the ability of service providers and support personnel to provide support for the HCP system.
Whether to install a new SSL server certificate. In this case, you need
to either have an SSL server certificate to install or have the information needed to generate a certificate signing request.
Whether to change the system data protection level (DPL) setting,

which is used by namespaces configured with dynamic DPL.
Whether to keep deletion records in the transaction log after objects

are deleted, and if so, how long to keep them.
Whether to enable the HDDS or HCP search facility. Whether to create a replication link (only if the customer has purchased
the replication feature) and, if so, the link properties. Configuration that requires the security role For post-installation configuration activities that require the security role, you need to know:
Whether to restrict access to the System Management Console to a

limited number of clients and, if so, the IP addresses of those clients
Whether to enable the HCP Management API Whether to create additional user accounts and, if so, the properties for
each account Configuration that requires either the administrator or security role For post-installation configuration activities that require either the administrator or security role, you need to know:
Whether to enable monitoring and/or modification of the HCP system

through SNMP and, if so, the IP addresses of the clients that can perform these functions.
Whether to enable syslog and/or SNMP logging. For syslog logging,

you need to know the IP addresses of the syslog servers. For SNMP logging, you need to know the IP addresses of the SNMP managers.
210
Configuration that requires the service role When HCP is installed, the disposition service is disabled by default. You need to know whether to enable it. Enabling a service requires the service role. Getting more information For more information on:
All of the above options except creating a replication link and enabling
the disposition service, see Administering HCP
Creating a replication link, see Replicating Tenants and Namespaces Enabling the disposition service, see Starting, stopping, enabling, and
disabling services on page 10-3
211
212
3
Installing an HCP system
After the HCP hardware is installed and configured, you need to install the Appliance OS and HCP software on each node. This chapter contains instructions for doing this. Important: This chapter assumes that the HCP system hardware is already installed, configured, and working properly.
Installing an HCP system Installing and Maintaining an HCP System
31
Installing the Appliance Operating System

Once the hardware for an HCP system is configured and healthy, you can install the Appliance OS on each node. Caution: Installing the Appliance OS destroys all data on the system volume and all primary storage volumes. Before installing, be very sure that the server on which youre installing is supposed to be an HCP node.
Resources you need for an OS installation

To install the Appliance OS, you need:
From the HCP distributor, one or more of the Appliance OS installation

CDs Tip: For the quickest installation, have one CD for each node in the HCP system. That way, you can install the OS on multiple nodes at the same time.
A keyboard and monitor
Before you begin the OS installation

Before you install the OS:
Complete the Appliance OS Installation Information form on page A-2

so the information you need for the OS installation is readily available.
Check the HCP release notes for any last minute installation
information.
Verify that:
For RAIN systems, the BIOS and RAID group are configured on each node. For instructions on configuring these items, see Maintaining HCP 300 System Hardware. For SAIN systems, the HBA BIOS is enabled and configured on each node. For instructions on loading and configuring the HBA BIOS, see Assembling and Configuring HCP System Hardware.
Note: This configuration is part of the HCP hardware setup, which must be completed before you install the OS.
32
Ensure that the BIOS date in the node is correct. The OS installation
fails if the BIOS date is one or more years in the past.
For SAIN systems, verify that:
The formatting of all LUNs used by HCP is 100 percent complete. (When a LUN is completely formatted, its state is Normal.) Each node has full connectivity to the SAN-attached storage. If it doesnt, please contact your HCP support center for help. LUN 0 (zero) is configured for the OS, and sufficient storage is allocated for the data volumes. If these conditions arent true, have the SAN administrator for the customer site reconfigure the storage.
Optionally, for SAIN systems, have the SAN administrator for the
customer site review the storage configuration (see Optionally reviewing the storage configuration below).
If the console youre using is not already connected to the node,

connect it. Tip: You need the console only for the first part of the OS installation. As soon as youre done with that part, you can disconnect the console. This enables you to share the console among nodes when installing the OS on multiple nodes at the same time.
Optionally reviewing the storage configuration

Before installing the OS for a system that uses SAN-attached storage, you may want to have the SAN administrator for the customer site review the storage configuration for one or more nodes. To review the storage configuration for a node: 1. If you have not already done so, connect the keyboard and monitor to the node. 2. Insert the Appliance OS installation CD into the node CD drive. 3. Power on or reboot the node.
33
The installation program prompts for the installation mode.
4. Either press Enter, or let the program default to the install option after 75 seconds. The installation program prompts for the procedure you want to perform.
5. Enter e. The installation program exits. 6. At the command prompt, enter:

fchbainfo | more
The console displays the first screenful of storage configuration information for the node:
To page through the rest of the information, use the space bar. To stop the display before youve viewed all the configuration information, press the Q key.
Performing the OS installation

To install the OS on a node: 1. If you have not already done so, connect the keyboard and monitor to the node. 2. Insert the Appliance OS installation CD into the node CD drive.
34
3. Power on or reboot the node. The installation program prompts for the installation mode.
4. Either press Enter, or let the program default to the installation option after 75 seconds. The installation program prompts for the procedure you want to perform.
5. Enter c. 6. In response to the confirming prompt, enter y.
7. When prompted, enter the front-end IP address for the node.
8. When prompted, enter the subnet mask for the front-end IP address or press Enter to accept the default.
9. When prompted, enter the IP address for the front-end network gateway or press Enter to accept the default.
35
Installing the HCP software
10.When prompted, enter the back-end IP address for the node.
The installation program displays the IP addresses and subnet mask you entered and prompts you to confirm them. 11.In response to the confirming prompt:
If the IP addresses and subnet mask are correct, enter y. If the IP addresses or subnet mask are not correct, enter n. In this case, the installation program repeats the prompts, starting again with the front-end IP address.
Tip: After youve successfully entered the IP addresses and subnet mask, you can disconnect the console from the node. You can then use them for the OS installation on another node. The installation program reformats the system volume and installs the OS. This process takes several minutes. While installing the OS, the program reports on its progress. When the installation is complete, the node ejects the CD tray. You can remove the CD from the tray any time after this. When you finish this procedure, the installation program reboots the node. If the node does not reboot automatically, the OS installation failed. In this case, please contact your HCP support center for help.

After installing the Appliance OS on every node in the system, you can install the HCP software. Caution: Installing the HCP software erases everything on the HCP nodes except the OS. Before installing, be very sure youre supposed to be doing a fresh installation and not an upgrade.
36
HCP Setup program

The program that installs the HCP software is named HCP Setup. This program includes a wizard that walks you through the HCP system configuration. Note: HCP Setup is also used in the HCP upgrade, node addition, and storage addition procedures.
Using the HCP Setup wizard menus

The HCP Setup wizard is menu driven. Below each menu, the wizard prompts for the number or letter of a menu option, as in this example showing the HCP Setup wizard New Install menu:
Each prompt for a menu selection displays a default value:
To accept the default, press Enter. To select a different option, type the number or letter you want. Then
press Enter. Menu navigation works like this:
Numbered menu options take you either to a lower-level menu or to a

prompt for a configuration value.
From any menu two or more levels below the main menu, the b option
takes you back up a level.
37
When you return to a menu, the default option is the one following the
one you just executed.
From any menu, the q option exits the wizard without saving your
work.
If the option you select cannot be undone (like the q option), the wizard
prompts twice for confirmation before executing the option. For pictures of all the HCP Setup wizard menus, see Appendix B, HCP Setup wizard menus.
Specifying configuration options

The lowest-level menus in the HCP Setup wizard prompt for configuration values, as in this example showing the HCP Time Options menu:
Most of the options in these menus display the currently configured value as the default. When you select a configuration option, the wizard prompts for a value, as in this example:
38
Configuration options work like this:
To accept the displayed default value after selecting an option, press

Enter.
For configuration options that prompt for yes or no, y, yes, n, and no are
all acceptable responses.
The wizard prompts you to confirm each value you specify for a
configuration option.
For certain options, changing the value may change the defaults for
other options. For example, changing the time server configuration from internal to a specific time server changes the Current Date and Time option to not available with time server.
After selecting a configuration option from a menu, you can press

Ctrl+C to return to that menu without changing the value.
At any time, you can use the v option on the New Install menu to
review the current configuration settings.
Saving the system configuration

At any time during the configuration process, you can save the configuration youve specified so far and exit the wizard. To do this, you use the w option on the New Install menu. When you restart the wizard, it reads in the saved configuration. This enables you to configure the HCP system in multiple wizard sessions. It also enables you to make advanced changes to the configuration, as described in Using a modified HCP configuration file on page 3-32.
Resetting configuration options to their default values

While using the HCP Setup wizard, you can return to any configuration option and change the value youve already set. You can also reset all the options to their default values. To do this, you use the r option on the New Install menu.
Running HCP Setup from a service shell

The normal way to open an install shell on a node is by logging directly into the system console. However, you can also get to the install shell from a laptop computer by using SSH to log into the applicable node as the service user.
39
If the laptop computer is connected to the front-end network, you access the node through its front-end IP address. You can also connect the computer to a back-end switch in the HCP system and access the node through its back-end IP address. For information on this option, see Appendix E, Accessing HCP interfaces through the back-end network. When you use SSH to log into the node, to ensure the continuity of the HCP Setup run, you should open the install shell in a virtual terminal session. The virtual terminal session enables HCP Setup to run even if your SSH session is disconnected. To open the install shell in a virtual terminal session: 1. Using the installation or service SSH key, use SSH to log into the applicable node in the HCP system as service. For instructions on doing this, see Appendix D, Setting up SSH key authentication. Note: You need the installation SSH key if the HCP core software is not yet installed. Otherwise, you need the service SSH key. 2. Enter this command to start the virtual terminal session:
screen
3. Enter this command to open the install shell:

/home/service/bin/install
Once a virtual terminal session is running, you can detach it from your SSH session. This enables you to shut down your computer while HCP Setup is running. To detach the session: 1. Press Ctrl+A. 2. Press the D key. If your computer shuts down unexpectedly, the virtual terminal session automatically detaches from the SSH session. To reattach a virtual terminal session to an SSH session, enter:
screen -rd
310
To close a virtual terminal session, enter:

exit
Notes:
You can open the install shell directly from the service shell (that is,
without using a virtual terminal session). After starting the HCP installation, you can exit the install shell without affecting the installation process. If you open the install shell again while the installation is still in progress, you return to the same session. However, after the installation process is complete, you cannot return to the session and, therefore, will not be able to see any messages HCP Setup may have issued during the installation.
If youre using PuTTY on a Windows laptop, be sure the Allow this

computer to turn off this device to save power option is not selected in
the power management settings for your network adapter.
Resources you need for an HCP software installation

To install the HCP software, you need
From the HCP distributor:

The HCP core software installation CD If the HCP system includes any search nodes, the two HCP search facility software installation CDs The name of the distributor key for the installation
A keyboard and monitor (or a laptop computer connected to the frontor back-end network)
Before you begin the HCP software installation

Before you install the HCP software:
Complete the HCP Software Installation Information form on page A-3

so the information you need for the HCP software installation is readily available.
Check the HCP release notes for any last minute installation
information.
311
Ensure that the Appliance OS is installed on each node in the system

and that each node is running. Note: If this is the first time youre installing an HCP system, be sure to read HCP Setup program on page 3-7 before you begin.
Performing the HCP software installation

To install the software for an HCP system: 1. Log into the highest-numbered node in the HCP system (see Step 1: Log in for the HCP software installation below). 2. Retrieve the HCP software installation files (see Step 2: Retrieve the HCP software installation files on page 3-15). 3. Configure the HCP System (see Step 3: Configure the HCP system on page 3-16). 4. Execute the installation (see Step 4: Execute the installation on page 3-19). Notes:
These sections describe the standard installation procedure. If you

want to use a file to list the node IP addresses or if you need to make special modifications to the system configuration, see Installing the HCP software advanced options on page 3-26 for the applicable procedures.
For HCP SAIN systems, if any of the nodes in the system do not have
the required version of the HBA firmware, the installation fails.
Step 1: Log in for the HCP software installation

To begin the procedure for installing the HCP software, you need to log in as the install user and then change the password for that user: 1. Connect the keyboard and monitor to the highest-numbered storage node in the system.
312
The console displays the login prompt.
Tip: If you dont see the login prompt, press Enter a few times.
Note: If youre using a laptop computer connected to the front- or back-end network, see Running HCP Setup from a service shell on page 3-9 for login instructions. 2. Insert the HCP core software installation CD into the CD drive in that node. 3. Enter: install A password prompt appears.
4. Enter: Chang3Me! As you type, the characters do not show. Note: If youve previously changed the password for the install user, enter that password instead. Then continue the installation procedure from where you ended the last session. If you havent previously changed the install user password, HCP Setup prompts for the current password again.
313
5. Enter: Chang3Me! The system prompts for a new password for the install user.
6. Enter a new password for the install user. Passwords must follow the standard rules for Unix passwords. In particular, to be valid, a password must include at least one character from two of these three groups: alphabetic, numeric, and special characters. Also, a password cannot be a word found in the dictionary. Changing the install user password in this way is a one-time event. If you need to change it again, use the procedure described in Resetting the install user password on page 11-3. Tip: For the new password, use hcpinsta11, where the last two characters are the number one. 7. When prompted, enter the new password again. The system changes the password and prompts you to continue.
8. Press Enter. The HCP 4.1 Configuration menu appears.
314
Step 2: Retrieve the HCP software installation files

To retrieve the HCP software installation files from the core software installation CD and, if applicable, HCP search facility software installation CDs: 1. From the HCP 4.1 Configuration menu, enter 1 (one) to retrieve the installation files from the CD. 2. In response to the confirming prompt, enter y or yes to confirm your entry or n or no to try again. When you enter y or yes, HCP Setup copies the HCP core software installation files from the CD to the node and ejects the CD tray. HCP Setup then asks whether you want to retrieve the search facility installation files.
3. Do either of these:
To retrieve the HCP search facility installation files:

1. Insert the first search facility software installation CD into the
CD drive.
2. Enter y or yes. 3. In response to the confirming prompt, enter y or yes to confirm
your entry or n or no to try again. When you enter y or yes, HCP Setup copies the first set of search facility installation files from the CD to the node. When this is done, HCP Setup ejects the CD tray and prompts for the second search facility software installation CD.
4. Insert the second search facility software installation CD into the
CD drive. Then press Enter.
315
HCP Setup copies the second set of search facility installation files from the CD to the node. When this is done, HCP Setup prompts you to continue.
5. Press Enter.
The HCP 4.1 Configuration menu reappears.
To continue the installation without retrieving the HCP search facility installation files:
1. Enter n or no. 2. In response to the confirming prompt, enter y or yes or n or no to
try again. When you enter y or yes, the HCP 4.1 Configuration menu reappears. The HCP 4.1 Configuration menu now includes the option to install HCP.
Note: Do not remove the last installation CD from the CD drive until the CD tray is ejected automatically.
Step 3: Configure the HCP system

To configure the HCP system: 1. From the HCP 4.1 Configuration menu, enter 2 to run the HCP Setup wizard. 2. In response to the confirming prompt, enter y or yes to confirm your entry or n or no to try again.
316
When you enter y or yes, the HCP Setup wizard New Install menu appears.
3. From the New Install menu, execute the options for configuring the HCP system. Each option in this menu either opens a lower-level menu with configuration options or leads directly to a configuration option. For pictures of all the HCP Setup wizard menus, see Appendix B, HCP Setup wizard menus. To configure the HCP system:
a. Enter 1 to change the distributor key. b. Enter 2 to change these networking settings:
Gateway router IP address Multicast IP address
c. Enter 3 to change these DNS settings:
Whether HCP is (or will be) in the corporate DNS setup DNS name for the HCP system (this is required regardless of whether the customer uses DNS) IP addresses of all the corporate DNS servers
317
Installing the HCP software d. Enter 4 to change these time settings:
Whether the time source is internal or external Current date and time (internal time source only) Time zone Tip: To see a list of valid time zones, enter view for this option. To return to the option from the list, press the Q key.
e. Enter 5 to change these internal settings:
Whether the HCP storage is internal or external Note: For external storage, you are also prompted to specify:
Whether the storage is shared with other applications. If

you answer yes, HCP Setup slows down its formatting during installation to minimize the disruption to those applications.
Whether the HCP system includes a WMS-100 or AMS-200

storage array. If you answer yes, HCP Setup reduces the amount of synchronous file system creation to avoid overloading the arrays.
Whether the HCP system uses blade servers. If you answer

yes, HCP Setup prompts for the IP addresses of the blade server chassis.

HCP system serial number Whether to enable replication Important: Do not enable replication if the customer has not purchased this feature. Doing so makes the system violate the customers license agreement.
Contact information for HCP customer support Tip: To specify no contact information, enter a space.
318
Installing the HCP software f.
Enter 6 to enable encryption.
g. Enter 7 to identify the nodes in the HCP system. Use the back-end
IP address to identify each node. Tip: If you chose to enter the node IP addresses as literal values, enter the IP address of the lowest-numbered node first. For subsequent IP addresses, HCP Setup presents a default value thats one greater than the previous one you entered.
Step 4: Execute the installation

Note: If possible, if you enabled encryption in the system configuration, have the customers security administrator present for this step. The security administrator can then be the sole person to see the encryption key. To execute the HCP software installation: 1. From the New Install menu, enter x. The wizard displays the current configuration. 2. Review the configuration. 3. Do either of these:
If the configuration is correct:

1. Enter y or yes. 2. In response to the confirming prompt, enter y or yes to confirm
your entry or n or no to try again.
319
When you enter y or yes, HCP Setup performs a set of installation prechecks and, if they are successful, installs the HCP software on all nodes in the system. This can take from several minutes to several hours, depending on the size of the logical volumes. Important: If you enabled encryption in the system configuration, HCP Setup displays the encryption key after doing some initial setup. It then prompts you to enter the key as you see it. Before entering the encryption key, write it down on paper. You can use the Encryption Key form on page A-5 for this purpose. After you enter the key, HCP Setup proceeds with the installation. You do not get a second chance to see the encryption key, and the key itself is not stored for later retrieval. When the installation is complete, HCP Setup logs you out and reboots the nodes. The console then displays the login prompt. If HCP Setup exits at any time before the installation processing is complete, make a note of all error messages and then contact your HCP support center for help.
If the configuration is not correct:

1. Enter n or no. 2. In response to the confirming prompt, enter y or yes to confirm
your entry or n or no to try again. When you enter y or yes, the wizard returns to the New Install menu, from which you can correct the configuration.
Verifying the HCP installation

To verify that the HCP system installed correctly, you can:
Use the HCP System Management Console. When you use this method,
you can also verify that you correctly entered the HCP system serial number.
Use the HCP Service menu. To use the HCP Service menu, you need a
laptop computer connected to the HCP system front- or back-end network. For more information on this, see Running HCP Setup from a service shell on page 3-9.)
320
Verifying the installation from the System Management Console

To use the HCP System Management Console to verify that the HCP system installed correctly: 1. Open the System Management Console for the HCP system in a web browser window on a client computer:
If the HCP system is configured for DNS, enter this URL:

https://admin.hcp-name.domain-name:8000
If the HCP system is not configured for DNS, enter this URL:
https://node-ip-address:8000 node-ip-address is the front-end IP address of any storage node in the HCP system.
Note: If you inadvertently use http instead of https in the URL, the browser prompts you to open or save a file. Cancel out of the prompt and try again, this time using https. 2. When prompted, accept the self-signed HCP SSL server certificate either permanently or temporarily for this Console session only. You would do the latter, for example, if the customer plans to install a trusted certificate later on. The System Management Console login page appears. If the browser cannot find the System Management Console login page, wait a few minutes; then try again. If the login page still doesnt open after a few minutes, please contact your HCP support center. 3. Check the serial number on the login page. If the serial number is incorrect, you can correct it in the System Management Console. For instructions on doing this, see Changing the HCP serial number on page 9-9. 4. Log into the System Management Console with this username and password: Username: security Password: Chang3Me!
321
The Console displays either the Change Password page or the Nodes page. If the Console displays the Nodes page, it means the nodes are still in the process of starting HCP. This process can take several minutes. When more than half the nodes have completed their startup processing, the Console automatically displays the Change Password page. If the Nodes page remains displayed after several minutes, please contact your HCP support center. 5. On the Change Password page:
In the Existing Password field, type: Chang3Me! In the New Password field, type a new password. Passwords can be up to 64 characters long and can contain any UTF-8 characters, including white space. The minimum length is six characters. To be valid, a password must include at least one character from two of these three groups: alphabetic, numeric, and other. For example, P@sswOrd is a valid password; password is not. Tip: For the new password, use hcpinsta11, where the last two characters are the number one.
In the Confirm New Password field, type your new password again.
Then click on the Update Password button. Important: After successfully verifying an installation, you need to change the password for the security account back to Chang3Me!. 6. In the top-level menu, click on Nodes. 7. On the Nodes page, for each node, check that:
The node status is Available Each logical volume is available and functioning properly (that is, the volume status indicator is )
322
If all the nodes and logical volumes are available, the installation was successful, and the customer can begin creating tenants and namespaces. However, the customer may not want to do this until additional system setup occurs. If any nodes are unavailable or if any logical volumes for available nodes are unavailable (that is, the volume status indicator is either missing or not ), please contact your HCP support center. 8. Do either of these:
Perform additional system configuration, as described in Setting additional configuration options on page 3-26. Do this only if the installation was successful. Change the security user password back to Chang3Me!. Then log out of the System Management Console and close the browser window to ensure that no one can return to the Console without a fresh login.
Verifying the installation from the HCP Service menu

To use the HCP Service menu to verify that the HCP system installed correctly: 1. Using the service SSH key, use SSH to log into the highest-numbered storage node in the system as service. For instructions on doing this, see Appendix D, Setting up SSH key authentication. 2. Enter this command to open the HCP install shell:
3. In response to the prompt to continue, press Enter.
323
The HCP 4.1 Configuration menu appears. (Option v is present only for SAIN systems.)
4. From the HCP 4.1 Configuration menu, enter s to display the HCP Service menu. 5. In response to the confirming prompt, enter y or yes to confirm your entry or n or no to try again. When you enter y or yes, the HCP Service menu appears.
6. From the HCP Service menu, enter 2 to display the current system status.
324
The HCP System Health display appears.
For each node in the HCP system, the display shows the front-end IP address and the status of the node:
HEALTHY All internal HCP components on the node are running and healthy. UNHEALTHY One or more internal HCP components on the node are not running properly. When the status of a node is UNHEALTHY, the display also shows the status of each internal component. The status of a component can be either AVAILABLE or UNAVAILABLE.
Note: While the HCP system is starting, nodes may show as UNHEALTHY. In this case, wait for the system to finish its startup processing. Then repeat this step to display the current system status again. 7. Review the HCP System Health display. Then press Enter to return to the HCP Service menu. 8. From the HCP Service menu, enter q to return to the HCP 4.1 Configuration menu. 9. In response to the confirming prompt, enter y or yes to confirm your entry or n or no to try again. 10.From the HCP 4.1 Configuration menu, enter q to log out of the install shell. 11.In response to the confirming prompt, enter y or yes to confirm your entry or n or no to try again. 12.If any nodes showed as UNHEALTHY after the system completed its startup processing, contact your HCP support center for help.
325
Installing the HCP software advanced options
Setting additional configuration options

After verifying that the HCP system installed correctly, you can perform any additional system configuration required by the customer. For example, you can enable syslog logging or disable ping. To set additional configuration options: 1. Log into the HCP System Management Console as the security user (if youre not already logged in). 2. Create a new user account with the administrator role. 3. Log out of the System Management Console. Then log in again using the new account with the administrator role. 4. Perform the required configuration activities. 5. Log out of the System Management Console. Then log in again using the security account. 6. Delete the user account you created in step 2 above. 7. If you havent already done so, reset the security user password to: Chang3Me! 8. Log out of the System Management Console and close the browser window to ensure that the customer cannot return to the Console without a fresh login. For information on creating user accounts and performing system configuration activities, see Administering HCP.

For most installations of the HCP software, you use only the procedure described in Performing the HCP software installation on page 3-12. However, two advanced installation options are available:
Identifying the nodes in the HCP system by listing them in a file Modifying the HCP configuration file manually for special installation
requirements
326
For these options, you need to perform at least part of the installation as the service user rather than the install user. To log in as the service user, you need to use a laptop computer connected to the HCP system front- or back-end network. For more information on this, see Running HCP Setup from a service shell on page 3-9.)
Using a node-list file

During a standard installation of HCP, you can specify node IP addresses individually or as a single range. As an advanced option, you can provide a list of node IP addresses in a file (herein called a node-list file). You might do this, for example, if the system includes a large number of nodes. A node-list file is a Unix-format file that contains a list of the back-end IP addresses of one or more HCP nodes, one per line. Such a file contains the IP addresses of either all the storage nodes in the system or all the search nodes but not both. During the installation, you can use a node-list file for either or both types of nodes. To use a file to specify the nodes in an HCP system: 1. Create the node-list file (see Step 1: Create the node-list file below). 2. Start the installation procedure (see Step 2: Start the installation procedure on page 3-28). 3. Identify the node-list file to the HCP Setup wizard (see Step 3: Identify the node-list file on page 3-30).
Step 1: Create the node-list file

To create a node-list file that specifies the storage or search nodes for an HCP system: 1. Using the installation SSH key, use SSH to log into the highestnumbered storage node in the HCP system as service. For instructions on doing this, see Appendix D, Setting up SSH key authentication. 2. Use a text editor such as vi to create the node-list file. Store the file in a location that the install user can access (for example, /home/service or /tmp). Give the file a meaningful name, such as storage_node_list. Tip: Create a directory using the customer name and store the nodelist file in that directory. For example, if the customer name is Example Company, create the /home/service/example directory.
327
3. Use the chmod command to change the permissions for the node-list file to 644. For example:
chmod 644 /home/service/example/storage_node_list
Step 2: Start the installation procedure

After you create the node-list file, start the HCP installation procedure from the service shell: 1. If you havent already retrieved the HCP software installation files, insert the HCP core software installation CD into the CD drive in the highest-numbered storage node in the system. If you retrieved the HCP software installation files in a previous installation session, you dont need to retrieve them again. 2. Enter this command to start a virtual terminal session:
screen
For more information on virtual terminal sessions, see Running HCP Setup from a service shell on page 3-9. 3. In the virtual terminal session, enter this command to open the HCP install shell:
4. In response to the prompt to continue, press Enter. The service version of the HCP 4.1 Configuration menu appears.
328
5. If you havent already retrieved the HCP software installation files, do so now. For instructions on doing this, see Step 2: Retrieve the HCP software installation files on page 3-15. If you retrieved the HCP software installation files in a previous installation session, you dont need to retrieve them again. 6. Enter 2 to run the HCP Setup wizard. 7. In response to the confirming prompt, enter y or yes to confirm your entry or n or no to try again. When you enter y or yes, the HCP Setup wizard New Install menu appears.
8. From the New Install menu, configure the HCP system, but wait for Step 3: Identify the node-list file below to specify the nodes for which you created a node-list file. For information on using the HCP Setup wizard to configure the system, see HCP Setup program on page 3-7. For configuration instructions, see Step 3: Configure the HCP system on page 3-16.
329
Step 3: Identify the node-list file

To identify a node-list file to the HCP Setup wizard: 1. From the New Install menu, enter 7 to specify the nodes in the HCP system. The HCP Nodes menu appears.
To provide a node-list file for the HCP storage nodes, enter 1. To provide a node-list file for the HCP search nodes, enter 2.
The Storage Node Back-end IP Addresses or Search Node Back-end IP Addresses configuration option appears, as applicable.
3. Enter: file 4. In response to the confirming prompt, enter y or yes to confirm your entry or n or no to try again.
330
When you enter y or yes, the wizard prompts for the node-list file.
5. Enter the full path and file name of the node-list file. 6. In response to the confirming prompt, enter y or yes to confirm your entry or n or no to try again. When you enter y or yes, the wizard lists the node IP addresses that are in the file and prompts you to confirm that theyre correct.
If the list of IP addresses is correct, enter y or yes. The wizard redisplays the HCP Nodes menu. From here, optionally specify a node-list file for the other type of node. Then continue the installation procedure, as described in Step 4: Execute the installation on page 3-19.
If the list of IP addresses is not correct, enter n or no. Then:

1. Press Ctrl+C to return to the New Install menu. 2. Enter w to save the configuration and exit the HCP Setup wizard. 3. Enter q to return to the virtual terminal session prompt. 4. Enter this command to return to the service shell:
exit
5. Correct the node-list file and repeat this procedure, starting with
Step 2: Start the installation procedure on page 3-28.
331
Using a modified HCP configuration file

The HCP Setup wizard prompts for configuration information for installing a typical HCP system. HCP Setup itself, however, uses additional information when building the system. Very rarely, you may need to modify this additional information to accommodate special customer needs. HCP Setup stores configuration information in a file named cluster.cfg in the /home/install/arc-deploy directory. Each time you save the configuration during a wizard session, HCP Setup updates this file. To perform a nonstandard installation, you need to modify this file manually. Caution: Do not modify the cluster.cfg file without explicit instructions from HCP support personnel. Doing so can cause the installation to fail or the HCP system not to run correctly after its installed. To install HCP with a modified configuration file: 1. Do the basic configuration of the HCP system (see Step 1: Do the basic configuration of the HCP system below). 2. Modify the configuration file (see Step 2: Modify the configuration file on page 3-33). 3. Restart the installation procedure (see Step 3: Restart the installation procedure on page 3-33). 4. Load the modified configuration file (see Step 4: Load the modified configuration file on page 3-35).
Step 1: Do the basic configuration of the HCP system

To do the basic configuration of the HCP system: 1. Follow the instructions in Step 1: Log in for the HCP software installation on page 3-12. 2. Follow the instructions in Step 2: Retrieve the HCP software installation files on page 3-15. 3. Follow the instructions in Step 3: Configure the HCP system on page 3-16. 4. From the New Install menu, enter w to save the basic configuration of the HCP system.
332
Step 2: Modify the configuration file

To modify the configuration file for an HCP installation: 1. Using the installation SSH key, use SSH to log into the highestnumbered storage node in the HCP system as service. For instructions on doing this, see Appendix D, Setting up SSH key authentication. 2. Copy the cluster.cfg file from the /home/install/arc-deploy directory to a location that the install user can access (for example, /home/service or /tmp). Tip: Create a directory using the customer name and store the cluster.cfg file in that directory. For example, if the customer name is Example Company, create the /home/service/example directory. Then enter:
cp /home/install/arc-deploy/cluster.cfg /home/service/example
3. Use a text editor such as vi to edit the copy of the cluster.cfg file as instructed by HCP support personnel. 4. Use the chmod command to change the permissions for the copy of the cluster.cfg file to 644. For example:
chmod 644 /home/service/example/cluster.cfg
Step 3: Restart the installation procedure

After you edit the copy of the cluster.cfg file, restart the HCP installation procedure from the service shell: 1. Enter this command to start a virtual terminal session:
screen
For more information on virtual terminal sessions, see Running HCP Setup from a service shell on page 3-9. 2. In the virtual terminal session, enter this command to open the HCP install shell:
333
The HCP 4.1 Configuration menu appears.
4. Enter 2 to run the HCP Setup wizard. 5. In response to the confirming prompt, enter y or yes to confirm your entry or n or no to try again. When you enter y or yes, the HCP Setup wizard New Install menu appears.
334
Step 4: Load the modified configuration file

To load the modified cluster.cfg file: 1. From the New Install menu, enter c to load the configuration file. The wizard prompts for the configuration file.
2. Enter the full path and file name of the modified configuration file. 3. In response to the confirming prompt, enter y or yes to confirm your entry or n or no to try again. When you enter y or yes, the wizard loads the modified configuration file and redisplays the New Install menu. 4. Continue the installation procedure, as described in Step 4: Execute the installation on page 3-19.
335
336
4
Upgrading an HCP system
You can upgrade an HCP system to release 4.1 from a 3.0.1 or later release. You cannot upgrade to release 4.1 directly from a release earlier than 3.0.1. An upgrade can occur with the HCP system online or offline. If an online or offline upgrade does not complete successfully, you can try restarting it. If an offline upgrade still does not complete successfully, you can return the system to its original state by rolling back the upgrade. Occasionally, you may need to apply hotfixes to particular customer installations. Hotfixes are designed to resolve customer problems on a one-off basis. This chapter contains complete instructions for:
Upgrading an HCP system to release 4.1 Restarting an upgrade Rolling back an offline upgrade Installing a hotfix
Upgrading an HCP system Installing and Maintaining an HCP System
41
About upgrading
About upgrading
Depending on the releases youre upgrading from and to, an upgrade of an HCP system entails upgrading either both the Appliance OS and the HCP software or only the HCP software:
For an upgrade from a 3.x release or later to release 4.1, you need to
upgrade both the OS and the HCP software.
For an upgrade from release 4.1 to a later 4.x release, you may or may
not need to upgrade the OS along with the HCP software. This depends on the versions of the HCP software youre upgrading from and to. All upgrades apply to both storage and search nodes, if the system includes any. During an upgrade, both the data stored on the storage nodes and the search index stored on the search nodes remain intact. To upgrade an HCP system, you use the same HCP Setup program as you do for a fresh installation. If youre not familiar with HCP Setup, read HCP Setup program on page 3-7 before beginning the upgrade procedure. Normally, you perform the upgrade procedure using a keyboard and monitor connected to the highest-numbered storage node in the HCP system. However, you can also perform the procedure from a laptop computer connected to the front- or back-end network. For more information on this, see Running HCP Setup from a service shell on page 3-9. During an upgrade, you cannot change HCP system configuration options the way you can with a fresh installation. After the upgrade completes, however, you can change certain configuration options from the System Management Console. For more information on what you can change, see Chapter 9, Reconfiguring HCP system software. You cannot upgrade an HCP system while a data migration is in progress. The customer can choose to let the migration finish, in which case, you need to finalize it before you can upgrade the system. Alternatively, the customer can cancel the migration and restart it after the upgrade is complete. For information finalizing a migration, see Finalizing a data migration on page 8-2.
42
Online and offline upgrades

An upgrade always starts with the HCP system running and healthy. During the upgrade, the system can remain online and available to users and applications. This is called an online upgrade. For an online upgrade, HCP Setup processes the HCP storage nodes one at a time, thereby leaving the repository accessible. However, it processes all the search nodes at the same time, thereby making the HCP search facility unavailable during the upgrade. Alternatively, HCP Setup can perform an offline upgrade. With this type of upgrade, HCP Setup shuts down the HCP system and upgrades all the nodes at the same time. As a result, an offline upgrade can be significantly faster than an online upgrade. However, users and applications have no access to the system while the upgrade is in process.
Considerations for online upgrades

You need to keep these considerations in mind when deciding with the customer whether an online or offline upgrade is more appropriate:
If possible, you should perform online upgrades while the load on the
HCP system is light.
During an online upgrade, users cannot create, modify, or delete user

accounts, tenants, namespaces, data access accounts, or retention classes.
During an online upgrade to HCP release 4.1, clients cannot store both
data and custom metadata for an object with a single request. This capability is new with release 4.1 and cannot be used until the upgrade is complete.
During an online upgrade under a heavy load, clients may report

errors while trying to communicate with nodes that are unavailable. These errors are expected and can be safely ignored. The customer should configure clients to retry on these transient errors.
During an online upgrade, while a node is being upgraded, it cannot be

used for repository access. As a result, NFS mounts from that node are disconnected. During this time, if NFS access to a namespace is required, clients need to remount the namespace from a different node.
43
After the upgrade of the node is finished, clients can remount from that node. If automounter is present on the client, the remount happens automatically.
During an online upgrade, data outages occur as each node is

upgraded. For any given namespace, the length of each outage depends on the DPL of the namespace and the current load on the system. No data is lost during a data outage, but clients cannot read or write to the namespace during this period.
When you start an online upgrade, all services, except shredding, are
disabled automatically. Those that were enabled before the upgrade started are automatically enabled again after the upgrade is complete. These additional considerations apply to services:
If you enable the protection service while the upgrade is in progress, the upgrade may fail. For a system in which indexing was enabled before the upgrade, objects added to search-enabled namespaces while the upgrade is in progress are not indexed or searchable until after the upgrade is complete.
During an online upgrade, users can submit queries in the Search

Console. However, if the active search system is HDDS, these queries may return partial results. If the active search system is HCP, these queries do not return any results and cannot be saved.
During an online upgrade of an HCP system with zero-copy failover

enabled, this feature is automatically disabled. After the upgrade is complete, it is automatically reenabled. Important: If the upgrade fails, zero-copy failover may remain disabled. In this case, do not reenable it. Instead, please contact your HCP support center for help.
You cannot use NDMP to back up the default namespace during an

online upgrade. If you try to do so, the backup operation will fail.
44
Before you begin an HCP upgrade
During an online upgrade of an HCP system thats either the primary

system or the replica in a replication link, you should not modify the link in any way or perform failover or data restore activities. Doing so may result in some objects not being replicated. Note: If both the primary system and the replica are already running HCP release 4.0 or later, the System Management Console on both systems does not allow you to modify the link during an online upgrade. If you're upgrading one system to release 4.0 or later and the other system is running a release earlier than 4.0, the System Management Console on the system being upgraded does not allow you to modify the link, but the Console on the other system does.
You cannot roll back an unsuccessful online upgrade. Rollback is an

option only for offline upgrades.
Monitoring HCP during an online upgrade

Online upgrades start with the lowest-numbered storage node in the HCP system. Therefore, to ensure the continued ability to monitor the system during an online upgrade, you should use the IP address of the highestnumbered storage node, and not the HCP system DNS name, to access the System Management Console. The Console remains available on that node until the upgrade process completes on all other storage nodes. After the upgrade process starts on the highest-numbered node, you get an unexpected result (such as display errors or a lost connection) from the next action taken in the Console. When this happens, you need to start a new Console session using either the DNS name of the system or the IP address of any node other than the highest-numbered one.

Before you begin the upgrade procedure:
Check the HCP release notes for any relevant last-minute information. Ensure that you have the resources you need, as described in
Resources you need for an upgrade on page 4-6.
Optionally, perform the upgrade precheck procedure to ensure that the

HCP system is ready for the upgrade, as described in Running upgrade prechecks on page 4-7.
45
If necessary, free space for the upgrade, as described in Freeing space

on the HCP nodes on page 4-14.
Information you need for an upgrade

For an HCP system upgrade, you need to know:
The physical location of the highest-numbered storage node in the

system.
The install user password for logging into the HCP nodes.
You also need to know whether the upgrade should occur with the system online or offline. For information on making this decision, see Online and offline upgrades on page 4-3.
Resources you need for an upgrade

To upgrade an HCP system, you need:
Conditionally, one Appliance OS installation CD for the new release. You need this only if you need to upgrade the OS along with the HCP software. The HCP core software installation CD. If youre upgrading an HCP system that includes search nodes, the two HCP search facility software installation CDs. You also need these CDs if the system does not yet include any search nodes, but you plan to add one or more after the upgrade.
To verify the upgrade, an HCP System Management Console user

account with the monitor, administrator, or service role
46
To complete the upgrade, a System Management Console user account

with the service role Important: Before beginning the upgrade procedure, log into the System Management Console with the service account to ensure that the account is enabled and that the password has not expired. To ensure that the password doesnt expire before the upgrade processing is complete, change the password now. Tip: Bring with you the files containing the service and root SSH keys for the new release, along with the passphrases for those keys. You may need these keys if the upgrade fails. (You get these files and passphrases from the HCP distributor.)
Running upgrade prechecks

When beginning the upgrade process, HCP Setup automatically runs a set of prechecks to ensure that the HCP system is ready to be upgraded. These checks include ensuring that the system has sufficient space for the upgrade to occur. You can choose to run the upgrade prechecks as a separate procedure at any time. For example, you may choose to do this if you want to verify ahead of time that sufficient space is available for the upgrade. If sufficient space is not available, you can use the procedure in Freeing space on the HCP nodes on page 4-14 to free space on the HCP nodes. Tip: Run the procedure to free space even if sufficient space for the upgrade is already available. With more free space, the upgrade process runs more quickly. Resources you need To perform this procedure, you need:
A keyboard and monitor connected to the highest-numbered storage

node in the system (or a laptop computer connected to the front- or back-end network)

The HCP core software installation CD If youre upgrading HCP on a system that includes search nodes, the two HCP search facility software installation CDs
47
What you do To run the upgrade prechecks: 1. Follow the steps in Step 1: Log in for the upgrade on page 4-17 to log into the highest-numbered storage node in the HCP system. 2. Follow the steps in Step 2: Retrieve the files for the upgrade on page 4-19 to retrieve the software installation files for the HCP release youre upgrading to. 3. From the HCP 4.1 Configuration menu, do either of these:
Enter 4 to run the upgrade prechecks for an offline upgrade. Enter 5 to run the upgrade prechecks for an online upgrade.
Tip: If you dont see these options, verify that the Configuration menu you see is for the release youre upgrading to. If the wrong menu is displayed, exit out to the login prompt. Then perform step 1 again, being sure to insert the HCP core software installation CD, as instructed. 4. In response to the confirming prompt, enter y or yes to confirm your entry or n or no to try again. When you enter y or yes:
If an OS upgrade is required, HCP Setup prompts you to insert the Appliance OS installation CD.
Insert the Appliance OS installation CD into the CD drive in the node youre logged into. Then press Enter. HCP Setup performs the upgrade prechecks, which include checking for available space. Note: Do not remove the OS installation CD from the CD drive until the CD tray is ejected automatically.
If an OS upgrade is not required, HCP Setup performs the upgrade prechecks, which include checking for available space.
48
For a list of all the prechecks HCP Setup performs, see Upgrade prechecks below. 5. In response to the prompt to continue, press Enter. The HCP 4.1 Configuration menu reappears. 6. From the HCP 4.1 Configuration menu, enter q to log out of the install shell. 7. In response to the confirming prompt, enter y or yes to confirm your entry or n or no to try again. Upgrade prechecks When you run the upgrade prechecks, HCP Setup performs only those that are needed to ensure a successful upgrade. For example, if the system doesnt include any search nodes, HCP Setup skips the search package precheck. The table below lists each precheck HCP Setup can perform and indicates what you should do if the precheck fails.
Precheck
all network links
Failure Description
The indicated NIC is not correctly connected to an Ethernet switch. possibly because the cable is not plugged in or is defective. The indicated node is unavailable.
Action
Ensure that the cable is plugged in at both ends and that it is not defective. If the connection appears to be correct, contact your HCP support center for help. Try restarting the node from the System Management Console. If that doesnt resolve the problem, contact your HCP support center for help. Log out of the HCP Configuration menu. Ensure that the correct CD for the upgrade is in the CD drive and that the drive tray is closed. Then log in again and retrieve the files for the upgrade. Have your SAN administrator correct the storage configuration.
all nodes available
correct menu
The version of the HCP Configuration menu currently being used is incorrect for the upgrade.
cross mapping
For SAIN systems with crossmapped nodes, the logical volumes are incorrectly mapped to the indicated node.
49

(Continued)
Precheck
database dump space
Failure Description
For offline upgrades only, insufficient space is available to ensure that a rollback can be performed in the event that the upgrade fails. Insufficient space is available for the upgrade. HCP does not appear to be configured correctly in the corporate DNS, or one or more DNS servers known to HCP are not currently available.
Action
Contact your HCP support center for help.
disk space DNS configuration
Free space as described in Freeing space on the HCP nodes below. Reconfigure the corporate DNS to correctly reference HCP, or update the HCP system to identify the correct DNS servers. For information on configuring HCP in the corporate DNS, see Administering HCP. For information on updating HCP DNS settings, see Changing DNS settings on page 9-2.
HBA firmware
The HBA card in one or more nodes does not have the required version of the HBA firmware and/or HBA BIOS.
Either: Enter y to continue the HCP upgrade or prechecks without upgrading the HBA firmware and BIOS. In this case, you need to upgrade the HBA firmware and BIOS, as needed, at a later time. Enter n to stop the HCP upgrade. Then upgrade the HBA firmware and BIOS, as needed, and start the upgrade again.
410

(Continued)
Precheck
HCP is not susceptible to bug 24747
Failure Description
The default namespace has retention classes and either or both of these is true: One or more directories in the namespace are included in an outbound or inbound replication link. The HCP system includes the HCP search facility and the default namespace is search enabled.
Action
Do either of these: To continue the upgrade, enter y. After the upgrade is complete, reassign the applicable retention classes to the affected objects. To cancel the upgrade, enter n.
Under the above conditions, if an object that was already replicated was subsequently assigned to a retention class, HCP did not replicate the change. Similarly, if an object that was already indexed was subsequently assigned to a retention class, HCP did not update the index with the change. If this precheck fails, HCP Setup gives you the option of continuing or canceling the upgrade. log rotate is not running multipath devices The HCP internal logs are currently being rotated. HCP could not communicate with one or more SAN storage arrays. Wait 20 minutes. Then run the prechecks again. Check the physical paths and correct any problems. If all path components appear to be correctly connected and functioning properly, contact your HCP support center for help. Check the physical paths and correct any problems. If all path components appear to be correctly connected and functioning properly, contact your HCP support center for help. End the process if possible. If thats not possible, contact your HCP support center for help.
multipathing
For SAIN systems with multipathed storage, one or both of the physical paths between the indicated node and the storage array are broken, or multipathing is not configured correctly on the node. An internal process is currently accessing /fcfs_*.
nobody using /fcfs_*
411

(Continued)
Precheck
node connections
Failure Description
The indicated node is unavailable.
Action
Try restarting the unavailable node from the System Management Console. If that doesnt resolve the problem, contact your HCP support center for help. Have your SAN administrator correct the search node storage configuration. Perform an offline upgrade instead of an online upgrade.
number of search volumes
For SAIN systems with search nodes, the search node storage is configured incorrectly. For online upgrades only, either: The HCP system does not include more than two storage nodes. A previous upgrade attempt was rolled back, and the HCP system includes at least one search node.
online upgrade allowed
run location
The node on which youre running HCP Setup is not the highestnumbered storage node or is not properly configured.
Run HCP Setup from the highestnumbered storage node. If the node youre running on is the highest-numbered storage node, contact your HCP support center for help. Restart HCP Setup as the install user. Execute option 1 from the HCP 4.1 Configuration menu and retrieve both the HCP core software installation files and the HCP search facility installation files. If no user accounts with the service role are enabled, ask the HCP security administrator to enable at least one. If the password for the user account you plan to use to complete the upgrade will expire within seven days, log into the HCP System Management Console using that account and change the account password.
running as install search package
You did not start HCP Setup as the install user. The HCP system includes one or more search nodes, but you did not retrieve the HCP search facility files for the upgrade. All user accounts with the service role either are disabled or will become disabled within seven days. You need an enabled user account with the service role to complete the upgrade.
service user available to finish upgrade
412

(Continued)
Precheck
smtp batches
Failure Description
More than one thousand batches of emails are waiting to be processed
Action
Disable the SMTP protocol in the Tenant Management Console for the default tenant. Then periodically run the upgrade prechecks until fewer than one thousand batches remain. The amount of time this takes depends on the number of outstanding batches. Try restarting the node from the System Management Console. If that doesnt resolve the problem, contact your HCP support center for help. Contact your HCP support center for help. Have your SAN administrator correct the storage configuration. Change the DNS name for the system to a well-formed DNS name other than hcap-name.domain.com. Contact your HCP support center for help. Contact your HCP support center for help. Check the HCP release notes for the supported upgrade paths. Then get the CDs you need to successfully upgrade the system.
SSH
The indicated node is not responding.
SSH keys standby volumes
A problem exists with the ssh keys used by the HCP system. For SAIN systems with zero-copy failover enabled, the indicated node has no standby volumes. The DNS name for the HCP system is either hcap-name.domain.com or not a well-formed DNS name. The indicated node cannot communicate with the other nodes in the HCP system. The HCP system is in a state in which it cannot be upgraded. Upgrades from the currently installed version to the new version are not supported.
system name
systemwide SSH
upgrade state versions
413
Freeing space on the HCP nodes

The procedure for freeing space on nodes works on all the nodes in the HCP system in a single execution, including on nodes that already have sufficient free space for an upgrade. It does not affect HCP operation. Resources you need To perform this procedure, you need:
A laptop computer connected to the HCP system front- or back-end

network
From the HCP distributor, the service SSH key, along with the
passphrase for that key What you do To free space on all the HCP nodes: 1. Ensure that all the nodes in the HCP system are running and healthy. You can use the HCP System Management Console to check this. 2. Insert the HCP core software installation CD into the CD drive in the highest-numbered storage node. Note: If the highest-numbered storage node was added after the system was last installed or upgraded, insert the CD into the storage node that had the highest number at the time of the installation or upgrade. 3. Using the service SSH key, use SSH to log into the node into which you inserted the CD as service. For instructions on doing this, see Appendix D, Setting up SSH key authentication. 4. Enter this command to open the HCP install shell:
414
8. From the HCP Service menu, enter 4 to free space on the HCP nodes.
415
Preparing the HCP system for an upgrade
9. In response to the confirming prompt, enter y or yes to confirm your entry or n or no to try again. When you enter y or yes, the operation to free space runs. When the operation finishes, a continuation prompt appears.
10.Press Enter to return to the HCP Service menu. 11.From the HCP Service menu, enter q to return to the HCP 4.1 Configuration menu. 12.In response to the confirming prompt, enter y or yes to confirm your entry or n or no to try again. 13.From the HCP 4.1 Configuration menu, enter q to log out of the install shell. 14.In response to the confirming prompt, enter y or yes to confirm your entry or n or no to try again. If, after this procedure, any nodes still have insufficient space for an upgrade, please contact your HCP support center for help.
Preparing the HCP system for an upgrade

To prepare the HCP system for the upgrade: 1. Ensure that all nodes in the HCP system are running and healthy. You can use the HCP System Management Console to check this. 2. If the HCP system is either the primary system or replica for a replication link, suspend replication on that link. For information on suspending replication, see Replicating Tenants and Namespaces. Note: When you execute the upgrade, HCP suspends replication automatically if it is not already suspended. However, the recommended procedure is for you to suspend it manually ahead of time.
416
Performing the upgrade

To upgrade an HCP system: 1. Log in for the upgrade (see Step 1: Log in for the upgrade below). 2. Retrieve the files for the upgrade, if you have not already done so for an upgrade precheck (see Step 2: Retrieve the files for the upgrade on page 4-19). 3. Set up the upgrade (see Step 3: Set up the upgrade on page 4-22). 4. Execute the upgrade (see Step 4: Execute the upgrade on page 4-23). 5. Verify the upgrade (see Step 5: Verify the upgrade on page 4-24) 6. Complete the upgrade (see Step 6: Complete the upgrade on page 4-25) Important: Be sure to verify the upgrade before you complete it.
7. Restore any suspended replication links (see Step 7 (conditional): Resume replication on page 4-27)
Step 1: Log in for the upgrade

To log in for the upgrade: 1. Insert the HCP core software installation CD into the CD drive in the highest-numbered storage node in the system. Note: Do not skip this step. If you do, you get the HCP Configuration menu for the currently installed release in step 5 instead of the menu for the new release. 2. If you have not already done so, connect the keyboard and monitor to that node.
417
The console displays the login prompt.
Tip: If you dont see the login prompt, press Enter a few times.
Note: If youre using a laptop computer connected to the front- or back-end network, see Running HCP Setup from a service shell on page 3-9 for login instructions. 3. Enter: install A password prompt appears.
4. Enter the password for the install user. The system prompts you to continue.
Note: If you dont know the install user password, you can reset it. For instructions on doing this, see Resetting the install user password on page 11-3. 5. Press Enter.
418
Note: If the node you logged into was not the highest-numbered storage node when the HCP system was installed or last upgraded, the HCP 4.1 Configuration menu displays only some of the options shown above.
Step 2: Retrieve the files for the upgrade

Note: If you have already retrieved the HCP software installation files for an upgrade precheck, as described in Running upgrade prechecks on page 4-7, you do not need to do so again. To retrieve the HCP software installation files from the core software installation CD and, if applicable, HCP search facility software installation CDs: 1. From the HCP 4.1 Configuration menu, enter 1 (one) to retrieve the installation files from the CD. 2. In response to the confirming prompt, enter y or yes to confirm your entry or n or no to try again.
419
When you enter y or yes, HCP Setup copies the HCP core software installation files from the CD to the node and ejects the CD tray. Note: If HCP Setup cannot copy all the files due to insufficient space, it displays this message:
Wrote only x of y bytes
In this case, you need to free space on the node, as described in Freeing space on the HCP nodes on page 4-14, and then try again to retrieve the installation files. What happens next depends on whether the HCP system includes any search nodes. HCP system with search nodes If the HCP system includes one or more search nodes, HCP Setup prompts for the first search facility software installation CD.
To retrieve the HCP search facility installation files: 1. Insert the first search facility software installation CD into the CD drive and press Enter. HCP Setup copies the first set of search facility installation files from the CD to the node. When this is done, HCP Setup ejects the CD tray and prompts for the second search facility software installation CD.
2. Insert the second HCP search facility software installation CD into the CD drive and press Enter. HCP Setup copies the second set of search facility installation files from the CD to the node. When this is done, HCP Setup prompts you to continue.
3. Press Enter.
420
The HCP 4.1 Configuration menu appears. Note: Do not remove the last installation CD from the CD drive until the CD tray is ejected automatically. HCP system without search nodes If the HCP system doesnt include any search nodes, HCP Setup asks whether you want to retrieve the search facility installation files.
Do either of these:

1. Insert the first search facility software installation CD into the CD
drive.
2. Enter y or yes. 3. In response to the confirming prompt, enter y or yes to confirm your
entry or n or no to try again. When you enter y or yes, HCP Setup copies the first set of search facility installation files from the CD to the node. When this is done, HCP Setup ejects the CD tray and prompts for the second search facility software installation CD.
4. Insert the second HCP search facility software installation CD into
the CD drive and press Enter. HCP Setup copies the second set of search facility installation files from the CD to the node. When this is done, HCP Setup prompts you to continue.
5. Press Enter.
421
To continue the upgrade without retrieving the HCP search

facility installation files:
1. Enter n or no. 2. In response to the confirming prompt, enter y or yes or n or no to try
again. When you enter y or yes, the HCP 4.1 Configuration menu reappears. Note: Do not remove the last installation CD from the CD drive until the CD tray is ejected automatically.
Step 3: Set up the upgrade

To set up the upgrade: 1. From the HCP 4.1 Configuration menu, enter 3 to run the HCP Setup wizard. 2. In response to the confirming prompt, enter y or yes to confirm your entry or n or no to try again. When you enter y or yes, either of these happens:
If an OS upgrade is required, HCP Setup prompts you to insert the Appliance OS installation CD.
Insert the Appliance OS installation CD into the CD drive in the node youre logged into. Then press Enter. Note: Do not remove the OS installation CD from the CD drive until the CD tray is ejected automatically.
422
The HCP Setup wizard Upgrade menu appears.
If an OS upgrade is not required, the HCP Setup wizard Upgrade menu appears.
Tip: You can enter s from the Upgrade menu to download and view the current system configuration.
Step 4: Execute the upgrade

To execute the upgrade: 1. From the Upgrade menu, enter u to perform an online upgrade or o to perform an offline upgrade of the HCP system. 2. In response to the prompt to continue, enter y or yes to continue or n or no to stop the procedure. 3. In response to the confirming prompt, enter y or yes to confirm your entry or n or no to try again. When you enter y or yes in both step 2 and this step, HCP Setup downloads and displays the current system configuration. 4. Review the system configuration. 5. Do either of these:

a. Enter y or yes. b. In response to the confirming prompt, enter y or yes.
423
HCP Setup performs the upgrade prechecks and, if they are successful, upgrades the software on all nodes in the system. If any of the prechecks fail, HCP Setup exits. In this case, fix the problem and then start the upgrade procedure again. For an online upgrade, HCP Setup reboots each node when the upgrade finishes on it. For an offline upgrade, HCP Setup reboots all the nodes at the same time after the installation finishes. Note: For an offline upgrade, after the nodes reboot, the repository is read-only and the HCP system does not accept configuration changes. The repository automatically becomes read-write and configuration changes are again accepted when you commit the upgrade, as described in Step 6: Complete the upgrade on page 4-25. When the upgrade processing is complete, the login prompt reappears. If HCP Setup exits at any time before the upgrade processing is complete, you can try restarting the upgrade, as described in Restarting an upgrade on page 4-27.
If the configuration is incorrect:

a. Enter n or no. b. In response to the confirming prompt, enter y or yes. c.
Exit the wizard and contact your HCP support center for help.
Step 5: Verify the upgrade

To verify that HCP Setup has successfully upgraded the software on each node in the system: 1. On a client computer, open a web browser window. 2. Clear the web browser cache.
424
3. Log into the HCP System Management Console using a user account with the monitor, administrator, or service role. Tip: Youll need the service role for Step 6: Complete the upgrade below. 4. In the top-level menu, click on Nodes. 5. On the Nodes page, for each node, check that:
If all the nodes and logical volumes are available, the upgrade was successful. Note: Search nodes may take significantly longer than storage nodes to become available. If any of the nodes or logical volumes are unavailable, wait a few minutes and then refresh the page. If any of the nodes or volumes continue to be unavailable, please contact your HCP support center for help. In this case, do not perform Step 6: Complete the upgrade or Step 7 (conditional): Resume replication below. 6. Log out of the System Management Console and close the browser window to ensure that the customer cannot return to the Console without a fresh login.
Step 6: Complete the upgrade

Once HCP Setup has finished upgrading an HCP system, you need to have the system itself complete the upgrade. You need to do this for both online and offline upgrades. For an offline upgrade, completing the upgrade commits it so that rollback is no longer possible. Note: Until you commit an offline upgrade, the repository is read-only. It automatically becomes read-write when you complete the upgrade, as described in this section.
425
To complete the upgrade of an HCP system: 1. On a client computer, open a web browser window. 2. Clear the web browser cache. 3. Log into the HCP System Management Console with the service role. 4. In the top-level menu, mouse over Configuration to display a secondary menu. 5. In the secondary menu, click on Upgrade. Note: The Upgrade page displays the current and capable internal messaging versions for the HCP system. If these versions are not the same, wait a few minutes and then click on Upgrade again to refresh the page. If the versions are still not the same, contact your HCP support center for help. 6. In the Total number of nodes field on the Upgrade page, type the total number of storage and search nodes in the HCP system. 7. Click on the Commit Upgrade button. A confirming message appears. 8. In the window with the confirming message, click in the checkbox to confirm that you understand the consequences of your action. Then click on the Yes button. The System Management Console displays a message indicating whether the upgrade was successfully completed. If the upgrade was not successfully completed, log out of the System Management Console. Then contact your HCP support center for help. Do not try the upgrade again. Tip: While the upgrade needs to be completed, the alerts on the System Management Console Overview page include:
Click here to complete the upgrade
You see this alert only when logged in as a user with the service role.
426
Restarting an upgrade
9. Log out of the System Management Console and close the browser window to ensure that the customer cannot return to the Console without a fresh login.
Step 7 (conditional): Resume replication

If you suspended replication on any replication links in Preparing the HCP system for an upgrade on page 4-16, resume replication on those links. For information on doing this, see Replicating Tenants and Namespaces. Note: If replication was suspended automatically during the upgrade, it is resumed automatically when the upgrade is complete.
If HCP Setup cannot successfully finish upgrading an HCP system, you can try restarting the upgrade. When you do this, HCP Setup starts upgrading the nodes from the point at which it failed. It does not start over from the beginning. You can restart both online and offline upgrades. You can try restarting an upgrade more than once. However, if the upgrade continues to fail, you should contact your HCP support center for help. When an upgrade fails, HCP Setup displays error messages in the system console and prompts you to press Enter. How long this takes depends on the reason for the failure, the point of failure, and whether the upgrade is being performed with the system online or offline.
427
When you press Enter, HCP Setup ejects the CD tray and returns to the HCP 4.1 Configuration menu. The menu displays only the options that are relevant for the situation.
Important: Do not reinsert the Appliance Operating System installation CD unless HCP Setup prompts for it during the upgrade restart. To restart an upgrade: 1. From the HCP 4.1 Configuration menu, enter 1 to restart the upgrade. 2. In response to the confirming prompt, enter y or yes to confirm your entry or n or no to try again. When you enter y or yes, HCP Setup restarts the upgrade. When the upgrade is complete, the login prompt reappears. If HCP Setup exits at any time before the upgrade processing is complete, you can try restarting the upgrade again, or you can contact your HCP support center for help. 3. Verify the upgrade, as described in Step 5: Verify the upgrade on page 4-24. 4. Complete the upgrade, as described in Step 6: Complete the upgrade on page 4-25. Important: Do not perform this step with out first performing step 3 above. 5. If you suspended replication on any replication links at the beginning of the upgrade procedure, resume replication on those links. For information on doing this, see Replicating Tenants and Namespaces.
428
Rolling back an offline upgrade

If an offline upgrade of an HCP system fails, you should first try restarting the upgrade, as described in Restarting an upgrade above. If it still fails, you should contact your HCP support center. Support personnel may then tell you to roll back the upgrade. Note: Do not try to roll back an upgrade unless explicitly told to do so by HCP support personnel. Rolling back an upgrade returns the storage nodes to the state they were in before you first began the upgrade procedure. However, it leaves the search nodes as they were at the point of failure and makes them unavailable. They remain unavailable until you do either of these:
Successfully upgrade the system. Until you upgrade the system,

indexing and querying are disabled.
Reinstall the original version of the HCP software on them. To do this,

add one of the search nodes back into the system as if it were a new node. This redeploys the software on each search node. However, it also destroys the search index on all the search nodes. After youve redeployed the software on the search nodes, if the HCP search facility is enabled (or when it is subsequently enabled), the facility automatically starts rebuilding the index. Searches performed while the index is being rebuilt may return incomplete results. The amount of time required to rebuild the search index depends on the number and size of the objects to be indexed. With a very large amount of data, this process can take several days. For instructions on adding nodes, see Chapter 5, Adding nodes to an HCP system. Note: You need to decide with the customer which of these options is better for them. You cannot roll back an online upgrade. If an online upgrade continues to fail after you restart it, contact your HCP support center for help.
429
Resources you need for an upgrade rollback

For the upgrade rollback procedure, you need:

network
From the HCP distributor, the service SSH key, along with the
passphrase for that key
Performing the upgrade rollback

To roll back an unsuccessful HCP system upgrade: 1. Log in for the upgrade rollback (see Step 1: Log in for the upgrade rollback below) 2. Execute the upgrade rollback (see Step 2: Execute the upgrade rollback on page 4-31) 3. Verify the upgrade rollback (see Step 3: Verify the upgrade rollback on page 4-32)
Step 1: Log in for the upgrade rollback

To log in for the upgrade rollback: 1. Using the service SSH key, use SSH to log into the highest-numbered storage node in the HCP system as service. For instructions on doing this, see Appendix D, Setting up SSH key authentication. 2. Enter this command to open the HCP install shell:
430
The HCP 4.1 Configuration menu appears. The menu displays only the options that are relevant for the situation.
Step 2: Execute the upgrade rollback

To execute the upgrade rollback: 1. From the HCP 4.1 Configuration menu, enter s to display the HCP Service menu. 2. In response to the confirming prompt, enter y or yes to confirm your entry or n or no to try again. When you enter y or yes, the HCP Service menu appears.
3. From the HCP Service menu, enter 8 to roll back the upgrade.
431
Installing a hotfix
4. In response to prompt to continue, enter y or yes to confirm your entry or n or no to try again. The wizard prompts again for confirmation. 5. In response to the confirming prompt, enter y or yes to confirm your entry or n or no to try again. When you enter y or yes in both step 4 and this step, HCP Setup rolls back the upgrade. When the rollback is complete, HCP Setup reboots each node. If any nodes do not restart automatically, reboot them manually. Important: If HCP Setup exits at any time before the rollback processing is complete, contact your HCP support center for help. Do not try the rollback again, and do not try to restart the upgrade.
Step 3: Verify the upgrade rollback

To the verify the success of the upgrade rollback, use the same procedure as for verifying a successful upgrade. This procedure is described in Step 5: Verify the upgrade on page 4-24. Note: HCP does not automatically resume activity on suspended replication links after an upgrade rollback. If the rollback was successful, you can safely resume activity on the suspended links. If the rollback was not successful, do not resume activity on any suspended links until HCP support personnel tell you it is safe to do so.
Installing a hotfix
A hotfix is an update to an HCP system that resolves a particular problem. Typically, hotfixes are installed only on systems that are experiencing that problem. Each hotfix is made available in a single file. The file name has this format:
hcp-release-number-Hhot-fix-number
For example, the file for hotfix number 1004 for HCP release 4.1 would have this name:
4.1.0-H1004
432
Installing a hotfix
For installation, the hotfix file must be located either on a CD or in a directory on the storage node that had the highest number when HCP was installed or last updated. To copy the file onto the node, you can use an SCP tool such as WinSCP. Tip: Create a directory named hotfix in the /home/service directory on the target node and store the hotfix file in there. When a hotfix is installed, its number and installation date are automatically added to the /opt/arc/hotfixes file on each node in the system. You can check this file to see which hotfixes have been installed on a system.
Resources you need for a hotfix installation

For a hotfix installation, you need:

The hotfix installation file The service SSH key, along with the passphrase for that key

network
Performing the hotfix installation

To install a hotfix: 1. Log in for the hotfix installation (see Step 1: Log in for the hotfix installation below) 2. Execute the hotfix installation (see Step 2: Execute the hotfix installation on page 4-34) 3. Verify the hotfix installation (see Step 3: Verify the hotfix installation on page 4-36)
433
Installing a hotfix
Step 1: Log in for the hotfix installation

To log in for the hotfix installation: 1. Using the service SSH key, use SSH to log in as service to the storage node that had the highest number when HCP was installed or last updated. For instructions on doing this, see Appendix D, Setting up SSH key authentication. 2. Enter this command to open the HCP install shell:
3. In response to the prompt to continue, press Enter. The HCP 4.1 Configuration menu appears. (Option v is present only for SAIN systems.)
Step 2: Execute the hotfix installation

To execute the hotfix installation: 1. From the HCP 4.1 Configuration menu, enter s to display the HCP Service menu. 2. In response to the confirming prompt, enter y or yes to confirm your entry or n or no to try again.
434
Installing a hotfix
When you enter y or yes, the HCP Service menu appears.
3. From the HCP Service menu, enter 5 to install the hotfix. The Install Hotfix menu appears.
If the hotfix file is located on a CD:

1. Insert the hotfix installation CD into the CD drive in the node
youre logged into.

2. From the Install Hotfix menu, enter 1.
HCP Setup installs the hotfix.
435
Installing a hotfix
When the installation is complete, HCP Setup ejects the CD tray and returns to the HCP 4.1 Configuration menu. Note: Do not remove the hotfix installation CD from the CD drive until the CD tray is ejected automatically.
If the hotfix file is located on the node youre logged into:

1. From the Install Hotfix menu, enter 2.
HCP Setup prompts for the hotfix file location>
2. Enter the full path and file name of the hotfix file.
HCP Setup installs the hotfix. When the installation is complete, HCP Setup returns to the HCP 4.1 Configuration menu. 5. From the HCP 4.1 Configuration menu, enter q to log out of the install shell. 6. In response to the confirming prompt, enter y or yes to confirm your entry or n or no to try again.
Step 3: Verify the hotfix installation

To verify the success of the hotfix installation, use the same procedure as for verifying an upgrade. This procedure is described in Step 5: Verify the upgrade on page 4-24.
436
5
Adding nodes to an HCP system
You can add nodes to an HCP system to expand its processing power or to increase its storage capacity. You can add both storage and search nodes. The procedure for adding nodes to an HCP system has both a hardware portion and a software portion. The hardware portion entails racking the new nodes and physically connecting them to the rest of the system. The software portion entails installing both the Appliance OS and the HCP software on the new nodes. This chapter contains instructions for the software portion of adding nodes to an HCP system. For the hardware instructions, see the applicable HCP hardware maintenance documentation.
Adding nodes to an HCP system Installing and Maintaining an HCP System
51
About adding nodes
About adding nodes

The procedure for adding one or more storage and/or search nodes to an HCP system is: 1. Prepare for the node addition, as described in Before you begin a node addition below. 2. Physically add the new nodes to the HCP system, as described in the applicable HCP hardware maintenance documentation. (As part of this preparation, you ensure that the BIOS time in each new node is within a few minutes of the HCP system time.) 3. Perform the software portion of the node addition, as described in Performing the node addition on page 5-7. 4. Verify the node addition, as described in Verifying the node addition on page 5-13. 5. Connect any new storage nodes to the front-end network, as described in applicable HCP hardware maintenance documentation. 6. After adding all the new storage nodes to the system, run the capacity balancing service, as described in Starting, stopping, enabling, and disabling services on page 10-3. You do not need to run this service if youve added only search nodes. To add nodes to an HCP system, you use the same HCP Setup program as you do for a fresh installation. If youre not familiar with HCP Setup, read HCP Setup program on page 3-7 before beginning the node addition procedure. Normally, you perform the add-nodes procedure using a keyboard and monitor connected to the highest-numbered storage node in the HCP system. However, you can also perform the procedure from a laptop computer connected to the front- or back-end network. For more information on this, see Running HCP Setup from a service shell on page 3-9.
52
About adding nodes
If DNS is in use at the customer site, the customer may want to add any new storage nodes as master servers in the HCP stub zone in the corporate DNS setup. For DNS configuration instructions, see Administering HCP. Notes:
For SAIN systems that use cross-mapped storage, you need to add
both nodes in a cross-mapped pair in the same node addition operation.
A protection set cannot include pairs of cross-mapped nodes.

Therefore, when you add only two cross-mapped nodes to a SAIN system, HCP cannot group them into a protection set for any namespaces with a DPL of two, and the capacity balancing service cannot take advantage of the new storage for those namespaces. To work around this issue, if the possibility exists that any namespaces have a DPL of two, you should add cross-mapped nodes in multiples of four.
Adding search nodes

When you add a search node to an HCP system that already includes one or more search nodes, HCP Setup reinstalls the HCP search facility software on all the search nodes. As a result, the existing search index on those nodes is deleted. This applies across all namespaces. After the software is reinstalled, if the HCP search facility is enabled (or when it is subsequently enabled), HCP automatically rebuilds the search index. Searches performed while the index is being rebuilt may return incomplete results. The amount of time required to rebuild the index depends on the number and size of the objects to be indexed. With a very large amount of data, this process can take several days.
53
Before you begin a node addition
Adding nodes in SAIN systems with CB 320 servers

Nodes in an HCP SAIN system can be blades in CB 320 servers. If the node youre adding is a blade in a server thats already included in the system, all you need to do is follow the procedure outlined in About adding nodes above. However, if the node youre adding is in a new CB 320 server, you also need to:
Make the IP address of the management module (also called the

chassis IP address) for the new CB 320 server known to HCP. To do this, you add the IP address to the list of chassis IP addresses on the Network Settings page in the HCP System Management Console. For instructions on doing this, see Changing network settings on page 9-4.
Add the lowest-numbered storage node in the HCP system to the list of
SNMP managers in the configuration of the new CB 320 server. To identify the node, use its front-end IP address. For the community name, use public. If any of the nodes youre adding has a lower number than the current lowest-numbered storage node in the HCP system, you need to update the list of SNMP managers in the configuration of each CB 320 server already included in the system. For information on configuring CB 320 servers, see the applicable Hitachi documentation.

Before you begin the node addition procedure:
Ensure that the HCP system is running and healthy. You can use the
HCP System Management Console to check this.
Check the HCP release notes for any relevant last-minute information. Complete the HCP Node Addition Information form on page A-6 so the
information you need for the node addition is readily available (see Information you need for a node addition below).
Ensure that you have the resources you need, as described in

Resources you need for a node addition on page 5-6.
54
Information you need for a node addition

To add one or more nodes to an existing HCP system, you need to know:
For each new node:

The physical location of the node The front-end IP address for the node The front-end subnet mask already set for the existing nodes in the system The IP address of the front-end network gateway already set for the existing nodes in the system The back-end IP address for the node Whether to make the node a storage or search node
For information on the front-end and back-end IP addresses for new nodes, see IP addresses for new nodes on page 5-7.
The physical location of the highest-numbered existing storage node in

the system
The install user password for logging into the HCP nodes The version numbers of the currently installed Appliance OS and HCP
software Use the HCP Node Addition Information form on page A-6 to record this information so its readily available when you perform the node addition procedure.
55
Resources you need for a node addition

To add one or more nodes to an HCP system, you need:
One or more of the Appliance OS installation CDs that match the version of the currently installed Appliance OS. Tip: For the quickest installation, have one CD for each new node. That way, you can install the OS on multiple nodes at the same time.
Conditionally, the HCP software installation CDs that match the version of the currently installed HCP software:
If the highest-numbered existing storage node in the HCP system is not the one that had the highest number when the system was installed or last upgraded (which can happen if storage nodes have previously been added to the system), and:
The HCP system doesnt include any search nodes and youre not adding any search nodes, you need only the HCP core software installation CD. The HCP system already has one or more search nodes and/ or youre adding one or more search nodes, you need both the HCP core software and the HCP search facility software installation CDs.
If the highest-numbered existing storage node in the HCP system is the one that had the highest number when the system was installed or last upgraded, you need both the HCP core software and the HCP search facility software installation CDs if all of these conditions apply:

The HCP system doesnt include any search nodes. Youre adding one or more search nodes. You did not retrieve the search facility installation files when the HCP software was installed or last upgraded.
Otherwise, you dont need any of the HCP software installation CDs.
56
Performing the node addition
To verify the success of the node addition, an HCP System Management

Console user account with the monitor, administrator, or service role
IP addresses for new nodes

When you install the Appliance OS on new nodes in an HCP system, follow these rules for assigning IP addresses:
Front-end IP addresses must be on the same front-end subnet as the

existing nodes.
Back-end IP addresses must be on the same back-end subnet as the

existing nodes. For the last octet of the back-end IP addresses, give the bottom new node the number thats one higher than the highest existing node number. Increment that number by one for each successive node going up the rack.
The IP address for the front-end network gateway must be the same as
on the existing nodes.

To add one or more nodes to an HCP system: 1. Install the Appliance OS on the new nodes (see Step 1: Install the OS on the new nodes below). 2. Log in for the node addition (see Step 2: Log in for the node addition on page 5-8). 3. Conditionally, retrieve the HCP software installation files (see Step 3 (conditional): Retrieve the HCP software installation files on page 5-9). 4. Set up the node addition (see Step 4: Set up the node addition on page 5-11). 5. Execute the node addition (see Step 5: Execute the node addition on page 5-12).
57
Step 1: Install the OS on the new nodes

Install the Appliance OS on each new node. For instructions on this, see Installing the Appliance Operating System on page 3-2.
Step 2: Log in for the node addition

To log in for the node addition: 1. Using a keyboard and monitor, log into the highest-numbered existing storage node in the system as the install user. Note: If youre using a laptop computer connected to the front- or back-end network, see Running HCP Setup from a service shell on page 3-9 for login instructions. The system prompts you to continue.
2. Press Enter. The HCP 4.1 Configuration menu appears. (Option v is present only for SAIN systems.)
58
Step 3 (conditional): Retrieve the HCP software installation files

If you need to retrieve the HCP software installation files, as described in Resources you need for a node addition on page 5-6: 1. Insert the HCP core software installation CD into the CD drive on the node youre logged into. 2. From the HCP 4.1 Configuration menu, enter 1 (one) to retrieve the installation files from the CD. 3. In response to the confirming prompt, enter y or yes to confirm your entry or n or no to try again. When you enter y or yes, HCP Setup copies the core software installation files from the CD to the node and ejects the CD tray. What happens next depends on whether the HCP system includes any search nodes. HCP system with search nodes If the HCP system includes one or more search nodes, HCP Setup prompts for the first search facility software installation CD.
59
3. Press Enter. The HCP 4.1 Configuration menu reappears. Note: Do not remove the last installation CD from the CD drive until the CD tray is ejected automatically. HCP system without search nodes If the HCP system doesnt include any search nodes, HCP Setup asks whether you want to retrieve the search facility installation files.
Do either of these:

drive.
510
Performing the node addition 5. Press Enter.
To continue the node addition without retrieving the HCP search

Step 4: Set up the node addition

To set up the node addition: 1. From the HCP 4.1 Configuration menu, enter 4 to run the HCP Setup wizard. 2. In response to the confirming prompt, enter y or yes to confirm your entry or n or no to try again. When you enter y or yes, the Membership Update menu appears.
3. If youre adding one or more storage nodes, from the Membership Update menu, enter 1. Then follow the on-screen instructions to identify each node.
511
4. If youre adding one or more search nodes, from the Membership Update menu, enter 2. Then follow the on-screen instructions to identify each node. Be sure to use the back-end IP address to identify each node. Tip: After you identify the nodes to be added, you can enter v from the Membership Update menu to download and view the current system configuration.
Step 5: Execute the node addition

To execute the node addition: 1. From the Membership Update menu, enter x to perform the node addition. The wizard displays an explanation of the node addition procedure.
2. In response to the confirming prompt, enter y or yes to confirm that you want to perform the procedure or n or no to back out. The wizard prompts again for confirmation. 3. In response to the confirming prompt, enter y or yes to confirm your entry or n or no to try again. When you enter y or yes in both step 2 and this step, the wizard downloads and displays the current system configuration. 4. Review the configuration.
512
Verifying the node addition

1. Enter y or yes. 2. In response to the confirming prompt, enter y or yes.
HCP Setup performs a series of prechecks and, if they are successful, installs the HCP software on all the new nodes. If any of the prechecks fail, HCP Setup exits. In this case, fix the problem and then start the node-addition procedure again. When the node addition is complete, HCP Setup reboots the new nodes, and the HCP 4.1 Configuration menu reappears. If HCP Setup exits at any time before the node addition processing is complete, please contact your HCP support center for help.
If the configuration is incorrect:

1. Enter n or no. 2. In response to the confirming prompt, enter y or yes. 3. Exit the wizard and contact your HCP support center for help.
6. From the HCP 4.1 Configuration menu, enter q to log out of the install shell. 7. In response to the confirming prompt, enter y or yes to confirm your entry or n or no to try again.

To verify that HCP Setup has successfully added the specified nodes to the HCP system: 1. Wait a few minutes to allow the new nodes to complete their startup processing. 2. Log into the HCP System Management Console using a user account with the monitor, administrator, or service role. 3. In the top-level menu, click on Nodes.
513
4. On the Nodes page, for each new node, check that:
If all the new nodes and logical volumes are available (which may take a few minutes), the node addition was successful. Note: Search nodes may take significantly longer than storage nodes to become available. If any of the new nodes or logical volumes are unavailable, wait a few minutes and then refresh the page. If any of the new nodes or volumes continue to be unavailable, please contact your HCP support center for help. 5. Log out of the System Management Console and close the browser window to ensure that the customer cannot return to the Console without a fresh login.
514
6
Maintaining HCP nodes
As a service provider, you are responsible for maintaining the nodes in an HCP system. You may need to:
Recover an existing storage node Recover an existing search node

Rarely, you may need to remove a node from an HCP system. For information on this, see Chapter 8, Retiring devices. This chapter contains instructions for recovering nodes. For information on the hardware side of node maintenance, see the applicable HCP hardware maintenance documentation.
Maintaining HCP nodes Installing and Maintaining an HCP System
61
Recovering an existing storage node

If HCP wont start on an existing storage node, support personnel may determine that one or more of the OS or data file systems on that node are corrupt:
If the OS file system is corrupt, you need to recover the OS. This
leaves the data file systems intact, so no data is lost. For this procedure, see Before you begin a storage node recovery below and Recovering the OS on a storage node on page 6-5. If zero-copy failover is in effect, recovering the OS enables the node to take back management of its failed-over logical volumes. For information on zero-copy failover, see Zero-copy failover on page 1-9. Caution: In HCP RAIN systems, do not use the OS recovery procedure in this chapter to recover a node in which the OS volume is not separate from the data volumes. Doing so can cause permanent data loss in namespaces with a DPL of one.
If the data file systems are corrupt. you need to recover all the logical
volumes on the node. This destroys all stored data in the volumes managed by the node. For this procedure, see Before you begin a storage node recovery below and Recovering all the logical volumes on a storage node on page 6-12. In namespaces with a DPL of two or higher, the protection service ensures that copies of all the objects stored on the node exist on other nodes as well. As a result, no data in those namespaces is lost during this recovery operation. Caution: Recovering all the logical volumes on a node can cause permanent data loss in namespaces with a DPL of one. Important: The node recovery procedures described in this section are for recovering one or more storage nodes in an otherwise healthy HCP system. Do not use either of these procedures to recover from a failed upgrade. In the case of a failed upgrade, restart the upgrade, as described in Restarting an upgrade on page 4-27.
62
Before you begin a storage node recovery

You can recover one or more storage nodes in a single node recovery procedure. Before you begin the procedure:
Check the HCP release notes for any relevant last-minute information. Ensure that at least one other storage node in the HCP system is
running and healthy.
Ensure that the BIOS time on each node to be recovered is within a few
minutes of the HCP system time.
Ensure that you have the information you need, as described in

Information you need for a storage node recovery below).

Resources you need for a storage node recovery on page 6-4. The storage node recovery procedures use both the Appliance OS installation program and the HCP software installation program (HCP Setup):
For the OS portion of each procedure, the node must be connected to a

keyboard and monitor.
For the HCP software portion of each procedure, you use SSH to log in
from a laptop computer.
Information you need for a storage node recovery

To recover one or more existing storage nodes, you need to know:
The physical location of each node The front-end IP address of each node The subnet mask for the front-end network The IP address of the front-end network gateway The back-end IP address of each node
63
Internally, HCP uses a database for metadata storage. When recovering the OS, you have the option of recovering this database as well. Before you begin the OS recovery procedure, you need to know whether or not to recover the database. Database recovery is possible only when more than half the nodes in the HCP system are running and healthy.
Resources you need for a storage node recovery

To recover existing storage nodes in an HCP system, you need:
One Appliance OS installation CD that matches the version of the currently installed Appliance OS. Tip: For the quickest node recovery time, have one CD for each node you want to recover. That way, you can install the OS on multiple nodes at the same time.
Conditionally, the HCP core software installation CD that matches the version of the currently installed HCP software. You need this only if any of these is true:
The highest-numbered storage node in the HCP system is not the one that had the highest number when the system was installed or last upgraded. This can happen if nodes have been added to the system. The highest-numbered storage node is not running and healthy. You have already retrieved the core software installation files for a later version of HCP. Typically, this would be in preparation for an upgrade of the HCP system.
If the HCP system includes any search nodes and you need the HCP core software installation CD, you also need the HCP search facility software installation CDs.
The file containing the service SSH key for the HCP system, along with the passphrase for that key.
A keyboard and monitor (for the OS portion of the procedure).
64

network (for the HCP software portion of the procedure). For information on this, see Running HCP Setup from a service shell on page 3-9.
To verify the success of the node recovery, a System Management

Console user account with the monitor, administrator, or service role.
Recovering the OS on a storage node

Recovering the OS on one or more storage nodes entails reinstalling the Appliance OS and then running HCP Setup in recovery mode. The major steps are: 1. Reinstall the Appliance OS on the target nodes (see Step 1: Reinstall the Appliance OS below). 2. Log in for the OS recovery (see Step 2: Log in for the OS recovery on page 6-6). 3. Conditionally, retrieve the installation files for the OS recovery (see Step 3 (conditional): Retrieve the installation files for the OS recovery on page 6-7). 4. Execute the OS recovery (see Step 4: Execute the OS recovery on page 6-10). Caution: To ensure that you do not destroy stored data in logical volumes managed by the nodes youre recovering, follow this procedure exactly as it is written. Do not perform this procedure unless explicitly told to do so by HCP support personnel.
65
Step 1: Reinstall the Appliance OS

Reinstall the Appliance OS on each node on which you want to recover the OS. To do this, follow the instructions in Performing the OS installation on page 3-4, with this VERY IMPORTANT EXCEPTION: In response to the prompt shown below, enter p instead of c.
Caution: When reinstalling the OS, be sure to use the same front- and back-end IP addresses as the node had before. Changing the IP addresses can cause the HCP system to become inoperable.
Step 2: Log in for the OS recovery

To log in for the OS recovery: 1. Using the service SSH key, use SSH to log into the highest-numbered healthy storage node as service. For instructions on doing this, see Appendix D, Setting up SSH key authentication. 2. Enter this command to open the HCP install shell:
66
Step 3 (conditional): Retrieve the installation files for the OS recovery

If you need to retrieve the HCP software installation files, as described in Resources you need for a storage node recovery on page 6-4: 1. Insert the HCP core software installation CD into the CD drive on the node youre logged into. 2. From the HCP 4.1 Configuration menu, enter 1 (one) to retrieve the installation files from the CD. 3. In response to the confirming prompt, enter y or yes to confirm your entry or n or no to try again. When you enter y or yes, HCP Setup copies the core software installation files from the CD to the node and ejects the CD tray. What happens next depends on whether the HCP system includes any search nodes.
67
HCP system with search nodes If the HCP system includes one or more search nodes, HCP Setup prompts for the first search facility software installation CD.
68
Do either of these:

drive.
5. Press Enter.
To continue the OS recovery without retrieving the HCP search

69
Step 4: Execute the OS recovery

To execute the OS recovery: 1. From the HCP 4.1 Configuration menu, enter s to display the HCP Service menu. 2. In response to the confirming prompt, enter y or yes to confirm your entry or n or no to try again. When you enter y or yes, the HCP Service menu appears.
3. From the HCP Service menu, enter 1 for recovery operations. 4. In response to the confirming prompt, enter y or yes to confirm your entry or n or no to try again. When you enter y or yes, HCP Setup displays the Node Recovery menu.
610
5. From the Node Recovery menu, do either of these:
To recover selected storage nodes, enter 1. Then follow the onscreen instructions to identify each node you want to recover. If you identify fewer than half the nodes in the HCP system, HCP Setup asks whether you want to recover the database on those nodes. In response:
1. Enter y or yes to recover the database along with the OS or n or
no to recover the OS without recovering the database.

2. In response to the confirming prompt, enter y or yes to confirm
your entry or n or no to try again. When you enter y or yes, HCP Setup displays a unique key and prompts you to enter it back. If you identify half or more of the nodes in the HCP system, HCP Setup displays a unique key and prompts you to enter it back.
To recover all storage nodes, enter 3. HCP Setup displays a unique key and prompts you to enter it back.
6. Enter the unique key exactly as it is shown. HCP Setup performs a series of prechecks and, if they are successful, recovers the OS on the selected nodes or all nodes, as applicable. If any of the prechecks fail, HCP Setup exits. In this case, fix the problem and then start the OS recovery procedure again. When the node recovery is complete, the HCP Service menu reappears. Important: If HCP Setup exits at any time before the OS recovery processing is complete, please contact your HCP support center for help. Do not try the OS recovery again. 7. From the HCP Service menu, enter q to return to the HCP 4.1 Configuration menu. 8. In response to the confirming prompt, enter y or yes to confirm your entry or n or no to try again. 9. From the HCP 4.1 Configuration menu, enter q to log out of the install shell.
611
10.In response to the confirming prompt, enter y or yes to confirm your entry or n or no to try again.
Recovering all the logical volumes on a storage node

Recovering all the logical volumes managed by an existing node entails reinstalling the Appliance OS and then running HCP Setup either in recovery mode or for a fresh installation, depending on whether youre recovering some or all of the nodes in the system. The major steps are: 1. Prepare for the logical-volume recovery (see Step 1: Prepare for the logical-volume recovery below). 2. Log in for the logical-volume recovery (see Step 2: Log in for the logical-volume recovery on page 6-13). 3. Conditionally, retrieve the HCP software installation files for the logicalvolume recovery (see Step 3 (conditional): Retrieve the installation files for the logical-volume recovery on page 6-14). 4. Execute the logical-volume recovery (see Step 4: Execute the logicalvolume recovery on page 6-16). Caution: Recovering the logical volumes managed by a node destroys all stored data on those volumes. Do not take this action unless explicitly told to do so by HCP support personnel.
Step 1: Prepare for the logical-volume recovery

To prepare for the logical volume recovery: 1. If you are recovering the logical volumes on only some of the nodes in the HCP system, disable zero-copy failover. For information on doing this, see Enabling or disabling zero-copy failover on page 9-10. 2. Reinstall the Appliance OS on each node on which you want to recover the logical volumes. To do this, follow the instructions in Performing the OS installation on page 3-4. Caution: When reinstalling the OS, be sure to use the same front- and back-end IP addresses as the node had before. Changing the IP addresses can cause the HCP system to become inoperable.
612
3. Using the service SSH key, use SSH to log into the highest-numbered healthy storage node as service. For instructions on doing this, see Appendix D, Setting up SSH key authentication. 4. Enter this command to set the FORCE_FORMAT environment variable:
export FORCE_FORMAT=1
Step 2: Log in for the logical-volume recovery

To log in for the logical-volume recovery: 1. Enter this command to open the HCP install shell:
613
Step 3 (conditional): Retrieve the installation files for the logical-volume recovery
If you need to retrieve the HCP software installation files, as described in Resources you need for a storage node recovery on page 6-4: 1. Insert the HCP core software installation CD into the CD drive on the node youre logged into. 2. From the HCP 4.1 Configuration menu, enter 1 (one) to retrieve the installation files from the CD. 3. In response to the confirming prompt, enter y or yes to confirm your entry or n or no to try again. When you enter y or yes, HCP Setup copies the core software installation files from the CD to the node and ejects the CD tray. What happens next depends on whether the HCP system includes any search nodes. HCP system with search nodes If the HCP system includes one or more search nodes, HCP Setup prompts for the first search facility software installation CD.
614
Do either of these:

drive.
5. Press Enter.

615
To continue the logical-volume recovery without retrieving the

HCP search facility installation files:
Step 4: Execute the logical-volume recovery

The procedure for this step depends on whether youre recovering logical volumes for some or all of the nodes in the HCP system. Executing the logical-volume recovery on some nodes To execute the logical-volume recovery on only some of the nodes in the HCP system: 1. From the HCP 4.1 Configuration menu, enter s to display the HCP Service menu. 2. In response to the confirming prompt, enter y or yes to confirm your entry or n or no to try again. When you enter y or yes, the HCP Service menu appears.
3. From the HCP Service menu, enter 1 for recovery operations.
616
4. In response to the confirming prompt, enter y or yes to confirm your entry or n or no to try again. When you enter y or yes, HCP Setup displays the Node Recovery menu.
5. From the Node Recovery menu, enter 1 to recover selected storage nodes. Then follow the on-screen instructions to identify each node you want to recover. HCP Setup displays a unique key and prompts you to enter it back. 6. Enter the unique key exactly as it is shown. HCP Setup performs a series of prechecks and, if they are successful, recovers all the logical volumes on the selected nodes or all nodes, as applicable. If any of the prechecks fail, HCP Setup exits. In this case, fix the problem and then start the logical-volume recovery procedure again. When the node recovery is complete, the HCP Service menu reappears. Important: If HCP Setup exits at any time before the logical-volume recovery processing is complete, please contact your HCP support center for help. Do not try the logical-volume recovery again. 7. From the HCP Service menu, enter q to return to the HCP 4.1 Configuration menu. 8. In response to the confirming prompt, enter y or yes to confirm your entry or n or no to try again. 9. From the HCP 4.1 Configuration menu, enter q to log out of the install shell.
617
10.In response to the confirming prompt, enter y or yes to confirm your entry or n or no to try again. 11.Reenable zero-copy failover on the HCP system. For information on doing this, see Enabling or disabling zero-copy failover on page 9-10. Executing the logical-volume recovery on all nodes To execute the logical-volume recovery on all the nodes in the HCP system, you need to perform a fresh installation of the HCP software. For instructions on doing this, see Step 3: Configure the HCP system on page 3-16 and Step 4: Execute the installation on page 3-19.
Verifying a storage node recovery

To verify the recovery of a node with corrupted file systems: 1. Log into the HCP System Management Console. 2. In the top-level menu, click on Nodes. 3. On the Nodes page, for each recovered node, check that:
If all the recovered nodes and all their logical volumes are available, the node recovery was successful. If any of the recovered nodes or any of their logical volumes are unavailable, wait a few minutes and then refresh the page. If any of the nodes or volumes continue to be unavailable, please contact your HCP support center for help. 4. Log out of the System Management Console and close the browser window to ensure that the customer cannot return to the Console without a fresh login.
618
Recovering an existing search node

If HCP wont start on an existing search node in the system, support personnel may determine that one or more of the file systems on that node are corrupt. To recover the node, you need to redeploy the HCP software on all existing search nodes. This means that the procedure can be used to recover any number of search nodes at the same time. The recovery process deletes the existing search index on the search nodes and then, if the HCP search facility is enabled, rebuilds it. As a result, searches performed while the index is being rebuilt may return incomplete results. The amount of time required to rebuild the search index depends on the number and size of the objects to be indexed. With a very large amount of data, this process can take several days. You can monitor reindexing progress by using the Objects graph on the Overview page in the System Management Console. Tenants can use the Tenant Management Console to monitor reindexing progress for their individual namespaces. Important: The node recovery procedure described in this section is for recovering one or more search nodes in an otherwise healthy HCP system. Do not use this procedure to recover from a failed upgrade. In the case of a failed upgrade, restart the upgrade, as described in Restarting an upgrade on page 4-27. You perform a search node recovery from a laptop computer connected to the HCP system front- or back-end network. For instructions on doing this, see Running HCP Setup from a service shell on page 3-9.
Before you begin a search node recovery

Before you begin a search node recovery, you need to ensure that all the storage nodes in the HCP system are running and healthy.
619
Resources you need for a search node recovery

To recover existing search nodes in an HCP system, you need:
One Appliance OS installation CD that matches the version of the currently installed Appliance OS. Tip: For the quickest node recovery time, have one CD for each node you want to recover. That way, you can install the OS on multiple nodes at the same time.
Conditionally, the set of HCP core software and HCP search facility software installation CDs that matches the version of the currently installed HCP software. You need this only if any of these is true:
The highest-numbered storage node in the HCP system is not the one that had the highest number when the system was installed or last upgraded. This can happen if nodes have been added to the system. You have already retrieved the HCP software installation files for a later version of HCP. Typically, this would be in preparation for an upgrade of the HCP system.
The file containing the service SSH key for the HCP system, along with the passphrase for that key.
To verify the success of the node recovery, an HCP System

Management Console user account with the monitor, administrator, or service role
620
Performing the search node recovery

To recover one or more search nodes: 1. Reinstall the Appliance OS on each search node you want to recover (see Step 1: Reinstall the OS below) 2. Log in for the search node recovery (see Step 2: Log in for the search node recovery below). 3. Conditionally, retrieve the HCP software installation files (see Step 3 (conditional): Retrieve the HCP software installation files on page 6-22). 4. Execute the search node recovery (see Step 4: Execute the search node recovery on page 6-23).
Step 1: Reinstall the OS

Reinstall the Appliance OS on each search node you want to recover. To do this, follow the instructions in Performing the OS installation on page 3-4. Caution: When reinstalling the OS, be sure to use the same front- and back-end IP addresses as the node had before. Changing the IP addresses can cause the HCP system to become inoperable.
Step 2: Log in for the search node recovery

To log in for a search node recovery: 1. Using the service SSH key, use SSH to log into the highest-numbered healthy storage node as service. For instructions on doing this, see Appendix D, Setting up SSH key authentication. 1. Enter this command to open the HCP install shell:
621

If you need to retrieve the HCP software installation files, as described in Resources you need for a search node recovery on page 6-20: 1. Insert the HCP core software installation CD into the CD drive on the node youre logged into. 2. From the HCP 4.1 Configuration menu, enter 1 (one) to retrieve the installation files from the CD. 3. In response to the confirming prompt, enter y or yes to confirm your entry or n or no to try again. When you enter y or yes, HCP Setup copies the core software installation files from the CD to the node. When this is done, HCP Setup prompts for the first search facility software installation CD.
4. Insert the first HCP search facility software installation CD into the CD drive and press Enter.
622
HCP Setup copies the first set of search facility installation files from the CD to the node. When this is done, HCP Setup ejects the CD tray and prompts for the second search facility software installation CD.
6. Press Enter. The HCP 4.1 Configuration menu reappears. Note: Do not remove the last installation CD from the CD drive until the CD tray is ejected automatically.
Step 4: Execute the search node recovery

To execute the search node recovery: 1. From the HCP 4.1 Configuration menu, enter s to display the HCP Service menu. 2. In response to the confirming prompt, enter y or yes to confirm your entry or n or no to try again.
623
When you enter y or yes, the HCP Service menu appears.
3. From the HCP Service menu, enter 1 for recovery operations. 4. In response to the confirming prompt, enter y or yes to confirm your entry or n or no to try again. When you enter y or yes, HCP Setup displays the Node Recovery menu.
5. From the Node Recovery menu, enter 2 to recover all search nodes. HCP Setup displays a unique key and prompts you to enter it back. 6. Enter the unique key exactly as it is shown. HCP Setup performs a series of prechecks and, if they are successful, redeploys the HCP software on all the search nodes in the system. If any of the prechecks fail, HCP Setup exits. In this case, fix the problem and then start the search node recovery procedure again.
624
When this process is complete, the HCP Service menu reappears. 7. From the HCP Service menu, enter q to return to the HCP 4.1 Configuration menu. 8. In response to the confirming prompt, enter y or yes to confirm your entry or n or no to try again. 9. From the HCP 4.1 Configuration menu, enter q to log out of the install shell. 10.In response to the confirming prompt, enter y or yes to confirm your entry or n or no to try again.
Verifying the search node recovery

To verify a search node recovery: 1. Log into the HCP System Management Console. 2. In the top-level menu, click on Nodes. 3. On the Nodes page, for all search nodes, check that:
The node status is Available Each logical volume is available and functioning properly (that is, the volume status indicator is ( )
If all the search nodes and all their logical volumes are available, the node recovery was successful. If any of the search nodes or their logical volumes are unavailable, wait a few minutes and then refresh the page. If any of the nodes or volumes continue to be unavailable, please contact your HCP support center for help. 4. Log out of the System Management Console and close the browser window to ensure that the customer cannot return to the Console without a fresh login.
625
626
7
Managing storage in an HCP SAIN system
To accommodate a growing repository, you can add storage to an HCP system. To add storage to a RAIN system, you need to add storage nodes. To add storage to a SAIN system, you can:
Add storage nodes Add logical volumes to existing storage nodes Replace an existing storage array with a larger array
Customers may also choose to replace an existing array for other reasons (for example, if they are repurposing the array). When replacing an existing storage array, you need to migrate the data from the old array to the new array. Additionally, customers may choose to virtualize an existing storage array behind an enterprise storage array without moving the data. In this case, the customer gains the storage management features associated with the enterprise array while keeping the data in place. This chapter contains instructions for:
Adding logical volumes to storage nodes in an HCP SAIN system

Note: Using the procedure in this chapter, you can add storage to storage nodes only. To add storage to a search node, you need to reinstall HCP on that node.
Reconfiguring HCP to use a new or different storage array
Managing storage in an HCP SAIN system Installing and Maintaining an HCP System
71
Adding storage to an HCP SAIN system

Adding storage to an HCP SAIN system entails: 1. Adding logical volumes (LUNs) to the storage array 2. Mapping the new volumes to HCP storage nodes 3. Making the new volumes known to HCP You perform this procedure together with the SAN administrator for the customer site. The SAN administrator is responsible for configuring the logical volumes at the storage tier. To make the new volumes known to HCP, you use the same HCP Setup program as you do for a fresh installation. If youre not familiar with HCP Setup, read HCP Setup program on page 3-7 before beginning the storage addition procedure. A storage addition always starts with the HCP system running and healthy. However, the storage addition itself can be performed with the system online or offline. During an online storage addition, HCP Setup processes the nodes one cross-mapped pair at a time. As a result, the system remains online and available to users and applications. With an offline storage addition, HCP processes all the nodes at the same time. As a result, an offline storage addition can be significantly faster than an online storage addition. However, users and application have no access to the system while the storage addition is in progress. At the beginning and end of the storage addition procedure, you use the HCP System Management Console on a client computer. For the steps in between, you need a keyboard and monitor. (Alternatively, you can use a laptop computer connected to the front- or back-end network. For more information on this, see Running HCP Setup from a service shell on page 3-9.)
Before you begin a storage addition

Before you begin the storage addition procedure:
Check the HCP release notes for any relevant last-minute information. Ensure that you have the information you need, as described in
Information you need for a storage addition below.
72

Resources you need for a storage addition on page 7-3.
Information you need for a storage addition

Before adding storage to an HCP system, you need to know the install user password for logging into the HCP nodes.
Resources you need for a storage addition

To add storage to an HCP system, you need:
Conditionally, from the HCP distributor, the HCP core software

installation CD that matches the version of the currently installed HCP software. You need this only if the highest-numbered existing storage node in the HCP system is not the one that had the highest number when the system was installed or last upgraded. This can happen if nodes have previously been added to the system. If the HCP system includes any search nodes and you need the HCP core software installation CD, you also need the HCP search facility software installation CDs.
A keyboard and monitor (or a laptop computer connected to the frontor back-end network).
If the protection service is enabled, an HCP System Management

Console account with the service role.
To verify the success of the storage addition, a System Management

Console user account with the monitor, administrator, or service role.
Performing the storage addition

To add storage to one or more nodes in an HCP system: 1. Configure the logical volumes at the storage tier (see Step 1: Configure the logical volumes at the storage tier below). 2. Prepare the HCP system for the storage addition (see Step 2: Prepare the HCP system for the storage addition on page 7-4). 3. Log in for the storage addition (see Step 3: Log in for the storage addition on page 7-4).
73
4. Conditionally, retrieve the HCP software installation files (see Step 4 (conditional): Retrieve the HCP software installation files on page 7-5). 5. Make the new logical volumes known to HCP (see Step 5: Make the new logical volumes known to HCP on page 7-8). 6. Complete the storage addition (see Step 6: Complete the storage addition on page 7-10).
Step 1: Configure the logical volumes at the storage tier

Tell the SAN administrator to configure the new logical volumes at the storage tier.
Step 2: Prepare the HCP system for the storage addition

To prepare the HCP system for the storage addition: 1. Ensure that the HCP system is running and healthy. You can use the HCP System Management Console to check this. 2. If the protection service is currently enabled, disable it. To do this, you need to log into the System Management Console with the service role. For instructions on disabling services, see Managing HCP services on page 10-2.
Step 3: Log in for the storage addition

To log in for the storage addition: 1. Using a keyboard and monitor, log into the highest-numbered existing storage node as the install user. Note: If youre using a laptop computer connected to the front- or back-end network, see Running HCP Setup from a service shell on page 3-9 for login instructions. 2. In response to the prompt to continue, press Enter.
74

If you need to retrieve the HCP software installation files, as described in Resources you need for a storage addition on page 7-3: 1. Insert the HCP core software installation CD into the CD drive on the node youre logged into. 2. From the HCP 4.1 Configuration menu, enter 1 (one) to retrieve the installation files from the CD. 3. In response to the confirming prompt, enter y or yes to confirm your entry or n or no to try again. When you enter y or yes, HCP Setup copies the core software installation files from the CD to the node and ejects the CD tray. What happens next depends on whether the HCP system includes any search nodes.
75
HCP system with search nodes If the HCP system includes one or more search nodes, HCP Setup prompts for the first search facility software installation CD.
76
Do either of these:

drive.
5. Press Enter.
To continue the storage addition without retrieving the HCP

search facility installation files:
77
Step 5: Make the new logical volumes known to HCP

To make the new logical volumes known to HCP: 1. From the HCP 4.1 Configuration menu, enter v to run the HCP Setup wizard. 2. In response to the confirming prompt, enter y or yes to confirm your entry or n or no to try again. The HCP Setup wizard Add Logical Volumes menu appears.
3. Enter u to perform an online storage addition or o to perform an offline storage addition. 4. In response to the prompt to continue, enter y or yes to continue or n or no to stop the procedure. Note: If you choose to stop the procedure, remember to reenable the protection service if it was enabled before you started. For instructions on doing this, see Managing HCP services on page 10-2. 5. In response to the confirming prompt, enter y or yes to confirm your entry or n or no to try again. When you enter y or yes in both step 4 and this step, HCP Setup performs a series of prechecks, including a check of the new LUN mappings and of whether the new LUNs are currently reserved by any other systems. If the prechecks are successful, HCP lists the new logical volumes and asks whether the list is correct.
78
Do either of these:
If the list of logical volumes is correct:

1. Enter y or yes.
Caution: Do not enter y or yes if the list of logical volumes is not correct. Doing so carries the risk of destroying data in the repository and/or leaving the system inoperable.
your entry or n or no to try again. When you enter y or yes, HCP Setup formats the new logical volumes and then prompts you to press Enter to continue. Note: If HCP Setup exits unexpectedly while its formatting the new logical volumes, please contact your HCP support center for help.
3. Press Enter.

4. From the HCP 4.1 Configuration menu, enter q to log out of the
install shell.
If the list of logical volumes is incorrect:

1. Enter n or no. 2. In response to the confirming prompt, enter y or yes to confirm
your entry or n or no to try again. When you enter y or yes, HCP Setup prompts you to press Enter to continue.
3. Press Enter.

4. Tell the SAN administrator to reconfigure the logical volumes at
the storage tier. Then start this procedure again.
79
If the prechecks are unsuccessful, HCP reports the errors it found and prompts you to press Enter to continue:
a. Press Enter.

b. Correct the errors. If the reported errors include problems with the
storage configuration, tell the SAN administrator to reconfigure the logical volumes at the storage tier. Then start this procedure again.
Step 6: Complete the storage addition

To complete the storage addition: 1. Verify the storage addition, as described in Verifying the storage addition below. 2. Reenable the protection service if it was enabled before you started this procedure. For instructions on doing this, see Managing HCP services on page 10-2.
Verifying the storage addition

To verify that the new logical volumes have been successfully added to the HCP system: 1. Log into the HCP System Management Console with the monitor, administrator, or service role. 2. In the top-level menu, click on Nodes. 3. On the Nodes page, check that for each node with one or more new logical volumes:
The node status is Available Each logical volume (old and new) is available and functioning properly (that is, the volume status indicator is )
If all the nodes and logical volumes are available, the storage addition was successful. If any of the nodes or logical volumes are unavailable, wait a few minutes and then refresh the page. If any of the nodes or volumes continue to be unavailable, please contact your HCP support center for help.
710
4. Log out of the System Management Console and close the browser window to ensure that the customer cannot return to the Console without a fresh login.

With SAIN systems, customers can choose to change the storage arrays theyre using. For example, a customer may decide to:
Upgrade an existing storage array to a larger model. Repurpose an existing storage array, thereby requiring a new or
different array for the HCP system.
Virtualize an existing modular storage array behind a new or existing

enterprise storage array. This allows enterprise features to be used in the management of modular storage. The first two examples listed above require that data on the existing array be migrated to the new or different array. You can migrate data in these ways:
Use HDS TrueCopy to migrate the data directly between the existing
array and the new or different array.
Virtualize the existing storage array behind an enterprise array and use
HDS Tiered Storage Manager to migrate the data to the enterprise array.
Virtualize the existing storage array and the new or different storage
array behind an enterprise array and use HDS Tiered Storage Manager to migrate the data to the new storage array.
Have the customer use the data migration service to migrate the data
off the node. Then finalize the migration, as described in Finalizing a data migration on page 8-2. Both TrueCopy and Tiered Storage Manager perform block-level replication of storage from one array to another.
711
All four migration methods listed above have brief periods during which the HCP system is unavailable. So does virtualizing storage without migrating the data. Important: The following procedures assume you are familiar with HCP 500 hardware, including the nodes, switches, and storage arrays. If you are not, please contact your HCP support center for help before performing any of these procedures.
Using TrueCopy to migrate data

To use TrueCopy to migrate data directly from an existing storage array to a new (or different) storage array: 1. Prepare the new array for the data migration (see Step 1: Prepare the new array below). 2. Use TrueCopy to migrate the data from the existing array to the new array (see Step 2: Use TrueCopy to migrate the data on page 7-13). 3. Reconfigure the SAN fabrics for the new array (see Step 3: Reconfigure the SAN fabrics on page 7-13). 4. Update the HCP nodes to recognize the new array (see Step 4: Update the HCP nodes to recognize the new array on page 7-14). For this step, you need:
From the HCP distributor, one Appliance OS installation CD that matches the version of the currently installed Appliance OS A keyboard and monitor
This step leaves the nodes powered off. 5. Power all the nodes back on (see Step 5: Power on the nodes on page 7-15). 6. Reenable zero-copy failover on the HCP system (see Step 6: Reenable zero-copy failover on page 7-15).
712
Step 1: Prepare the new array

To prepare the new storage array for the data migration: 1. Configure the LUNs and host groups on the new array. If possible, use the same LUN numbers on the new array as on the existing array. 2. Physically cable the two arrays together.
Step 2: Use TrueCopy to migrate the data

To use TrueCopy to migrate the data from the existing storage array to the new storage array: 1. Using TrueCopy, start the process of copying the data on the existing array to the new array. For information on how to do this, see the applicable TrueCopy documentation. 2. Wait until TrueCopy activity is almost synchronized with current HCP activity. 3. Disable zero-copy failover on the HCP system. For instructions on doing this, see Enabling or disabling zero-copy failover on page 9-10. 4. Shut down the HCP system. To do this, use the Stop System button on the Nodes page in the HCP System Management Console. If the nodes in the system do not power off automatically, wait a few minutes. Then power them off manually. 5. Wait until TrueCopy finishes copying the data from the existing array to the new array. 6. Break the TrueCopy links.
Step 3: Reconfigure the SAN fabrics

To reconfigure the SAN fabrics for the new storage array: 1. Physically cable the new array to the two SAN fabrics that connect the HCP storage to the HCP nodes. 2. Rezone the SAN fabrics to allocate LUNs on the new array to the HCP nodes.
713
Step 4: Update the HCP nodes to recognize the new array

At this point, you need to update the HCP storage and search nodes to recognize the new array. You need to perform this update procedure on each node individually. Do not proceed to Step 6: Reenable zero-copy failover below until after you have updated every node in the HCP system. For each node in the HCP system: 1. Connect the keyboard and monitor to the node. 2. Enter the HBA BIOS setup on the node. To do this:
a. Reboot the node. b. While the BIOS boot sequence is initializing the HBAs, press Ctrl+E
to enter the HBA BIOS configuration. 3. In the HBA BIOS configuration, reconfigure both paths to point to the new boot LUN by WWID. To do this, for each path:
a. Delete the existing boot LUN. b. Scan for the new boot LUN.
If the node cannot find the new boot LUN, verify that the new storage array and the SAN fabric are correctly configured. If the configuration is correct, contact your HCP support center for help. 4. Insert the Appliance OS installation CD into the CD drive in the node. 5. Reboot the node. The installation program prompts for the installation mode.
714
6. Enter this command:

san_update
You have 75 seconds to do this. If you wait too long, the node continues to boot from the CD. In this case, press Ctrl+Alt+Delete to reboot the node and start over.
san_update is a script that identifies the WWIDs for the new LUNs,
updates configuration files on the node, and builds a new boot image. When the script finishes processing, it automatically ejects the CD tray and shuts down the node. Note: Do not remove the OS CD from the CD drive until the CD tray is ejected automatically.
Step 5: Power on the nodes

Step 4: Update the HCP nodes to recognize the new array above leaves each node in the HCP system powered off. To restart the HCP system, power on all the nodes within as short a time interval as possible.
Step 6: Reenable zero-copy failover

When all the HCP nodes are running and healthy, reenable zero-copy failover on the HCP system. To check the node status, use the Nodes page in the HCP System Management Console. If any nodes are not running and healthy, do not reenable zero-copy failover. In this case, contact your HCP support center for help. For information on reenabling zero-copy failover, see Enabling or disabling zero-copy failover on page 9-10.
Virtualizing an existing storage array behind an enterprise array

To virtualize an existing array behind an enterprise array: 1. Prepare the existing storage array and the enterprise storage array for the virtualization (see Step 1: Prepare the storage arrays for the virtualization below) 2. Prepare the HCP system for the virtualization (see Step 2: Prepare the HCP system for the virtualization on page 7-17)
715
3. Update the HCP nodes to recognize the new array (see Step 4: Update the HCP nodes to recognize the new array on page 7-14). For this step, you need:
From the HCP distributor, one Appliance OS installation CD that matches the version of the currently installed Appliance OS A keyboard and monitor
4. Reenable zero-copy failover on the HCP system (see Step 6: Reenable zero-copy failover above).
Step 1: Prepare the storage arrays for the virtualization

To prepare the existing storage array and the enterprise storage array for the virtualization: 1. Decide which physical ports on the existing array you want to use for the virtualization links. If possible, these should be dedicated ports. 2. Decide which physical ports on the enterprise array you want to use for the virtualization links. These must be dedicated external ports. 3. On both the existing array and the enterprise array, configure the ports for the virtualization links with these properties:
Fixed speed (not auto-negotiate) Fabric disabled Mode FC-AL Host type Windows Extended (C2)
4. On the existing array:

a. Do either of these:
If LUN security is required, add the WWPNs from the enterprise array to the applicable default host group. If LUN security is not required, disable it on the ports used for the virtualization links.
716
Reconfiguring HCP to use a new or different storage array b. Map the LUNs to be virtualized to the default host group on the
applicable ports. Note: If the ports are not dedicated, you need to create a new host group for this purpose instead of using the default host group. 5. On the enterprise array:
a. Create a path group consisting of the ports on the existing array. b. Map the LUNs to the LUNs on the existing array, specifying the
external volume number and control unit number as part of the process. If possible, use the same LUN numbers on the enterprise array as on the existing array, but use a unique control unit number.
c. Create host groups on the ports to which the HCP nodes will connect
through the SAN fabrics. Make these host groups the same as the host groups that use the equivalent LUNs on the existing array.
Step 2: Prepare the HCP system for the virtualization

To prepare the HCP system for the virtualization: 1. Disable zero-copy failover on the HCP system. For instructions on doing this, see Enabling or disabling zero-copy failover on page 9-10. 2. Shut down the HCP system. To do this, use the Stop System button on the Nodes page in the HCP System Management Console. If the nodes in the system do not power off automatically, wait a few minutes. Then power them off manually. From here, continue by performing:
Step 4: Update the HCP nodes to recognize the new array on

page 7-14
Step 6: Reenable zero-copy failover on page 7-15

In these procedures, the term new array means the enterprise array.
717
Using Tiered Storage Manager to migrate data

After virtualizing an existing storage array, you can migrate data from that array to the enterprise array or to another array virtualized behind the same enterprise array. In either case, you use Tiered Storage Manager to perform the migration. With Tiered Storage Manager, the HCP system remains available throughout the entire migration process. For information on using Tiered Storage Manager, see the applicable Tiered Storage Manager documentation.
718
8
Retiring devices
To properly retire a node in an HCP RAIN or a storage array in a SAIN system, you do not simply unplug the device from the system. For a device that stores objects, you first need to move the data off the device to another location in the system. For both storage and search nodes, you also need to shut down the HCP software on the node in an orderly fashion. The normal way to retire a storage node in an HCP RAIN system is to finalize a data migration that was started by an HCP system administrator. Rarely, however, you may need to retire a node by removing it from the system without first migrating the data off it. For SAIN systems, the normal way to retire a storage array, is to finalize a data migration that was started by an HCP system administrator. You dont normally retire storage nodes in a SAIN system; instead, you replace them. For both RAIN and SAIN systems, you retire a search node by removing it from the system without first migrating the data off it. This chapter contains instructions for finalizing a data migration and for removing a node without first migrating the data off it.
Retiring devices Installing and Maintaining an HCP System
81
Finalizing a data migration

The HCP migration service migrates data off selected storage nodes in an HCP RAIN system or selected storage arrays in an HCP SAIN system in preparation for retiring those devices. During a data migration, the service copies objects from the selected devices to free storage on the remaining devices. After copying an object, the service deletes it from the source device. The HCP system administrator configures and runs data migrations from the HCP System Management Console. Once a migration is complete, you need to finalize it from the HCP Service menu and remove the retired devices from the system. For a SAIN system, before the HCP system administrator can start a data migration, you need to work with the SAN storage administrator for the site to add logical volumes (LUNs) from new or existing storage arrays to any storage nodes with all their LUNs on arrays that are being retired. In particular, if the OS LUN for a node comes from an array thats being retired, the SAN administrator must map a replacement OS LUN to that node. You cannot upgrade an HCP system while a data migration is in progress. The customer can choose to let the migration finish, in which case, you need to finalize it before you can upgrade the system. Alternatively, the customer can cancel the migration and restart it after the upgrade is complete. For more information on data migration, see Administering HCP. Finalizing a migration in a RAIN system When you finalize a data migration in a RAIN system, HCP Setup shuts down HCP on each node being retired and then, if possible, powers off those nodes. It then removes the nodes from the HCP system configuration. The hardware configuration determines whether any given node powers off automatically. To complete the node retirement, you may need to power off the node manually. To ensure that HCP has enough time to shut down cleanly, wait a few minutes before powering the node off manually. If the node is connected to a console, the console displays System halted when you can safely power off the node.
82
Because HCP is shut down only on the nodes being retired, the system remains available to users and applications while the data migration is being finalized. After a data migration is finalized, the Nodes page in the HCP System Management Console displays Removed in the controls column for the retired nodes. Finalizing a migration in a SAIN system When you finalize a data migration in a SAIN system, HCP Setup: 1. Shuts down HCP on all nodes in the HCP system. 2. Migrates the system-level data off the storage arrays that are being retired. 3. Removes the data and OS LUNs on the selected arrays from the HCP configuration. 4. Powers off all nodes in the system. If any nodes do not power off automatically, you need to power them off manually. For information on powering off nodes, see Finalizing a migration in a RAIN system above. Because HCP is shut down on all nodes during this procedure, the system is unavailable to users and applications while the data migration is being finalized. After finalizing the migration, you need to update the fibre channel HBA card BIOS in each node from which the OS LUN was retired so that the node boots off the new OS LUN. Additionally, if any search nodes had LUNs on the retired arrays, the SAN storage administrator for the site needs to map new LUNs to those nodes. Then, if the HCP search facility is enabled (or when it is subsequently enabled), you need to rebuild the search index.
Before you begin finalizing a migration

Before you begin finalizing a data migration:
Check the HCP release notes for any relevant last-minute information. Ensure that the status of the data migration in the HCP System
Management Console is Migrated.
83
Ensure that you have the information and resources you need, as
described below.
Resources you need for finalizing a migration

To finalize a data migration, you need:

network. For information on this, see Running HCP Setup from a service shell on page 3-9.
From the HCP distributor, the file containing the service SSH key for the
HCP system, along with the passphrase for that key.
Finalizing the migration and removing the retired devices

To finalize a data migration and remove the retired devices: 1. Log in for the migration finalization (see Step 1: Log in for the migration finalization below). 2. Execute the migration finalization (see Step 2: Execute the migration finalization on page 8-6). 3. Physically remove the retired devices from the HCP system (see Step 3: Remove the retired devices from the HCP system on page 8-7). 4. For SAIN systems with any search nodes that had LUNs on the retired storage arrays, have the SAN storage administrator at the site map new LUNs to those nodes. Then reinstall the Appliance OS on those nodes (see Step 4 (SAIN systems only; conditional): Reinstall the Appliance OS on the affected search nodes on page 8-8). 5. For SAIN systems with retired storage arrays that had OS LUNs, update the fibre channel HBA card BIOS in each affected node (see Step 5 (SAIN systems only; conditional): Update the fibre channel HBA firmware and reboot the nodes on page 8-8). 6. For SAIN systems that have any search nodes with new LUNs, rebuild the search index (see Step 6 (SAIN systems only; conditional): Rebuild the search index on page 8-8).
84
Step 1: Log in for the migration finalization

To log in for the migration finalization: 1. Using the service SSH key, use SSH to log into the highest-numbered healthy storage node as service. For RAIN systems, this is the highestnumbered storage node that is not being retired. For instructions on using SSH to log in, see Appendix D, Setting up SSH key authentication. 2. Enter this command to open the HCP install shell:
85
Step 2: Execute the migration finalization

To execute the migration finalization: 1. From the HCP 4.1 Configuration menu, enter s to display the HCP Service menu. 2. In response to the confirming prompt, enter y or yes to confirm your entry or n or no to try again. When you enter y or yes, the HCP Service menu appears.
3. From the HCP Service menu, enter 6 for the migration finalization. The wizard prompts you to continue.
4. Enter y or yes to confirm your entry or n or no to try again. 5. In response to the confirming prompt, enter y or yes to confirm your entry or n or no to try again.
86
When you enter y or yes in step 4 and this step, HCP Setup performs a series of prechecks, including checking that the data migration is complete and, for SAIN systems, that all the nodes have the required version of the fibre channel HBA firmware. In the latter case, you can continue with the migration finalization. However, after the migration is finalized, the firmware must be upgraded to enable the nodes to operate correctly. If all the other prechecks are successful, HCP Setup finalizes the data migration. If any of the prechecks fail, HCP Setup exits. In this case, fix the problem and then start the upgrade procedure again. If HCP Setup exits at any time before it finishes finalizing the migration, please contact your HCP support center for help. 6. For a RAIN system, in response to the prompt to continue, press Enter. The HCP 4.1 Configuration menu reappears. 7. From the HCP 4.1 Configuration menu, enter q to log out of the install shell. 8. In response to the confirming prompt, enter y or yes to confirm your entry or n or no to try again.
Step 3: Remove the retired devices from the HCP system

After you have successfully finalized the data migration, you can physically remove the retired devices from the HCP system: 1. Unplug the retired nodes (RAIN systems) or, if applicable, the retired storage arrays (SAIN systems) from the power distribution units (PDUs) in the HCP system rack or racks. 2. Do either of these:
For a RAIN system:

1. Unplug the power cords from the retired nodes. 2. Disconnect the front-end Ethernet cables from the retired nodes. 3. Disconnect the back-end Ethernet cables from the retired nodes. 4. Remove the retired nodes from the rack.
87
For a SAIN system:

1. Disconnect the fibre channel cables from the retired storage
arrays.
2. For modular arrays housed in HCP system racks: a. Disconnect both ends of the cables that connect the trays of
the retired arrays to each other.

b. Remove the trays of the retired arrays from the racks.
Step 4 (SAIN systems only; conditional): Reinstall the Appliance OS on the affected search nodes
For a SAIN System, if any search nodes had LUNs from the retired arrays: 1. Have the SAN storage administrator for the site map new LUNs to those nodes. 2. Install the Appliance OS on those nodes. For instructions on this, see Installing the Appliance Operating System on page 3-2.
Step 5 (SAIN systems only; conditional): Update the fibre channel HBA firmware and reboot the nodes
After finalizing a data migration in a SAIN system, for each node:
If the migration finalization prechecks indicated that the node does not
have the required version of the fibre channel HBA firmware, upgrade the HBA firmware in the node.
If the OS LUN for the node was on a retired array, update the fibre
channel HBA BIOS in the node so that the node boots off the new OS LUN. Then reboot the nodes one at a time. For information on configuring the fibre channel HBA firmware, see Assembling and Configuring HCP System Hardware.
Step 6 (SAIN systems only; conditional): Rebuild the search index

For a SAIN system that has any search nodes with new LUNs, redeploy the HCP software on those nodes. This forces HCP to rebuild the search index (if or when the HCP search facility is enabled).
88
Removing nodes
To redeploy the HCP software on the search nodes, you use the procedure for adding nodes to the HCP system, specifying the existing search nodes as the nodes to be added. For information on this procedure, see Chapter 5, Adding nodes to an HCP system. Redeploying the software on the search nodes causes whats left of the existing index to be deleted. While the index is being rebuilt, searches may return incomplete results. The amount of time required to rebuild the search index depends on the number and size of the objects to be indexed. With a very large amount of data, this process can take several days.
Removing nodes
Very rarely, you may need to remove a storage or search node from an HCP system. This can happen, for example, if you need the node IP address for some other system. You cannot use the HCP System Management Console to remove a node while a data migration is in progress, even if that node is not a source for the migration. Caution: Do not remove a storage node from an HCP SAIN system unless all data has been migrated off all storage arrays with LUNs mapped to that node. Doing so can result in data loss. When you remove the node after the data migration has been finalized, also remove the node with which that node forms a cross-mapped pair. About removing storage nodes Removing a storage node tells the HCP system that the back-end IP address of the node is no longer part of the system. Node removal is not part of the normal procedure for replacing a storage node that has failed or for upgrading to newer hardware. In these cases, the new node can use the same back-end IP address as the one being replaced. You cannot remove a storage node if doing so would result in fewer than half of the nodes in the system being available. For example, if the HCP system includes four storage nodes but only two of the nodes are available, you cannot remove either of the two available nodes.
89
Removing nodes
After you remove a storage node, the Nodes page in the HCP System Management Console displays Removed in the controls column for the retired nodes. Important:
Once you remove a storage node from the HCP system, the system can
never again include a node with the same back-end IP address as that node. If you remove a search node, the back-end IP address of that node can be reused.
In RAIN systems, to prevent data loss in DPL-one namespaces that are

not being replicated, always have the customer migrate data off the node before you remove the node from the HCP system. Then, to remove the node, use the procedure described in Finalizing a data migration above. About removing search nodes In an HCP system with a single search node, removing that node deletes the search index thats maintained by the HCP. If you later add a search node back into the system, if the HCP search facility is enabled (or when its subsequently enabled), HCP automatically rebuilds the index. If you remove a search node from an HCP system with more than one such node, the search index breaks. In this case, you need to redeploy the HCP software on the remaining search nodes to force HCP to rebuild the index. To do this, you use the procedure for adding nodes to the HCP system, specifying the existing search nodes as the nodes to be added. For information on this procedure, see Chapter 5, Adding nodes to an HCP system. Redeploying the software on the remaining search nodes causes the existing index on those nodes to be deleted. As a result, searches performed while the index is being rebuilt may return incomplete results. The amount of time required to rebuild the search index depends on the number and size of the objects to be indexed. With a very large amount of data, this process can take several days. What you do To remove a node from an HCP system: 1. Log into the HCP System Management Console with the service role. 2. In the top-level menu, click on Nodes.
810
Removing nodes
The System Management Console displays the Nodes page, which shows the list of nodes in the HCP system. 3. If the node you want to remove is currently running, click on the shutdown control ( ) for it. A confirming message appears. 4. In the Reason field in the message window, type the reason why youre shutting down the node. This text can be up to 1,024 characters long and can contain any UTF-8 characters, including white space. 5. Click on the Shut Down Node button in the message window. HCP shuts down the node. 6. Click on the remove control ( ) for the node. This control appears only while the node is unavailable. Tip: You can also use buttons at the bottom of the Storage/Search Node page to shut down and remove a node. 7. In response to the confirming message, click on the Remove Node button. HCP removes the node from the system. 8. If the node you removed was a search node and:
The system still includes at least one other search node, add any one such node to the system as if it were a new node. For instructions on doing this, see Chapter 5, Adding nodes to an HCP system. It was the last search node in the system:
1. In the top-level menu in the System Management Console,
mouse over Services to display a secondary menu.

2. In the secondary menu, click on Search. 3. On the Search page, deselect both the Enable Querying and
Enable Indexing options.

4. Click on the Update Settings button.
811
Removing nodes
812
9
Reconfiguring HCP system software
You can reconfigure HCP system software in several ways:
These reconfiguration options require a system restart:

Changing DNS information Changing the network configuration Changing time settings Disabling the replication feature
Important: System restarts disrupt namespace access.
These reconfiguration options do not require a system restart:
Adding and removing chassis IP addresses in an HCP SAIN system that uses CB 320 servers Changing the HCP serial number Changing the HTTP persistent connection timeout interval Enabling or disabling zero-copy failover Enabling or disabling DPL one in RAIN systems Changing the region count Enabling the replication feature
Reconfiguring HCP system software Installing and Maintaining an HCP System
91
Changing DNS settings
You perform most system software reconfiguration activities from the HCP System Management Console. Each of those activities requires a user account with the service role. To enable or disable DPL one, change the region count, or enable or disable the replication feature, you use SSH to log into an HCP node. To do this, you need a laptop computer connected to the HCP system front- or backend network. For information on this, see Running HCP Setup from a service shell on page 3-9. This chapter describes the procedure for each of the activities listed above. For information on creating a user account with the service role, see Administering HCP.

When you install the core software for an HCP system configured for DNS, you specify both its DNS name and the IP addresses of all the corporate DNS servers. If the customer is not using DNS, you specify a dummy DNS name for the system. After the HCP system is installed, you can enable or disable the use of DNS as needed. You can change the DNS name of the system at any time while the system is running. If the system is configured for DNS, you can also change the DNS servers it uses. You might do this, for example, if a DNS server fails. Additionally, you can change the hostname prefix used to name the nodes in the HCP system. You would need to do this for one system, for example, if the customer has two HCP systems and either of these is true:
The customer uses Active Directory authentication for CIFS access to

the default namespace
The two systems have one or more node numbers in common
92
Notes:
If the system is configured for DNS and you change the DNS name of
the system, make sure you change the name in the corporate DNS setup as well. For consistency, if you change the hostname prefix, you should also change that in the DNS setup.
After you change the DNS name, any clients connected to the System
Management Console at the time of the change need to close all browser windows to clear cookies so they can access the Console using the new name. This also applies to clients accessing the Tenant Management Console. Important: Do not change the DNS settings for an HCP system while any nodes are unavailable. If you do, those nodes will have incorrect DNS information when they rejoin the system. For more information on using DNS with HCP, see Domain name system on page 2-4. To change DNS settings for an HCP system: 1. In the top-level menu in the System Management Console, mouse over Configuration to display a secondary menu. 2. In the secondary menu, click on DNS Settings. 3. On the DNS Settings page:
Select or deselect the Use DNS option to have HCP use or not use DNS, respectively. To change the DNS name of the HCP system, in the System Name field, type the new DNS name for the system. This name can contain only lowercase letters, numbers, and hyphens (-). Additionally, it must consist of at least three segments, separated by periods (.).
93
Changing network settings
To change the hostname prefix, in the Hostname Prefix field, type the new hostname prefix. The hostname prefix can be from one through 12 characters long and can contain only lowercase letters, numbers, and hyphens (-). Tip: To make node names easier to read, end the hostname prefix with a hyphen (-).
To change the DNS servers HCP uses, in the DNS Servers field, type a comma-separated list of the IP addresses of all the current corporate DNS servers. Spaces are allowed.
4. Click on the Update Settings button. A warning message appears asking you to confirm the changes youve made. 5. In the field in the message window, type YES. This is case sensitive. 6. Click on the Update Settings button. The HCP system restarts with the new settings. This takes a few minutes, during which the system is inaccessible to users and applications.

The network configuration for an HCP system includes:
The IP address of the gateway for routing requests from the HCP
system to the front-end network
The IP address the HCP nodes use to multicast messages to all other
nodes
The maximum packet sizes for data sent on the front-end and back-end
networks
For each node, the front-end IP address and subnet mask and the backend IP address and subnet mask For an HCP SAIN system that uses CB 320 servers, the network configuration also includes the IP addresses of the server chassis. For information on modifying this list, see Adding and removing chassis IP addresses below.
94
This network configuration is set during the Appliance OS and HCP core software installation. Changes in the network topology, however, or a move to a new location may make changes to this configuration necessary. You can change the first three items listed above at any time while the HCP system is running. You can change the network settings for a node only while the node is running and available. All nodes must be on the same front-end subnet and the same back-end subnet. If you change the front- or back-end subnet for one node, you need to change it for all nodes. You should do this in a single operation. Important: Do not change the multicast IP address while any nodes are unavailable. If you do, the unavailable nodes will not be able to rejoin the system. Notes:
Do not change the multicast IP address, front- or back-end MTU, or

back-end IP addresses or subnet masks without explicit instructions from HCP support personnel.
You cannot change the last octet in the back-end IP address for a node. You cannot change the back-end subnet mask for a node to anything
other than 255.255.255.0. For more information on the gateway router and the front- and back-end IP addresses and subnet masks for the HCP nodes, see Information for an OS installation on page 2-2. For more information on the multicast address for an HCP system, see Networking on page 2-4. To change network settings for an HCP system: 1. In the top-level menu in the System Management Console, mouse over Configuration to display a secondary menu. 2. In the secondary menu, click on Network Settings.
95
3. On the Network Settings page, do one or more of these:
To change the IP address of the front-end gateway router, in the Default Gateway field, type the new gateway router IP address. Note: Changing this IP address may disrupt the network connection between your computer and the HCP system, thereby causing the System Management Console session to end. If this happens, start a new Console session on a computer with access to the new front-end network.
To change the multicast IP address for the HCP system, in the Multicast Address field, type the new multicast IP address. This address must begin with 238. To change the maximum packet size for the front-end network, in the Front-end MTU field, select the packet size you want. To change the maximum packet size for the back-end network, in the Back-end MTU field, select the packet size you want. To change network settings for an available node, do one or more of these:
In the Front-end IP field for the node, type a new front-end IP address. Note: If you change the front-end IP address for the node to which your System Management Console session is connected, the session ends. If this happens, start a new session.
In the Front-end Netmask field for the node, type a new subnet mask for the front-end IP address. In the Back-end IP field for the node, type a new back-end IP address. In the Back-end Netmask field for the node, type a new subnet mask for the back-end IP address.
4. Click on the Update Settings button. A warning message appears asking you to confirm the changes youve made.
96
Adding and removing chassis IP addresses
5. In the field in the message window, type YES. This is case sensitive. 6. Click on the Update Settings button. The HCP system restarts with the new settings. This takes a few minutes, during which the system is inaccessible to users and applications.
Adding and removing chassis IP addresses

In an HCP SAIN system that uses CB 320 servers, HCP needs to know the chassis IP address of for each server. The list of chassis IP addresses is initially specified during the installation of the HCP core software. You need to modify this list if you add or remove a CB 320 server to or from the system. You also need to modify this list if the chassis IP address of a server changes. Tip: If you remove all the blades from a server without removing the server chassis from the system, you should remove the applicable chassis IP address from the list of chassis IP addresses. This way, the HCP System Management Console wont report unnecessary IPMI information for that server. To modify the list of chassis IP addresses: 1. In the top-level menu in the System Management Console, mouse over Configuration to display a secondary menu. 2. In the secondary menu, click on Network Settings. 3. In the Chassis IP Addresses section on the Network Settings page:
To add one or more IP addresses to the list:

1. In the field above the list, type either a single IP address or a
comma-separated list of IP addresses.

2. Click on Add.
To remove a single IP address from the list, click on the delete control ( ) for that address. To remove all the IP addresses from the list, click on Delete All.
The changes you make to this list take effect immediately.
97
Changing time settings
Changing time settings

HCP displays its current time, including its time zone, above the top-level menu in the System Management Console whenever a page is first displayed or is refreshed. The time source for the HCP system can be internal (that is, from the nodes themselves) or external (that is, from one or more time servers outside the system). The time source, current time, and time zone for an HCP system are initially set during the core software installation. However, you can change these time settings at any time while the system is running. For example, you might change the current time immediately after installing the system if the time was incorrectly specified during the installation. Important: Changing the time source or current time for an HCP system has certain implications and may cause the system to no longer be in compliance with some government regulations:
Changing the time source from internal to external or from one

external time server to another can cause the current system time to shift either forward or backward. Changing the time source from external to internal or leaving it as internal can also cause a shift in the current system time if you explicitly specify a new current time.
When a shift occurs in the current system time, HCP does not adjust
object retention times. As a result of a shift forward, some objects due to expire in the future may immediately become expired. As a result of a shift backward, some objects may take longer to expire than originally intended. HCP stores all times (such as creation dates and retention settings) in Coordinated Universal Time (UTC) and uses its time zone setting only for display purposes. Changing the time zone for an HCP system, therefore, has no affect on object retention. For more information on HCP time settings, see Time on page 2-5. To change time settings for an HCP system: 1. In the top-level menu in the System Management Console, mouse over Configuration to display a secondary menu. 2. In the secondary menu, click on Time Settings.
98
Changing the HCP serial number
3. On the Time Settings page, do one or more of these:
To change the time source, in the Time Servers field, type either internal or the IP addresses or hostnames of one or more time servers. Use commas to separate multiple IP addresses and hostnames. To change the current system time when the time source is internal, in the Current Time field, type the new time. The format for the time is MMDDhhmmYYYY, where MM is the two-digit month, DD is the two-digit day, hh is hours on a 24-hour clock, mm is minutes, and YYYY is the year. If the time source is internal and you leave this field blank, the current system time doesnt change.
To change the time zone, in the Time Zone field, select the new time zone.
4. Click on the Update Settings button. A warning message appears asking you to confirm the changes youve made. 5. In the field in the message window, type YES. This is case sensitive. 6. Click on the Update Settings button. The HCP system restarts with the new settings. This takes a few minutes, during which the system is inaccessible to users and applications.
Changing the HCP serial number

An HCP system has a serial number for the purpose of identification. This serial number is assigned to the system during installation. You can change the serial number for an HCP system at any time while the system is running. You would need to do this, for example, if the number had been entered incorrectly when HCP was installed. For more information on serial numbers, see Serial number on page 2-7.
99
Changing the HTTP persistent connection timeout interval
To change the serial number of an HCP system: 1. In the top-level menu in the System Management Console, mouse over Configuration to display a secondary menu. 2. In the secondary menu, click on Miscellaneous Settings. 3. In the Serial Number field, type the new serial number for the system. Valid values can contain only alphanumeric characters, hyphens (-), underscores (_), number signs (#), and spaces. 4. Click on the Update Settings button.
Changing the HTTP persistent connection timeout interval

HCP supports HTTP persistent connections. A persistent connection is one that is kept open for a set amount of time so that it can be used by a subsequent request. Persistent connections enhance performance because they avoid the overhead of opening and closing multiple connections. The HTTP persistent connection timeout setting applies across all namespaces. By default, this setting is ten minutes. If namespace users report that this interval is too long or, more typically, too short, you can change it. To change the HTTP persistent connection timeout interval: 1. In the top-level menu in the System Management Console, mouse over Configuration to display a secondary menu. 2. In the secondary menu, click on Miscellaneous Settings. 3. In the HTTP Persistent Connection Timeout Interval field, type the new interval, in milliseconds. Valid values are integers equal to or greater than zero. 4. Click on the Update Settings button.
Enabling or disabling zero-copy failover

Zero-copy failover is the process of one HCP node automatically taking over management of storage previously managed by another node that has failed. Its available only for systems that use cross-mapped SANattached storage.
910
Enabling or disabling DPL one in RAIN systems
Zero-copy failover is first enabled or disabled during HCP installation. Its automatically disabled and then reenabled during an online upgrade of the HCP system. You can enable or disable zero-copy failover at any time while the HCP system is running. However, you should do this only when specifically told to do so by HCP support personnel. For more information on zero-copy failover, see Zero-copy failover on page 1-9. To enable or disable zero-copy failover: 1. In the top-level menu in the System Management Console, mouse over Configuration to display a secondary menu. 2. In the secondary menu, click on Miscellaneous Settings. 3. On the Miscellaneous Settings page, select or deselect the Enable Zerocopy Failover option, as applicable. 4. Click on the Update Settings button.
Enabling or disabling DPL one in RAIN systems

For an HCP RAIN system, the default system-level DPL setting is two. HCP system administrators can change this value to two, three, or four, but not to one. Tenant administrators can set the DPL for a namespace to two, three, four, or dynamic, but not to one. (By default, DPL one is allowed on HCP SAIN systems.) In certain circumstances, allowing a DPL of one in a RAIN system may be appropriate. For example, if a namespace is being replicated, the objects in it are protected without needing the redundancy provided by a higher DPL. You can reconfigure an HCP RAIN system to allow a value of one for the system-level DPL setting and namespace DPLs. You can also change the system back to disallowing it. These considerations apply:
Enabling DPL one does not cause any change to either the system-level
DPL setting or any namespace DPLs.
With DPL one enabled, HCP system administrators can change the
system-level DPL setting to one, and tenant administrators can set namespace DPLs to one.
911
Changing the region count
If DPL one is disabled while the system-level DPL setting is one, that
setting automatically changes to two.
If DPL one is disabled while the DPL for a namespace is explicitly set to
one (not dynamic), the namespace DPL does not change. To enable or disable DPL one: 1. Using the service SSH key, use SSH to log into any storage node in the HCP system as service. For instructions on doing this, see Appendix D, Setting up SSH key authentication. 2. Do either of these:
To enable DPL one, enter:

admin jvm cluster enableDpl1
To disable DPL one, enter:

admin jvm cluster disableDpl1

A region is an HCP-internal logical grouping of objects and certain types of system data (such as system log messages and usage statistics). HCP uses regions to distribute object metadata and system data among the storage nodes in the system. Objects and other data are assigned to regions based on specific properties. For objects, this property is the object directory path.
About the region count

When HCP is first installed, it has a default number of regions based on the number of storage nodes in the system. This number remains the same, even when storage nodes are added to or removed from the system. As more nodes are added to an HCP system, each node manages fewer regions. This results in the nodes being underused. On the other hand, if enough nodes are removed from the system, the remaining nodes become overloaded.
912
To accommodate a growing or shrinking HCP system, you can explicitly change the number of regions (referred to as the region count). The minimum and maximum values for the region count are determined by the system configuration. The number you specify:
Must be at least four times the number of storage nodes in the system Cannot be less than one quarter of the current region count Cannot be greater four times the current region count
As a general rule, an HCP system should have from eight to twelve regions per storage node. When you change the region count, HCP splits or merges existing regions to arrive at the new number of regions. While this is happening, HCP services may pause temporarily. Additionally, system responses to client requests may be slightly slower. You cannot change the region count while the HCP system is being upgraded or while a data migration is in progress. Conversely, you cannot start an upgrade or data migration while the number of regions is changing. You can cancel a change to the region count while HCP is in the process of making the change. When you do this, HCP finishes any split or merge that was in progress, and the region count then remains where it is at that moment. The count is not rolled back to the previous number of regions.
Changing or canceling a change to the region count

To change or cancel a change to the region count: 1. Using the service SSH key, use SSH to log into the highest-numbered storage node in the system as service. For instructions on doing this, see Appendix D, Setting up SSH key authentication. 2. Enter this command to open the HCP install shell:
913
914
6. From the HCP Service menu, enter 7 to display the Change Region Count menu.
To change the region count:

1. Enter 1.
HCP Setup displays the current number of regions in the HCP system and prompts for the new region count.
2. Enter the number of regions you want. 3. In response to the confirming prompt, enter y or yes to confirm
your entry or n or no to try again. When you enter y or yes, HCP begins splitting or merging regions, as applicable, and the HCP Service menu reappears.
To cancel a change to the region count:

1. Enter 2.
915
Enabling or disabling support for replication


3. In response to the next confirming prompt, enter y or yes to
confirm your entry or n or no to try again. When you enter y or yes in both step 2 and this step, HCP stops splitting or merging regions, as applicable, and the HCP Service menu reappears. 8. From the HCP Service menu, enter q to return to the HCP 4.1 Configuration menu. 9. In response to the confirming prompt, enter y or yes to confirm your entry or n or no to try again. 10.From the HCP 4.1 Configuration menu, enter q to log out of the install shell. 11.In response to the confirming prompt, enter y or yes to confirm your entry or n or no to try again.

Replication is an add-on feature for HCP. If the feature was included in the customers initial HCP purchase, it was enabled during HCP installation. If the customer buys replication after HCP is installed, you need to enable the feature as a separate operation. Occasionally, you may also need to remove support for replication by disabling the feature. You do not need to restart the HCP system when enabling the replication feature. However, you do need to restart the system to complete the process of disabling the feature. For more information on replication, see Replicating Tenants and Namespaces.
Enabling support for replication

To enable support for replication: 1. Using the service SSH key, use SSH to log into any storage node in the system as service. For instructions on doing this, see Appendix D, Setting up SSH key authentication.
916
2. Enter:
admin jvm replication enable
Disabling support for replication

To disable support for replication: 1. Ensure that the customer has deleted all replication links for which the system is either the primary system or the replica. 2. Using the service SSH key, use SSH to log into any storage node in the system as service. For instructions on doing this, see Appendix D, Setting up SSH key authentication. 3. Enter:
admin jvm config set cluster replication.enabled false
4. Restart the HCP system:

a. Log into the HCP System Management Console with the
administrator or service role.

b. In the top-level menu, click on Nodes. c. At the bottom of the Nodes page, click on the Restart System button.
The Restart System window appears.

d. In the Reason field, type the reason why youre restarting the
system. This text can be up to 1,024 characters long and can contain any UTF-8 characters, including white space.
e. Click on the Restart System button.
917
918
10
Managing an HCP system
For a variety of reasons, a few HCP system management activities are considered to be more advanced than general system monitoring and configuration. These activities are:
Managing HCP services Resetting the checkpoint for a replication link Disabling chained writes Accessing the administrative interface for the HCP search facility
To manage HCP services, you need an HCP System Management Console user account with the service role. To reset replication checkpoints or disable chained writes, you use SSH to log into an HCP node. To do this, you need a laptop computer connected to the HCP system front- or back-end network. For information on this, see Running HCP Setup from a service shell on page 3-9. This chapter describes the procedure for each of the activities listed above. For information on creating a user account with the service role, see Administering HCP.
Managing an HCP system Installing and Maintaining an HCP System
101
Managing HCP services

HCP services are responsible for optimizing the use of system resources and maintaining the integrity and availability of the stored data. Each of the eleven services protection, content verification, scavenging, indexing, shredding, compression, duplicate elimination, disposition, garbage collection, capacity balancing, and replication performs a specific function that contributes to the overall health and viability of the system. For in-depth information on HCP services except replication, see Administering HCP. For information on replication, see Replicating Tenants and Namespaces.
Service management
Normally, HCP services require no human intervention; they run according to a schedule or when triggered by specific events or are ongoing. With an HCP System Management Console user account with the monitor or administrator role, you can monitor services. However, with these roles, you have limited ability to affect when and how most services work. With the service role, you have the ability to start, stop, enable, and disable individual services:
Starting a service starts a full service run unless the service is already
running. If the service is already running, starting it has no effect.
Stopping a service stops the current run. Enabling a service allows it to run as it normally would. Enabling a
service does not start it.
Disabling a service prevents it from running at all. If the service is

already running, disabling it also stops that run. Disabled services remain disabled across a system restart. Note: Do not disable services for any significant length of time. While services are disabled, the HCP system cannot ensure its own health.
102
You can perform all of these actions on all the services except indexing, shredding, and replication. Examples of when you would perform these actions include starting a particular service to force repairs ahead of schedule or disabling a service to temporarily lighten the load on the system. Services, except indexing, shredding, and replication, use internal storage segments to limit the load they put on the system. When running as scheduled or when triggered, each service processes one segment on each node during each run. When started manually, however, each service processes all the segments on each node sequentially in the same run. This can significantly increase the load the service puts on the system. The indexing service is ongoing and is managed differently from the other services. The shredding service is ongoing. You cannot start, stop, enable, or disable it. The replication service works through a separate mechanism. For more information on managing this service, see Replicating Tenants and Namespaces.
Starting, stopping, enabling, and disabling services

You start, stop, enable, and disable services on the Overview page of the HCP System Management Console. To perform these actions: 1. In the top-level menu in the HCP System Management Console, click on Overview. 2. In the Services section on the Overview page, do one of these:
To start a service, click on the start control ( To stop a service, click on the stop control (
) for that service. ) for that service. ) for that
To enable a service, click on the enable control ( service. To disable a service, click on the disable control ( service.
) for that
While a service is disabled, its start/stop control is hidden.
103
Resetting the checkpoint for a replication link
Resetting the checkpoint for a replication link

The replication service periodically checkpoints the progress of replication or data recovery on each configured replication link. If replication or recovery is not proceeding as expected on a link, HCP support personnel may direct you to reset the link checkpoint to a specific time in the past. This causes the replication service to restart replication or data recovery on the link, as applicable, starting with objects with the oldest changes made at or after the time you specify. Important: Do not reset replication checkpoints unless explicitly told to do so by HCP support personnel. Link IDs To reset the checkpoint for a replication link, you need to know the link ID. To find the ID, you use a command that lists all the links for which the HCP system is either the primary system or the replica. The output of this command includes the name, ID, status, and other information about each link. Heres an example of the first part of the output for one link:
'cac3d212-f2b1-4b42-9891-e8cb0a25372d' : { 'inboundContentSelections' : { }, 'isUsingScheduler' : False, 'linkId' : 'cac3d212-f2b1-4b42-9891-e8cb0a25372d', 'linkName' : 'MA-CA', 'linkSchedule' : '[{"start":0,"intensity":"MEDIUM"}]', 'linkStatus' : 'REPLICATING', . . .
In this output, the linkName field specifies the name of the replication link, and the linkId field specifies the link ID. Finding the link ID and resetting the checkpoint To find the ID of a replication link and reset the checkpoint for that link: 1. Using the service SSH key, use SSH to log into any storage node as service. For instructions on doing this, see Appendix D, Setting up SSH key authentication. 2. Enter this command to list the currently configured replication links:
admin jvm replication list
104
Disabling chained writes
3. In the list of replication links, find the ID for the link you want. 4. Enter this command to reset the checkpoint for the link:
replication setProgress link-id checkpoint-time
checkpoint-time is the time to which you want to reset the checkpoint, specified as seconds since January 1, 1970, at 00:00:00. For example, this command resets the checkpoint for link MA-CA to October 20, 2010, at 9:45 a.m.:
replication setProgress cac3d212-f2b1-4b42-9891-e8cb0a25372d 1287567900
Disabling chained writes

Under certain circumstances, HCP support personnel may determine that you need to disable chained writes for an HCP system. Chaining is a technique both CIFS and NFS use to enhance performance on write operations. However, this technique does not always work well with some clients. To disable chained writes, you need to change the initial system configuration on each node in the HCP system. You can do this on each node individually, or you can do this on one node and use the cluster_put system service tool to propagate the change to the other nodes. For information on the cluster_put tool, see Using the system service tools on page 11-15. After you change the initial system configuration on each node, you need to restart the HCP system for the change to take effect. To do this, use the Restart System button on the Nodes page in the HCP System Management Console. Note: The setting to disable chaining is not retained across system upgrades. To disable chained writes on a node: 1. Using the root SSH key, use SSH to log into the node as root. For instructions on doing this, see Appendix D, Setting up SSH key authentication. 2. Change to the /opt/arc/config directory:
cd /opt/arc/config
105
Accessing the HCP search facility administrative interface
3. Using a text editor such as vi, open the init.conf file for editing. 4. Remove the comment character (#) from this line:
# RISFS_DATA_CHAIN="--disable-chaining"
5. Save and close the init.conf file.
Accessing the HCP search facility administrative interface

The administrative interface for the HCP search facility allows you to provide advanced support for that facility. You should make changes in this interface only when told to do so by HCP support personnel. The username and initial password for logging into the administrative interface are: Username: admin Password: Chang3Me! You should change the password for the admin user the first time you log in. Access to the administrative interface for the HCP search facility is enabled through the HCP System Management Console. For information on enabling this access, see Administering HCP. To log into the administrative interface for the HCP search facility: 1. In a web browser window, go to the search facility administrative interface. The URL for this interface consists of the front-end IP address of the lowest-numbered search node followed by port 16000 (for example, http://192.168.210.19:16000). The interface prompts for a username and password. 2. In the User Name field, type: admin 3. In the Password field, type the password for the admin user. 4. Click on the OK button. The search facility administrative interface opens.
106
11
Troubleshooting an HCP system
The HCP system is designed to be self-healing and to run without human intervention. Occasionally, however, things do go wrong. As a service provider, you can do some basic troubleshooting, but more often, your role is to help HCP support personnel resolve customer problems. This chapter describes some actions you can take to help troubleshoot problems with the HCP system:
Understanding service alerts Retrieving the HCP Setup logs Resetting the install user password Reset a security user password Setting HCP internal logging levels Downloading the logs for a single node Creating a snapshot of the HCP JVM Using the HCP system service tools Viewing the HCP system status pages Ejecting the CD tray from a node
Troubleshooting an HCP system Installing and Maintaining an HCP System
111
Understanding advanced alerts
Understanding advanced alerts

The HCP System Management Console uses graphics to report system status on the Overview, Nodes, and Storage/Search Node pages. These graphics, called alerts, are accompanied by text, which you can see by mousing over them. Each graphic represents some particular element of the system. The accompanying text can report either that the element is okay or that problems exist. Most alerts are visible when you log into the Console with the monitor or administrator role. For information on these alerts, see Administering HCP. A few alerts, considered advanced, are visible only when you log in with the service role. The table below describes these advanced alerts.
Alert graphic
Messages
Online upgrade in progress
Description
The HCP system software is currently being upgraded while the system is online. For information on upgrading, see Chapter 4, Upgrading an HCP system.. The HCP system software has been upgraded with the system offline, and the upgrade now needs to be completed. The HCP system has been upgraded. You need to commit the upgrade now. For instructions on doing this, see Step 6: Complete the upgrade on page 4-25. For an HCP system that uses SAN-attached storage, the storage is configured for multipathing. For information on this feature, see Multipathing on page 1-9. Zero-copy failover is enabled. For information on this feature, see Zero-copy failover on page 1-9. Zero-copy failover is disabled.
Offline upgrade in progress
Click here to complete the upgrade
Multipath enabled
ZCF enabled
ZCF disabled
Repository encrypted using length-bit encryption-algorithm
All data and metadata in the repository is encrypted using the specified encryption algorithm. For information on this feature, see Security on page 2-8.
112
Retrieving the HCP Setup logs
Retrieving the HCP Setup logs

As HCP Setup runs on each node, it reports on its processing in a log file on the node. This file is stored as /boot/deploy_share/arc-deploy-node-number.log, where node-number is the number of the current node (for example, /boot/ deploy_share/arc-deploy.016.log). If HCP Setup fails to complete its processing, it collects the relevant logs from all nodes in the HCP system and stores them in the /home/install/arcdeploy/logs directory on the node on which its running. (At the beginning of a run, HCP Setup compresses any existing log files into a single file and stores that file in /home/install/arc-deploy.) When requesting help from HCP customer support after the failure of a procedure that uses HCP Setup, you need to provide support personnel with the entire contents of the /home/install/arc-deploy/logs directory, including all its subdirectories, recursively, along with the logs from any previous runs of HCP Setup. If the failure is due to a node powering off or becoming inaccessible due to a network failure, HCP Setup may not be able to retrieve the relevant log file from that node. In this case, you need to use SCP to copy the file from that node to the /home/install/arc-deploy/logs directory on the node on which HCP Setup ran. Once the /home/install/arc-deploy/logs directory contains all the relevant logs, you can package the logs and send the resulting file to HCP customer support. To retrieve the contents of the /home/install/arc-deploy/logs directory, as well as the logs from any previous runs of HCP Setup, and compress them into a single file, use this command:
tar zcf adlogs.tgz /home/install/arc-deploy/log*
Resetting the install user password

When you first log into an HCP node to perform a system installation, you are required to change the install user password. For all subsequent activities that require a node login, you need to log into the applicable node using the new password. If the new password is lost or if it is compromised and needs to be changed, you need to reset it. To do this: 1. Using the service SSH key, use SSH to log into any storage or search node in the HCP system as service. For instructions on doing this, see Appendix D, Setting up SSH key authentication.
113
Resetting a security user password
2. Enter this command to change the install user password:

sudo /sbin/install_passwd -c node-ip-address-list
node-ip-address-list is a comma-separated list of the back-end IP addresses of all the nodes (both storage and search) in the HCP system. Spaces are not allowed.
For example:
sudo /sbin/install_passwd -c 10.4.1.15,10.4.1.16,10.4.1.17,10.4.1.18,10.4.1.19
A prompt for a new password appears.
3. Enter a new password for the install user. (As you type, the characters do not show.) The new password must follow the standard rules for Unix passwords. In particular, to be valid, a password must include at least one character from two of these three groups: alphabetic, numeric, and special characters. Also, a password cannot be a word found in the dictionary. A prompt to confirm the new password appears.
4. Enter the new password again. The install user password changes on all nodes in the HCP system.
Resetting a security user password

For any given customer, the likelihood of all system-level users with the security role forgetting their passwords at the same time is extremely low. However, should this happen, the customer would have no administrators who could manage user accounts. To fix this situation, you need to reset the password of at least one security user. To do this, you need to know the system-generated user ID.
114
Managing the HCP internal logs
To find a user ID, you use a command that lists all the currently defined user accounts. The output of this command includes the user ID, username, roles, and other information about each account. Heres an example of the output for one account:
'authenticationScheme' : 'LOCAL', 'description' : 'Primary HCP administrator', 'enableAtTime' : 'Wed Dec 31 19:00:00 EST 1969', 'failedLoginCount' : '2', 'fullName' : 'Lee Green', 'isEnabled' : True, 'isHidden' : False, 'isPasswordChangeRequired' : False, 'isPaswordExpired' : False, 'lastLoginTime' : 'Wed Nov 18 12:12:42 EST 2009', 'lastPasswordChangeTime' : 'Mon Nov 09 10:36:46 EST 2009', 'name' : 'lgreen', 'parentUUID' : '34142e35-6d9d-4d99-acd4-85656444a29a', 'roles' : 'MONITOR,SYSTEM,SECURITY', 'userId' : '101'
To find the ID of a security user and reset that users password: 1. Using the service SSH key, use SSH to log into any storage node in the system as service. For instructions on doing this, see Appendix D, Setting up SSH key authentication. 2. Enter this command to list the currently defined user accounts:
admin jvm adminUser list
3. In the list of user accounts, find the user ID you want. 4. Enter this command to reset the password for the user you want:
admin jvm adminUser update cluster user-id --password new-password
For example:
admin jvm adminUser update cluster 101 --password Chang3Me!

The HCP internal logs record the activity of various C and Java components of the HCP core software. You can control the level of detail recorded for these components on a given node by setting the logging levels on that node.
115
Customers can use the HCP System Management Console to download the internal logs to send to the HCP support center. The logs can help support personnel identify and resolve problems with the system. A log download from the System Management Console includes all the logs for each node. As a service provider, you can also download the logs for a single node. When HCP prepares the internal logs for downloading on a node thats a Hitachi or Dell server, it includes the file in which it saves the server system event log. For information on this file, see Server system event log on page 11-8. For information on downloading the HCP internal logs from the System Management Console, see Administering HCP. For information on downloading the logs for a single node, see Downloading the logs for a single node on page 11-12.
HCP component log files

Each HCP component has two types of log files one for internal messages that come from the component itself, the other for standard output and standard error messages. Note: To view the contents of the HCP internal logs for a node without downloading the logs, you need to use SSH to log into the node as root. Current log files The log files for the current day for each component are located in the /var/ ris/logs directory. Symbolic links to these files are located in the /opt/ris/logs and /opt/arc/logs directories. The format for the name of the current file containing the internal messages differs for C and Java components:
For each C component, the file name has this format:

component-name.node-number.log
For example, the current file with internal messages for the timelord component on the node with the back-end IP address 10.4.1.15 is timelord.15.log.
116
For the Java virtual machine (JVM), which has multiple components,
messages are written to different files based on their severity level. Also, HCP creates multiple instances of the JVM log files to support concurrent logging. Each file name has this format:
jvm1.node-number.instance.severity-level
For example, one of the current files with WARNING-level internal messages for the JVM component on the node with the back-end IP address 10.4.1.15 is jvm1.15.0.WARNING. For both the C and Java components, the name of the current file containing the standard output and error messages has this format:
R00.node-number.component-name-out
For example, the current file with standard output and error messages for the timelord component on the same node as above is R00.15.timelord-out. Daily logs During any given day, if the .log file for a component grows larger than three MB, HCP appends .0 to the file name and starts a new .log file. For example, if the .log file for the timelord component on node 10.4.1.15 grows larger than three MB, HCP renames the file timelord.15.log.0 and creates a new timelord.15.log file. If it happens again, HCP increments the timelord.15.log.0 file name to timelord.15.log.1, changes the name of the timelord.15.log file to timelord.15.log.0, and creates a new timelord.15.log file. This can repeat up to 99 times, after which HCP deletes the oldest file each time a new file is started. If, during the day, a node restarts, HCP increments the first part of the -out file name for each component and creates a new -out file. For example, if node 10.4.1.15 restarts, HCP changes the name of the R00.15.timelord-out file to R01.15.timelord-out and creates a new R00.15.timelord-out file. This can also repeat up to 99 times. Log rotation HCP rotates all the log files on a node daily at 4:02 a.m. It does this by compressing all the files into a single tar file with this name:
logs-YYYYMMDD-hhmmss.tar.bz2
117
It stores this file in the /var/ris directory. If the storage allocated for the /var/ris directory fills beyond 70 percent, HCP moves all the tar files in the directory to the /RIS/archivehighest-archive-number directory. If that directory fills beyond 70 percent, HCP deletes the tar files in it one at a time, starting with the oldest, until the directory storage is less than 70 percent full. Each time it rotates the log files, HCP checks both the /var/ris and /ris/archivehighest-archive-number directories for tar files that are over 35 days old. If it finds any, it deletes them.
Server system event log

The server system event log (SEL) records hardware events, such as malfunctioning fans or power supplies, that happen within a node. To prevent the SEL on a node from growing too large, once a week, HCP: 1. Writes the current contents of the SEL to the end of the /var/log/ipmi_ sel.log file on the node. 2. Clears the SEL.
Note: Timestamps in the SEL are in UTC.
Changing logging levels

The logging level for an HCP component on a node determines how much information is logged for that component. You can change the logging level for any given component on any given node. Logging level changes are persistent across node restarts for the JVM but not for C components. The procedure for changing the logging level differs for C components and the JVM.
Logging levels
Logging levels correspond to message severity levels:
FINEST Messages at this level record all processing steps. Logging

at this level provides the greatest amount of detail.
FINER Messages at this level record enough processing steps to

enable full sequence tracing.
118
FINE Messages at this level record basic sequence execution. INFO Messages at this level record normal behavior. This is the
default logging level for C components.
WARNING Messages at this level record availability events and

other rare, but not catastrophic, events. This is the default logging level for the JVM.
SEVERE Messages at this level record errors and unexpected or

extremely unusual events. Logging at this level provides the least amount of detail. For any given logging level, the messages logged are those with the corresponding severity level, as well as those with greater severity levels. Note: The greater the level of activity logged, the higher the load on the HCP system. Do not increase logging levels unless explicitly told to do so by HCP support personnel. Caution: Do not leave logging levels set higher than the defaults for more than a few minutes. Doing so, especially when the system is experiencing a heavy load, can quickly fill all available storage space and result in HCP system failure.
Changing the logging level for C components

To change the logging level for a given C component on a node: 1. Using the service SSH key, use SSH to log into any node in the HCP system as service. For instructions on doing this, see Appendix D, Setting up SSH key authentication. 2. Enter this command to change the logging level for the component you want:
admin -a node-ip-address:port-number loglevel logging-level
In this format:
node-ip-address is the back-end IP address of the node on which you want to change the logging level.
119
port-number is the component-dependent port number. The table below shows the port number for each component for which you can set the logging level.
Component
mailgate risfs-0 risfs-1 slabd storman-n 2500 20000 20001 1234 10000 + n Each logical volume has its own STORMAN process. These processes are numbered starting from 1 (one) on each node. sysmon timelord 12000 12001
Port
logging-level is one of the logging levels listed above. This is case sensitive.
Changing the logging level for the JVM

The JVM has multiple components. You can change the logging level for individual components or for the entire JVM. When a change is necessary, HCP support personnel will tell you exactly what to change. The instructions for changing to something other than the default logging level differ from the instructions for changing back to the default. Changing to a nondefault JVM logging level To change the logging level for the JVM or a JVM component on a node to a level other than the default: 1. Using the root SSH key, use SSH to log into the node as root. For instructions on doing this, see Appendix D, Setting up SSH key authentication. 2. Change to the /opt/ris/config directory:
cd /opt/ris/config
3. Use a text editor such as vi to open the log.properties file for editing.
1110
4. Add a line specifying the new logging level to the end of the file. For this line, use the exact format provided by HCP support personnel. Heres an example of a line that specifies a new logging level for the com.ris.server component of the JVM:
com.ris.server.level=INFO
To change the logging levels of multiple components, add one line for each change. 5. Save and close the log.properties file. 6. To put the new logging level into effect, enter:
/opt/ris/bin/admin RELOAD_JAVA_LOG_CONFIG
Tip: To make the same change on all other nodes: 1. Use the cluster_put tool to propagate the log.properties file to the other nodes. 2. Use the cluster_ssh tool to execute the admin command above on the other nodes. For information on cluster_put and cluster_ssh, see Using the system service tools on page 11-15. Changing JVM components back to the default logging level To change the logging levels for all JVM subcomponents on a node back to the default: 1. Using the root SSH key, use SSH to log into the node as root. For instructions on doing this, see Appendix D, Setting up SSH key authentication. 2. Change to the /opt/ris/config directory:
cd /opt/ris/config
3. Copy the existing log.properties file to log.properties.n, where n creates a unique name for the file; for example:
cp log.properties log.properties.1
1111
This saves the file with the nondefault logging levels, so you can use it again if needed. 4. Copy the /opt/ris/config/log.properties.nologs file to /opt/ris/config/log.properties:
cp log.properties.nologs log.properties
5. To put the default logging levels into effect, enter:

/opt/ris/bin/admin RELOAD_JAVA_LOG_CONFIG
Downloading the logs for a single node

To help with troubleshooting, you can download the internal logs for a single node and send them to your HCP support center. For ease of handling, HCP can collect the logs into a single file on the node before you download them. To download the logs for a single node: 1. Using the service SSH key, use SSH to log into the node as service. For instructions on doing this, see Appendix D, Setting up SSH key authentication. 2. Enter this command to collect the logs into a single file:
sudo /opt/arc/bin/logdl newest-day number-of-days [nocrypt] > output-file-spec.tar.bz2.gpg
In this format:
newest-day is the number of days ago for the most recent logs you want. number-of-days is the number of days of logs to download, working backwards in time from the value specified for newest-day.
nocrypt specifies that the downloaded logs should not be encrypted.
This parameter is optional.
output-file-spec.tar.bz2.gpg specifies the directory path and file name for the compressed file containing the downloaded logs. output-filespec can include an absolute path or a path relative to the current directory.
1112
3. Using an SCP tool such as scp or winscp, download the file with the collected logs to another computer. Tip: scp is already installed on each node in the HCP system.
Here are two examples of the logdl command:
This command collects only the logs from the current day:
sudo /opt/arc/bin/logdl 0 0 > temp/todaysLogs.tar.bz2.gpg
This command collects the logs from seven days ago through six days
ago (that is, it packages two days worth of logs):
sudo /opt/arc/bin/logdl 6 2 > temp/oldLogs.tar.bz2.gpg
Heres an example of an scp command that downloads the file created by the first logdl command shown above:
scp temp/todaysLogs.tar.bz2.gpg lgreen@192.168.57.42:/home/lgreen/hcpLogs
Managing encryption keys for log downloads

When downloading the internal logs from the System Management Console or from a single node, HCP encrypts them to ensure their privacy. By default, HCP uses a system-supplied key for the encryption. Only HCP support personnel have the private key required to decrypt the logs. Rarely, however, a customer may need access to the content of the downloaded logs. For example, to meet confidentiality requirements, the customer may need to remove all object names from the logs before the logs are given to the support team. To give a customer access to the log content, you upload one or more encryption keys for HCP to use when encrypting the logs (in addition to using the system-supplied key). You need to upload the keys before the customer begins the log download procedure. Once the logs are downloaded, the customer can use the private key corresponding to any one of the uploaded keys to decrypt and view or edit the log content.
1113
Creating a snapshot of the JVM
In addition to uploading an encryption key, you can list the keys HCP currently has and delete any of those keys. When you list the keys, the information displayed for each key includes the key fingerprint. When deleting a key, you identify it by its fingerprint. Important: Do not delete the system-supplied public encryption key or signing sub key, which are identified by <support@hds.com> in the user ID. If you do, support personnel will not be able to decrypt the downloaded logs. To upload, list, and delete the encryption keys used for log downloads: 1. Using the service SSH key, use SSH to log into any storage node in the HCP system as service. For instructions on doing this, see Appendix D, Setting up SSH key authentication. 2. Do one of these:
To upload an encryption key, enter:

admin jvm logdl addKey key-file-spec
For example:
admin jvm logdl addKey /home/service/temp/logEncryptKey
To list the encryption keys HCP currently has, enter:

admin jvm logdl listKeys
To delete an encryption key, enter:

admin jvm logdl deleteKey key-fingerprint
For example:
admin jvm logdl deleteKey E96FFEA44F67290FBF2DF49D77D9ED35CD2B3BF8
Creating a snapshot of the JVM

To help identify a problem with an HCP system, your HCP support center may request a snapshot of the JVM on one or more nodes. This snapshot is a combination of a JVM heap dump and a Java core file. HCP does not automatically create JVM snapshots. When asked for one, you need to explicitly generate it.
1114
Using the system service tools
When you generate the JVM snapshot, HCP appends it to a file in the /var/ ris/logs directory, which is the directory that contains the HCP internal logs. The name of this file is always R00.node-number.jvm-out (for example, R00.16.jvm-out). At any time, if the JVM on a node restarts, HCP increments the first part of the file name to R01 and creates a new R00.node-number.jvm-out file. If the JVM restarts again during the same day, HCP increments R01 to R02 and R00 to R01. This can repeat up to 99 times. When HCP rotates the log files each day, it includes any JVM snapshot files that start with R01 or higher. For information on how HCP rotates the log files, see Log rotation on page 11-7. When the HCP internal logs are downloaded either from the System Management Console or from an individual node, the JVM snapshot files for the applicable time period are automatically downloaded with them. To generate a snapshot of the JVM on a node: 1. Using the service SSH key, use SSH to log into the node as service. For instructions on doing this, see Appendix D, Setting up SSH key authentication. 2. To generate the snapshot files, enter:
sudo /opt/arc/bin/generate-thread-dump.sh.

HCP includes tools that let you act on all nodes in an HCP system with a single command. These Python tools, called system service tools, use SSH to log into all the target nodes at the same time and perform the actions you request:
cluster_ssh does any of these:

Issues one or more Linux commands on each target node Reboots or shuts down each target node Retrieves internal log messages from each target node Runs a user-supplied script on each target node
cluster_get retrieves files from the HCP nodes.
1115
cluster_put stores files on the HCP nodes.

Running the tools To use the system service tools: 1. Using the service SSH key, use SSH to log into any HCP node in the system as service. For instructions on doing this, see Appendix D, Setting up SSH key authentication. Note: Depending on the action you want to perform, you may also need the root SSH key in your keychain. 2. Enter the command to run the tool you want. Target HCP system By default, the system service tools act on the HCP system that includes the node from which you run them. However, you can tell the tool youre using to act on a different HCP system. To identify the system on which the tool should act:
You can use the -n or --node parameter in the tool command to specify
the front-end IP address of a node in the target system.
You can use the -c or --config-file parameter in the tool command to

specify an HCP configuration file located on the node you logged into. This file must identify the system that includes that node. For information on HCP configuration files, see Using a modified HCP configuration file on page 3-32.
You can set the TARGET_CLUSTER environment variable to the frontend IP address of a node in the target system. The tool looks first for an -n, --node, -c, or --config-file parameter. If none is present, it looks for the TARGET_CLUSTER variable. If this variable is not set, it uses the current system. Target nodes By default, the system service tools act on all nodes in the target HCP system. However, you can limit the requested operation to only one type of node:
You can use the -a parameter in the tool command to limit the
operation to only storage nodes.
1116
You can use the -s parameter in the tool command to limit the
operation to only search nodes. If you specify both the -a and -s parameters in a single command, the tool returns an error. System service tool output Each system service tool writes all output both to the console and to log files on the node from which you run the tool. The log files, one for each target node, are located in the /opt/arc/tools/cluster/logs directory. The name of each log file is node-ip-address.output (for example, 192.168.210.16.output). The system service tools all use the same set of log files. As each tool runs, it appends its output to the text that already exists in the files.
cluster_ssh
The cluster_ssh tool does any of these:
Issues one or more Linux commands to be executed on each target

node
Reboots or shuts down each target node Retrieves internal log messages from the target nodes
Syntax for this tool is:
cluster_ssh.py (-h|--help) |--version |([(-c|--config-file) config-file-spec] [(-n|--node) node-ip-address] [-a|-s] [(-u|--user) username] [-v|--verbose] (((-l|--level) message-severity-level) |(-r|--reboot) |(-H|--halt) |((-i|--inputScript) script-file-spec) |"command-string"))
1117
cluster_ssh parameter descriptions The table below describes the parameters for the cluster_ssh command.
Parameter -h|--help --version (-c|--config-file) config-file-spec Description
Displays and describes the syntax for the cluster_ssh command; then exits. Displays the version number of the currently installed cluster_ssh tool; then exits. Specifies an HCP configuration file that identifies the system on which cluster_ssh should act. The configuration file you specify must be located on the node you logged into and must identify the system that includes that node. For information on using this parameter, see Target HCP system on page 11-16.
(-n|--node) node-ip-address
Specifies a node that identifies the HCP system on which cluster_ssh should act. node-ip-address is the front-end IP address of the node. For information on using this parameter, see Target HCP system on page 11-16.
-a -s (-u|--user) username -v|--verbose
Tells cluster_ssh to act only on storage nodes. Tells cluster_ssh to act only on search nodes. Tells cluster_ssh to log into each node in the target system with the specified username. Reports cluster_ssh activity in greater detail. If an error occurs while cluster_ssh is running, rerunning the same command with this parameter can help identify the problem. Retrieves all the HCP internal log messages with the specified severity level or a more severe level from the log files in the /var/ris/logs directory on each target node. Valid values for message-severity-level are: FINEST FINER FINE INFO WARNING SEVERE These values are case sensitive. This parameter is mutually exclusive with the -r, --reboot, -H, --halt, -i, --inputScript, and command-string parameters. For information on the HCP internal logs, see Managing the HCP internal logs on page 11-5.
(-l|--level) messageseverity-level
1118

(Continued)
Parameter -r|--reboot
Description
Tells cluster_ssh to shut down HCP on each target node and then reboot the node. This parameter is mutually exclusive with the -l, --level, -H, --halt, i, --inputScript, and command-string parameters.
-H|--halt
Tells cluster_ssh to shut down HCP on each target node and then power off the node. This parameter is mutually exclusive with the -l, --level, -r, --reboot, -i, --inputScript, and command-string parameters. Note: If any node does not power off automatically, wait 30 minutes before powering it off manually. If the node is connected to a console, the console displays System halted when you can safely power off the node.
(-i|--inputScript) script-file-spec
Specifies a user-supplied script for cluster_ssh to execute on each target node. script-file-spec is the name of the script file with or without a directory path. If included, the path can be absolute or relative to the current directory. After running the specified script, cluster_ssh removes the script from the node. This parameter is mutually exclusive with the -l, --level, -r, --reboot, -H, --halt, and command-string parameters.
1119

(Continued)
Parameter "command-string"
Description
Specifies one or more commands for cluster_ssh to execute on each target node. cluster_ssh executes multiple commands on any given node in the order in which you specify them. Use a semicolon (;) or a double ampersand (&&) to separate each command from the next: The semicolon means proceed with the next command even if the current command fails. The double ampersand means do not proceed if the current command fails.
The double quotation marks (") are required when you specify more than one command or when you specify a single command with one or more options. You can omit the quotation marks around a single command with no options. This parameter is mutually exclusive with the -l, --level, -r, --reboot, -H, --halt, -i, and --inputScript parameters. Note: To reboot nodes, use the -r or --reboot parameter instead of a reboot command. With a reboot command, the current node may shut down before it can communicate the request to the other target nodes. Similarly, use the -H or --halt parameter instead of a shutdown command.
1120
cluster_ssh examples Here are three sample cluster_ssh commands:
The cluster_ssh command below echoes the ping command on each

HCP node. You can use an echo command like this to check whether the nodes are online.
cluster_ssh.py "echo ping"
The console output from this command looks like this:

NODES: Storage: 10.4.1.15 10.4.1.16 Search: 10.4.1.17 COMMAND: echo ping STDOUT: ================== || 10.4.1.15: ================== ping ================== || 10.4.1.16: ================== ping ================== || 10.4.1.17: ================== ping ==================== 3 Success 0 Error ==================== Command results saved at /var/opt/arc/tools/cluster/logs
1121
The cluster_ssh command below changes each HCP node to runlevel

three and then displays the runlevel. At runlevel three, the nodes are healthy but are not running the HCP software. The command tells cluster_ssh to log into each node with the username root in order to have the permissions required for the specified commands.
cluster_ssh.py -u root "init 3;runlevel"

NODES: Storage: 10.4.1.15 10.4.1.16 Search: 10.4.1.17 COMMAND: init 3;runlevel STDOUT: ================== || 10.4.1.15: ================== 43 ================== || 10.4.1.16: ================== 43 ================== || 10.4.1.17: ================== 43 ==================== 3 Success 0 Error ==================== Command results saved at /var/opt/arc/tools/cluster/logs
1122
The cluster_ssh command below retrieves all the messages with a

severity level of SEVERE from the log files in the /var/ris/logs directory on each HCP node. The messages appear in the cluster_ssh output.
cluster_ssh.py --level SEVERE

NODES: Storage: 10.4.1.15 10.4.1.16 Search: 10.4.1.17 COMMAND: grep SEVERE /var/ris/logs/*; exit 0 STDOUT: ================== || 10.4.1.15: ================== /var/ris/logs/R01.15.storman1-out:10/10 11:28:11 storman-1 20429 15 (SEVERE): PANIC AT register.c:93:restart_admin Unable to connect to the admin after 2 tries /var/ris/logs/R01.15.storman2-out:10/10 11:28:13 storman-2 20498 15 (SEVERE): PANIC AT register.c:93:restart_admin Unable to connect to the admin after 2 tries . . . /var/ris/logs/sysmon.15.log:10/10 11:17:59 sysmon 3418 15 (SEVERE): Admin handler returned failure: -1 Broken pipe /var/ris/logs/sysmon.15.log:10/10 11:18:04 sysmon 3418 15 (SEVERE): PANIC AT register.c:93:restart_admin Unable to connect to the admin after 0 tries ================== || 10.4.1.16: ================== /var/ris/logs/R01.16.storman1-out:10/10 11:28:11 storman-1 20713 16 (SEVERE): PANIC AT register.c:93:restart_admin Unable to connect to the admin after 2 tries /var/ris/logs/R01.16.storman2-out:10/10 11:28:11 storman-2 20749 16 (SEVERE): PANIC AT register.c:93:restart_admin Unable to connect to the admin after 2 tries . . . /var/ris/logs/sysmon.16.log:10/10 11:17:59 sysmon 3418 16 (SEVERE): Admin handler returned failure: -1 Broken pipe /var/ris/logs/sysmon.16.log:10/10 11:17:59 sysmon 3418 16 (SEVERE): PANIC AT register.c:93:restart_admin Unable to connect to the admin after 0 tries
1123
Using the system service tools ================== || 10.4.1.17: ================== /var/ris/logs/R01.17.storman1-out:10/10 11:28:11 storman-1 20713 17 (SEVERE): PANIC AT register.c:93:restart_admin Unable to connect to the admin after 2 tries /var/ris/logs/R01.17.storman2-out:10/10 11:28:12 storman-2 20749 17 (SEVERE): PANIC AT register.c:93:restart_admin Unable to connect to the admin after 2 tries . . . /var/ris/logs/sysmon.17.log:10/10 11:18:01 sysmon 3418 17 (SEVERE): Admin handler returned failure: -1 Broken pipe /var/ris/logs/sysmon.17.log:10/10 11:18:03 sysmon 3418 17 (SEVERE): PANIC AT register.c:93:restart_admin Unable to connect to the admin after 0 tries ==================== 3 Success 0 Error ==================== Command results saved at /var/opt/arc/tools/cluster/logs
cluster_get
The cluster_get tool retrieves one or more files from each target node and stores them on the current node. By default, the files are stored in subdirectories, one per node, of a specified directory, but you can choose to have them stored directly in that directory. The name of each subdirectory is the IP address of the applicable node. If the cluster_get command uses -n or --node parameter to identify the target system, this is the front-end IP address. Otherwise, its the back-end IP address. Syntax for the cluster_get tool is:
cluster_get.py (-h|--help) |--version |([(-c|--config-file) config-file-spec] [(-n|--node) node-ip-address] [-a|-s] [(-u|--user) username] [-d|--disable-unique-ify] [-v|--verbose] ((-l|--logs) |(-i|--image) |((source-file-spec|source-path) destination-path))
1124
cluster_get parameter descriptions The table below describes the parameters for the cluster_get command.
Displays and describes the syntax for the cluster_get command; then exits. Displays the version number of the currently installed cluster_get tool; then exits. Specifies an HCP configuration file that identifies the system on which cluster_get should act. The configuration file you specify must be located on the node you logged into and must identify the system that includes that node. For information on using this parameter, see Target HCP system on page 11-16.
Specifies a node that identifies the HCP system on which cluster_get should act. node-ip-address is the front-end IP address of the node. For information on using this parameter, see Target HCP system on page 11-16.
-a -s (-u|--user) username -d|--disable-uniqueify
Tells cluster_get to retrieve the requested files only from storage nodes. Tells cluster_get to retrieve the requested files only from only on search nodes. Tells cluster_get to log into each target node with the specified username. Puts all the returned files directly in the directory identified by the destination-path parameter. With this parameter, files with the same name that come from different nodes overwrite each other Without this parameter, cluster_get puts each returned file in a directory named with the IP address of the node from which it was retrieved. These directories are located in the directory specified by the destination-path parameter. This parameter is not valid with the -l, --logs, -i, and --image parameters.
-v|--verbose
Reports cluster_get activity in greater detail. If an error occurs while cluster_get is running, rerunning the same command with this parameter can help identify the problem.
1125

(Continued)
Parameter -l|--logs
Description
Retrieves and compresses the HCP internal log files from the /var/ris/logs directory on each target node. The log files from each node are compressed into a single file named node-front-end-ip-address_logs.tar.gz, which is then stored in a node-specific subdirectory of the current directory. The name of each subdirectory is the IP address of the applicable node. If the cluster_get command uses -n or --node parameter to identify the target system, this is the front-end IP address. Otherwise, its the backend IP address. If you use this parameter, do not include the -i, --image, source-filespec, source-path, or destination-path parameter in the cluster_get command. For information on the HCP internal log files, see Managing the HCP internal logs on page 11-5.
-i|--image
Retrieves and compresses the HCP internal log files from the /var/ris/logs directory on each target node, along with other node information thats useful for identifying problems. The log files and other information from each node are compressed into a single file named node-front-end-ipaddress_image.tgz, which is then stored in a node-specific subdirectory of the current directory. The name of each subdirectory is the IP address of the applicable node. If the cluster_get command uses -n or --node parameter to identify the target system, this is the front-end IP address. Otherwise, its the backend IP address. If you use this parameter, do not include the -l, --logs, source-filespec, source-path, or destination-path parameter in the cluster_get command. For information on the HCP internal log files, see Managing the HCP internal logs on page 11-5. Important: This operation is CPU intensive. It also makes it more difficult for support personnel to review the logs in real time. Do not do this unless explicitly told to do so by HCP support personnel.
source-file-spec
Specifies the file cluster_get should retrieve from all the target nodes. The file specification must include the absolute path to the file.
1126

(Continued)
Parameter source-path
Description
Specifies a directory from which cluster_get should retrieve all files, including the files in any directories below that directory (that is, all subdirectories, recursively). cluster_get retrieves the files from all the target nodes.
source-path must be an absolute path. destination-path

Specifies the directory in which cluster_get puts the returned files. cluster_get puts returned files only on the current node. If the specified directory doesnt already exist, cluster_get creates it.
destination-path can be an absolute path or the path relative to the

current directory. This parameter is not valid with the -l, --logs, -i, and --image parameters.
cluster_get examples Here are three sample cluster_get commands:
The cluster_get command below retrieves the build.version file from each
HCP node and stores it in a directory named with the back-end IP address of that node. cluster_get creates these directories in the current directory. You can use this command to verify that the same version of HCP is installed on all nodes in the system.
cluster_get.py /opt/arc/build.version ./

NODES: Storage: 10.4.1.15 10.4.1.16 Search: 10.4.1.17 COMMAND: scp [remoteNode]:/opt/arc/build.version to Destination ./ Retrieving files from [nodes]:/opt/arc/build.version to .//[NODEIP]/ ==================== 3 Success 0 Error ====================
1127
The cluster_get command below retrieves a compressed copy of the

contents of /var/ris/logs from each HCP node. Each copy is stored in a subdirectory of the current directory. The subdirectory is named with the back-end IP address of the applicable node. The command tells cluster_get to log into each node with the username root in order to have the permissions required for the requested action.
cluster_get.py -u root -l

NODES: Storage: 10.4.1.15 10.4.1.16 Search: 10.4.1.17 COMMAND: mkdir /RIS/archive1/cluster_tool_tmp; tar cjvf /RIS/archive1/cluster_ tool_tmp/\${RIS_FRONTEND_IP}_log.tar.bz2 -C /var/ris/ logs > /dev/null ==================== 3 Success 0 Error ==================== Retrieving files from [nodes]:/RIS/archive1/cluster_tool_tmp/\${RIS_ FRONTEND_IP}_log.tar.bz2 to .//[NODEIP]/ ==================== 3 Success 0 Error ==================== COMMAND: rm -rf /RIS/archive1/cluster_tool_tmp ==================== 3 Success 0 Error ====================
1128
The cluster_get command below retrieves the /opt/arc/build.version file

from each node in the HCP system identified by the front-end IP address 192.168.103.10. It stores the retrieved files directly in the /opt/arc/tools/cluster/replica directory. Because the retrieved files all have the same name, the result is only one file.
cluster_get.py -n 192.168.103.11 -d /opt/arc/build.version replica

NODES: Storage: 192.168.103.10 192.168.103.11 Search: 192.168.103.12 COMMAND: scp [remoteNode]:/opt/arc/build.version to Destination replica Retrieving files from [nodes]:/opt/arc/build.version to replica/ ==================== 3 Success 0 Error ====================
cluster_put
The cluster_put tool stores one or more files on each target node. Syntax for this tool is:
cluster_put.py (-h|--help) |--version |([(-c|--config-file) config-file-spec] [(-n|--node) node-ip-address] [-a|-s] [(-u|--user) username] [-v|--verbose] (source-file-spec|source-path)) [destination-path])
1129
cluster_put parameter descriptions The table below describes the parameters for the cluster_put command.
Displays and describes the syntax for the cluster_put command; then exits. Displays the version number of the currently installed cluster_put tool; then exits. Specifies an HCP configuration file that identifies the system on which cluster_put should act. The configuration file you specify must be located on the node you logged into and must identify the system that includes that node. For information on using this parameter, see Target HCP system on page 11-16.
Specifies a node that identifies the HCP system on which cluster_put should act. node-ip-address is the front-end IP address of the node. For information on using this parameter, see Target HCP system on page 11-16.
-a -s (-u|--user) username -v|--verbose
Tells cluster_put to store the specified files only on storage nodes. Tells cluster_put to store the specified files only on search nodes. Tells cluster_put to log into each target node with the specified username. Reports cluster_put activity in greater detail. If an error occurs while cluster_get is running, rerunning the same command with this parameter can help identify the problem. Specifies the file cluster_put should store. cluster_put takes this file only from the node on which you issue the command. The file specification can include an absolute path or a path relative to the current directory.
source-file-spec
source-path
Specifies a directory containing the files cluster_put should store, including the files in any directories below that directory (that is, all subdirectories, recursively). cluster_put takes these files only from the node on which you issue the command.
source-path can be an absolute path or a path relative to the current

directory.
1130

(Continued)
Parameter destination-path
Description
Specifies the directory in which cluster_put stores the files on each target node. The specified directory must already exist.
destination-path must be an absolute path.

If you omit this parameter, cluster_put uses the source path as the destination path. If the specified source path is a relative path, cluster_ put creates an absolute destination path by appending the specified source path to the path to the current directory.
cluster_put examples Here are two sample cluster_put commands:
The cluster_put command below copies the system_status_info file from

the /opt/arc/tools/cluster/temp directory on the current node to the existing /opt/arc/tools/cluster/temp directory on each node in the HCP system.
cluster_put.py temp/system_status_info /opt/arc/tools/cluster/temp

NODES: Storage: 10.4.1.15 10.4.1.16 Search: 10.4.1.17 Copying local file temp/system_status_info to [NODES]:/opt/arc/tools/cluster/ temp ==================== 3 Success 0 Error ====================
Tip: To create the /opt/arc/tools/cluster/temp directory on all the nodes in the system, you could use the cluster_ssh tool.
1131
Viewing the system status pages
The cluster_put command below copies the temp/cluster.cfg file from the
current node to all the nodes in the HCP system, overwriting the cluster.cfg file in the /opt/arc/config directory on each. The command tells cluster_put to log into each node with the username root in order to have the permissions required for the requested action. This command is useful for propagating configuration changes to all the nodes in the system.
cluster_put.py -u root temp/cluster.cfg /opt/arc/config
Important: Do not modify the cluster.cfg file unless explicitly told to do so by HCP support personnel. The console output from this command looks like this:
NODES: Storage: 10.4.1.15 10.4.1.16 Search: 10.4.1.17 Copying local file temp/cluster.cfg to [NODES]:/opt/arc/config ==================== 3 Success 0 Error ====================
Viewing the system status pages

The HCP System Management Console has four pages that provide information on the current state of the system. These pages present information in XML format. This format lets you easily spot unexpected conditions. The four system status pages are:
getAdminView.action Provides a snapshot of the HCP nodes.

The information on this page is presented by node. For each node, the page shows the status of elements such as services, namespace access protocols, storage, and file systems.
1132
Ejecting the CD tray from a node
getUUIDs.action Provides information on the current storage

configuration. This page lists the logical volumes that are currently accessible to the HCP system. For each volume, it shows the unique identifier (UUID) and the name of the storage manager responsible for it.
getProtectionSets.action Lists the nodes in each of the current

protection sets.
getMM.action Shows the current status of the HCP-internal regions.

To display a system status page: 1. Log into the System Management Console with the monitor, administrator, or service role. 2. In the browser address field, enter the URL for the page you want. The format for this URL is:
https://admin.hcp-name.domain-name:8000/cluster/page-name
For example:
https://admin.hcp-ma.example.com:8000/cluster/getAdminView.action
Note: To view the system status pages, you must be in an active HCP System Management Console session. If your session times out due to inactivity, you cannot display these pages.

The room in which the HCP nodes are located typically contains many servers. Locating a particular node by its IP address can be difficult. Locating a node with an ejected CD tray is easier. You can eject the CD tray from a node through the HCP System Management Console. When you do this, the node beeps its node number in Morse code once.
1133
To eject the CD tray from an HCP node: 1. In the top-level menu in the HCP System Management Console, click on Nodes. 2. In the list of nodes, click on the number of the node you want. 3. At the bottom of the node details page, click on the Eject CD button.
1134
A
Forms
This appendix contains forms that you can use to facilitate some of the procedures presented in this book:
Appliance OS Installation Information lets you record the information

you need for an Appliance OS installation.
HCP Software Installation Information lets you record the information

you need for an HCP software installation.
Encryption Key lets you record the encryption key displayed during HCP
installation processing.
HCP Node Addition Information lets you record the information you
need for a node addition.
Forms Installing and Maintaining an HCP System
A1
Appliance OS Installation Information
Appliance OS Installation Information

Use this form to record the information for an Appliance OS installation. Front-end IP address/back-end IP address of each node: ____________
__________________________________________________________ __________________________________________________________ __________________________________________________________ __________________________________________________________ __________________________________________________________ __________________________________________________________ __________________________________________________________ __________________________________________________________ __________________________________________________________ __________________________________________________________ __________________________________________________________ __________________________________________________________ __________________________________________________________ __________________________________________________________ __________________________________________________________ __________________________________________________________ __________________________________________________________ __________________________________________________________ __________________________________________________________ __________________________________________________________ Front-end subnet mask: ______________________________________ Front-end network gateway IP address: _________________________
A2
HCP Software Installation Information

Use this form to record the information for an HCP software installation. Distributor key Distributor key: Networking Front-end gateway IP address: Multicast IP address: DNS Enable DNS: Yes No ________________________________ ____________________________________________
________________________________________
DNS name of the HCP system: _________________________________ If DNS enabled, IP addresses of corporate DNS servers: ____________
__________________________________________________________ __________________________________________________________ Time Time source: Internal External ________________________
If external, IP addresses of time servers:
__________________________________________________________ Time zone: Internals Storage configuration: If external, shared storage: If external, WMS-100 or AMS-200: If external, blade servers If blade servers, chassis IP addresses: Internal External Yes Yes Yes No No No ________________________________________________
__________________________
__________________________________________________________ Serial number: _____________________________________________ Enable replication: Yes No
Customer support contact information: __________________________ __________________________________________________________
(Continued on next page)
A3
Security Enable encryption: Storage nodes Back-end IP addresses of storage nodes (list, range, or file name): ____ __________________________________________________________ __________________________________________________________ __________________________________________________________ __________________________________________________________ __________________________________________________________ __________________________________________________________ __________________________________________________________ __________________________________________________________ __________________________________________________________ __________________________________________________________ __________________________________________________________ __________________________________________________________ __________________________________________________________ __________________________________________________________ __________________________________________________________ __________________________________________________________ __________________________________________________________ __________________________________________________________ __________________________________________________________ __________________________________________________________ Search nodes Back-end IP addresses of search nodes (list, range, or file name): _____ __________________________________________________________ __________________________________________________________ __________________________________________________________ __________________________________________________________ Yes No
A4
Encryption Key
Encryption Key
Use this form to record the key displayed during installation processing for an HCP system with encryption enabled. HCP system serial number: Encryption key: ___________________________________
____________________________________________
Caution: KEEP THIS KEY IN A SECURE LOCATION. Loss of this key will most likely result in unrecoverable data in the case of catastrophic system failure.
A5
HCP Node Addition Information
HCP Node Addition Information

Use this form to record the information for adding nodes to an HCP system. Physical location of the highest-numbered existing storage node in the system: __________________________________________________
install user password for logging into the HCP nodes: _______________ Version number of the Appliance Operating System currently installed on the existing HCP nodes: ___________________________________
Version number of the HCP software currently installed on the existing HCP nodes: ________________________________________________ Front-end IP address/back-end IP address of each new storage node: __________________________________________________________ __________________________________________________________ __________________________________________________________ __________________________________________________________ __________________________________________________________ __________________________________________________________ __________________________________________________________ __________________________________________________________ Front-end IP address/back-end IP address of each new search node: __________________________________________________________ __________________________________________________________ __________________________________________________________ __________________________________________________________ Front-end subnet mask already set for the existing nodes in the system: __________________________________________________________ Front-end network gateway IP address already set for the existing nodes in the system: _______________________________________________
For SAIN systems that use CB 320 servers, chassis IP address of each new server: ___________________________________________________
A6
B
HCP Setup wizard menus
The program you use to install, upgrade, or add nodes to an HCP system is called HCP Setup. It uses a menu-based wizard to step you through the procedure you choose to perform. This appendix shows each of the wizard menus. It presents them in the order in which they occur when you follow the default path through the wizard for the applicable procedure. For general instructions on using the HCP Setup wizard, see HCP Setup program on page 3-7.
HCP Setup wizard menus Installing and Maintaining an HCP System
B1
Installation menus
Installation menus
Option 2 on the HCP 4.1 Configuration menu displays the wizard New Install menu.
When you select option 2 on the New Install menu, the wizard presents the HCP Networking Options menu.
When you select option 3 on the New Install menu, the wizard presents the HCP DNS Options menu.
B2
Installation menus
When you select option 4 on the New Install menu, the wizard presents the HCP Time Options menu.
When you select option 5 on the New Install menu, the wizard presents the Internal Configuration Settings menu.
When you select option 6 on the New Install menu, the wizard presents the HCP Security Settings menu.
B3
Upgrade menu
When you select option 7 on the New Install menu, the wizard presents the HCP Nodes menu.
Upgrade menu
Option 3 on the HCP 4.1 Configuration menu displays the Upgrade menu.
Node addition menu

Option 4 on the HCP 4.1 Configuration menu displays the Membership Update menu.
B4
Storage addition menu

Option v on the HCP 4.1 Configuration menu displays the Add Logical Volumes menu.
B5
B6
C
HCP configuration file
The HCP Setup program you use to install the HCP software uses configuration information stored in a file named cluster.cfg. This file is located in the /home/install/arc-deploy directory on the node from which youre installing. Caution: Do not modify the cluster.cfg file in the /home/install/arc-deploy directory without explicit instructions from HCP support personnel. Doing so can cause the installation, upgrade, or node addition to fail or the HCP system not to run correctly after an installation, upgrade, or node addition. Once an HCP system is installed, you can view its cluster.cfg file in the /opt/ arc/config directory on each node. HCP Setup uses this file for system upgrades and node additions. Caution: Do not modify the cluster.cfg file in the /opt/arc/config directory. Doing so can cause upgrades and node additions to fail or the HCP system not to run correctly after an upgrade or node addition. This appendix contains a table of the parameters in the cluster.cfg file. The parameters are broken out by the sections of the file in which they occur. Within each section, they are presented in alphabetical order. For each parameter, the table provides a description and an example and indicates whether the HCP Setup wizard prompts for a value for it. For information on modifying the cluster.cfg file in the /home/install/arc-deploy directory, see Using a modified HCP configuration file on page 3-32.
HCP configuration file Installing and Maintaining an HCP System
C1
Parameter
install config_key
Description
Prompt
Version number of the configuration file. Example: config_key = 4.1.0 Caution! Do not change the value set by the HCP Setup wizard. Doing so will cause the installation or upgrade to fail.
No
type
Type of installation. Example: type = install-fresh Caution! Do not change the value set by the HCP Setup wizard. Doing so will cause the installation or upgrade to fail.
No
arc_nodes In /home/install/arc-deploy/ Back-end IP address of a storage node in the HCP system. The configuration file includes one line for each storage node. cluster.cfg: nodenode-number In the cluster.cfg file in the /home/install/arc-deploy directory, the node numbers in the parameter keywords start from 0 In /opt/arc/config/ (zero). cluster.cfg: node_node-IP-address In the cluster.cfg file in the /opt/arc/config directory, the IP address in each parameter keyword is the same IP address as the parameter value. Examples: In /home/install/arc-deploy/cluster.cfg: Yes
node0 node1 node2 node3
= = = =
172.21.210.15 172.21.210.16 172.21.210.17 172.21.210.18
In /opt/arc/config/cluster.cfg:
node_172.21.210.15 node_172.21.210.16 node_172.21.210.17 node_172.21.210.18
= = = =
172.21.210.15 172.21.210.16 172.21.210.17 172.21.210.18
C2
(Continued)
Parameter
search_nodes
Description
Prompt
In /home/install/arc-deploy/ Back-end IP address of a search node in the HCP system. The configuration file includes one line for each search node. cluster.cfg: nodenode-number In /opt/arc/config/ cluster.cfg: node_node-IP-address In the cluster.cfg file in the /home/install/arc-deploy directory, the node numbers in the parameter keywords start from 0 (zero). In the cluster.cfg file in the /opt/arc/config directory, the IP address in each parameter keyword is the same IP address as the parameter value. Examples: In /home/install/arc-deploy/cluster.cfg:
Yes
node0 = 172.21.210.19 node1 = 172.21.210.20

In /opt/arc/config/cluster.cfg:
node_172.21.210.19 = 172.21.210.19 node_172.21.210.20 = 172.21.210.20

settings backend_mtu Maximum packet size, in bytes, for data sent on the back-end network. Valid values are positive integers. The default is 1500. Example: backend_mtu = 1500 blade_servers Indication of whether the HCP system uses CB 320 servers. Valid values are True (the system uses CB 320 servers) and False (the system does not use CB 320 servers). The default is False. Example: blade_servers = False capablemsgversion Used for upgrades only, highest version of the internal messaging protocol the HCP system will be able to handle. The HCP Setup wizard automatically supplies this value. Example: capablemsgversion = 410 Caution! Do not change the value set by the HCP Setup wizard. Doing so will cause the installation or upgrade to fail. No Yes No
C3
(Continued)
Parameter
chassis_ip_csl
Description
Chassis IP addresses of the CB 320 servers used by the HCP system, if any. Use commas to separate multiple IP addresses. Spaces are not allowed. This parameter applies only when the blade_servers parameter is set to True. Example: chassis_ip_csl = 192.168.73.16
Prompt
Yes
clname
DNS name of the HCP system, specified as hcp-name. domain-name. Example: clname = hcp-ma.example.com
Yes
contact_info
Contact information for the HCP support center. The default depends on the distributor key. Example: contact_info = 'United States: (800) 446-0744.
Yes
Outside the United States: (858) 547-4526'
current_time
Time on a 24-hour clock to which the HCP nodes initially set their system clocks, in this format:
Yes
hhmmssMMDDYYYY
This parameter applies only when the timeserver parameter is set to internal. Example: current_time = 13450003222011 currentmsgversion Used for upgrades only, version of the internal messaging protocol the existing HCP system uses. The HCP Setup wizard automatically supplies this value. Example: currentmsgversion = 410 disk_flush Option to have HCP flush the disk cache after every write operation. Valid values are True (flush the cache) and False (do not flush the cache). The default is False. Example: disk_flush = False distributor_key Name that determines the SSH keys required to use SSH to log into HCP nodes when the lockdown parameter is set to True. These SSH keys are initially provided by HDS and are unique for each authorized distributor of HCP. Yes No No
C4
(Continued)
Parameter
dns_enabled
Description
Option to use DNS with the HCP system. Valid values are True (use DNS) and False (do not use DNS). The default is true. If this parameter is set to True, the dns_servers parameter must specify the IP addresses of one or more DNS servers. Example: dns_enabled = True
Prompt
Yes
dns_servers
IP addresses of one or more corporate DNS servers. Use commas to separate multiple IP addresses. The default is the single IP address 192.168.100.10. For the HCP system to use DNS, you need to specify True for the dns_enabled parameter. Example: dns_servers = 192.168.107.10, 192.168.107.34
Yes
enable_replication
Option to allow the HCP system to replicate to another system or to be the target of replication. Valid values are True (allow replication) and False (disallow replication). The default is False. Example: enable_replication = True Note: Allowing replication doesnt cause the system to replicate. Replication starts with the creation of a replication link.
Yes
encryption
Option to encrypt data and metadata in the repository. Valid values are True (encrypt) and False (do not encrypt). The default is False. If this parameter is set to True, the encryption_cipher, encryption_ivmode, and encryption_key_length parameters must specify values. Example: encryption = True
Yes
encryption_cipher
Method used to encrypt the data and metadata in the repository. Valid values are the names of encryption methods. The default is AES. This parameter applies only when the encryption parameter is set to True. Example: encryption_cipher = AES
No
C5
(Continued)
Parameter
encryption_ivmode
Description
Initialization vector for the encryption cipher. This parameter applies only when the encryption parameter is set to True. Example: encryption_ivmode = essiv:sha256
Prompt
No
encryption_key_length
Length of the encryption key. Valid values are positive integers. This parameter applies only when the encryption parameter is set to True. Example: encryption_key_length = 256
No
filesystem
File system to use for storing objects. The only valid value is ext3. Example: filesystem = ext3
No
frontend_mtu
Maximum packet size, in bytes, for data sent on the front-end network. Valid values are positive integers. The default is 1500. Example: frontend_mtu = 1500
No
gateway
IP address of the front-end network gateway. The default is the gateway IP address specified during the Appliance OS installation. Example: gateway = 192.168.210.1
Yes
hostname_prefix
Prefix used in the names of nodes in the HCP system. This can be blank or a text string followed by a hyphen. Valid characters are lowercase letters, numbers, and hyphens (-). The complete hostname is formed by combining the prefix with the fourth octet of the back-end IP address for the node. The default for this parameter is hcp-node-. Example: hostname_prefix = hcp-node-
Yes
kerberos_ssh
Option to enable Kerberized SSH login to the HCP nodes. Kerberized SSH can be used for certain rare support situations. Valid values are True (enable Kerberized SSH login) and False (disable Kerberized SSH login). The default is False. Example: kerberos_ssh = False Note: Additional setup is required for Kerberized SSH to work. For information on this setup, see http:// www.visolve.com/security/ssh_kerberos.php.
No
C6
(Continued)
Parameter
lockdown
Description
Option to disable the ability to use SSH to log into HCP nodes without the SSH key supplied by the HCP distributor (applies after HCP Setup has finished processing). Valid values are True (disallow SSH without supplied key) and False (allow SSH without supplied key). The default is True. If this parameter is set to True, the distributor_key parameter must specify the name for the SSH keys provided by HDS. Example: lockdown = True Note: For customer installations, this parameter must be set to True.
Prompt
Yes
max_simul_mkfs
Maximum number of logical volumes HCP Setup will format at the same time on each node. Valid values are integers. The default is 4. The recommended maximum is 6. Example: max_simul_disks = 4
No
max_simul_nodes
Maximum number of nodes on which HCP Setup will simultaneously install HCP. Valid values are integers. The default is 10. However, if the HCP system is sharing external storage with other applications, this parameter is set to 2. Example: max_simul_nodes = 10
Prompt for shared storage
mcast
IP address of the multicast network for the HCP system. The default is 238.177.1.1. If two systems share back-end switches, each system must have a different multicast network and be on a different backend subnet. Example: mcast = 238.144.60.2
Yes
mdpl
Number of copies of the metadata for each object HCP must maintain in the repository. Valid values are 2, 3, and 4. The default is 2. Example: mdpl = 2 Caution! Do not set mdpl to 1 (one). Doing so can result in many irreparable objects if a node fails. Also, values higher than 2 have not been tested.
No
mm_filesystem
File system to use for the internal drives in CR 220 servers in HCP SAIN systems. The only valid value is ext3. Example: mm_filesystem = ext3
No
C7
(Continued)
Parameter
old_software_version
Description
Used for upgrades only, currently installed version of the HCP software. The HCP Setup wizard automatically supplies this value. Example: old_software_version = 4.0.1.169 Caution! Do not change the value set by the HCP Setup wizard. Doing so will cause the upgrade to fail.
Prompt
No
policy_enabled
Option to have services initially enabled on the HCP system. Valid values are True (services are enabled) and False (services are disabled). The default is True. This parameter applies to the capacity balancing, compression, content verification, disposition, duplicate elimination, garbage collection, protection, and scavenging services. Example: policy_enabled = True
No
regionmapsize
Number of regions to create. Valid values are integers greater than or equal to four times the number of storage nodes in the system. The default for this parameter is based on the initial number of storage nodes in the system. For a small system (for example, four nodes), the default is 32. Example: regionmapsize = 44
No
serial_number
Serial number for the HCP system. Example: serial_number = '12345'
Yes
single_node
Indication of whether the HCP system has one or more storage nodes. Valid values are True (the system has a single storage node) and False (the system has multiple storage nodes). The default is False. Example: single_node = False Note: Single-node systems are intended for testing purposes only. For customer installations, this parameter must be set to False. If this parameter is set to True and the cluster.cfg file lists multiple storage nodes, the installation or upgrade will fail.
No
C8
(Continued)
Parameter
software_version
Description
Version of the HCP software being installed. Example: software_version = 4.1.0.94 Caution! Do not change the value set by the HCP Setup wizard. Doing so will cause the installation or upgrade to fail.
Prompt
No
storage_type
Type of storage used by the HCP system. Valid values are internal (non-SAN-attached storage) and external (SANattached storage). The default is Not Set. Example: storage_type = external
Yes
timeserver
Source the HCP nodes use to adjust their system clocks. Valid values are the IP addresses of one or more servers outside the HCP system and internal, which causes the nodes to synchronize their system clocks to each other. Use commas to separate multiple IP addresses. The default is internal. Example: timeserver = internal
Yes
timezone
Time zone in which the HCP system is installed. Select the time zone from the list presented by the HCP Setup wizard. The default depends on the distributor key. Example: timezone = America/Los_Angeles
Yes
zero_copy_failover
Option to enable zero-copy failover. Valid values are true (enable zero-copy failover) and false (disable zero-copy failover). The default depends on the distributor key. Example: zero_copy_failover = true
Yes
C9
C10
D
Setting up SSH key authentication
To perform service procedures on HCP nodes, you need to use SSH to log into the applicable node. To use SSH, you need an SSH key. HCP has three SSH keys:
One for logging into the nodes before the HCP core software is installed
(the installation key)
One for logging into the HCP nodes as the service user after theyre
locked down (the service key)
One for logging into the HCP nodes as the root user after theyre locked
down (the root key) You get these keys, along with their passphrases, from the HCP distributor. The procedure for getting these keys depends on the distributor. To use SSH keys for HCP procedures, you first add the applicable keys to your keychain on your computer. Then you forward them to the node youre logging into. Each time you log into a node, you need to forward the applicable SSH keys. The procedure for adding and forwarding SSH keys differs for Windows and Linux. This appendix contains instructions for both. Caution: Never give the any of the HCP SSH keys to customers. Doing so puts the integrity of the stored data at risk.
Setting up SSH key authentication Installing and Maintaining an HCP System
D1
Adding and forwarding an SSH key in Windows
Adding and forwarding an SSH key in Windows

The Windows procedure for adding and forwarding SSH keys requires a PuTTY client with the Pageant SSH authorization agent. You can download the PuTTY installation file for Windows (putty-version-installer.exe) at:
http://www.putty.nl/download.html
This file installs both PuTTY and Pageant. To add one or more SSH keys to your keychain on a Windows computer and forward them to an HCP node: 1. Start Pageant. 2. In the Pageant Key List window, click on the Add Key button. 3. In the Select Private Key File window, select the file containing the key you want. Then click on the Open button. 4. In the Pageant: Enter Passphrase window, type the passphrase for the key youre adding. Then click on the OK button. 5. Repeat steps 2 through 4 for any additional keys you need for the procedure you want to perform. 6. In the Pageant Key List window, click on the Close button. 7. Start PuTTY. 8. In the PuTTY Configuration window:
a. In the Category list, select Session (if its not the current category). b. In the Host Name (or IP address) field, type the IP address of the
node you want to log into.

c. For the Protocol option, select SSH. d. In the Category list, select Connection SSH Auth.
e. In the Authentication parameters section, select Allow agent forwarding. f.
Click on the Open button.
D2
Adding and forwarding an SSH key in Linux
A PuTTY window opens and prompts for a username. Note: If a message window appears before the PuTTY window opens, click on the OK or Yes button to close it. 9. In response to the login prompt in the PuTTY window, enter service or root, as applicable, for the procedure you want to perform. Tip: If youre prompted for a password at this point, it means your SSH keychain does not include the correct SSH keys for what youre trying to do.

To add one or more SSH keys to your keychain on a Linux computer and forward them to an HCP node: 1. Open a bash shell inside the ssh-agent:
ssh-agent bash
2. In the bash shell, change to the .ssh directory:

cd /directory-path/.ssh
3. For each SSH key you want to add to your keychain:

a. Give yourself read and write permissions for the file containing the
key:
chmod 600 /directory-path/key-file-name b. Add the key to your keychain: ssh-add /directory-path/key-file-name c. When prompted, enter the passphrase for the key.
D3
4. Use SSH with the -A option to log into the node you want as service or root, as applicable, for the procedure you want to perform:
ssh -A username@node-ip-address
Tip: If youre prompted for a password at this point, it means your SSH keychain does not include the correct keys for what youre trying to do.
D4
E
Accessing HCP interfaces through the back-end network
When diagnosing or resolving a problem with an HCP system, you may need to access one of these interfaces:
The HCP Configuration menu on a node in the system The HCP System Management Console The Tenant Management Console for a particular tenant The Namespace Browser for a particular namespace
HCP users access these interfaces through their corporate network. Often, however, customers do not allow outside service personnel to connect to their corporate network. As a result, you may need to access these interfaces through the HCP back-end network. This appendix provides instructions for doing this.
Accessing HCP interfaces through the back-end network Installing and Maintaining an HCP System
E1
Connecting to the back-end network
Connecting to the back-end network

To connect your laptop computer to the HCP back-end network:
Use the Windows Internet Protocol (TCP/IP) Properties window to set

the IP address for your computer to an IP address on the HCP back-end subnet. Be sure to use an address thats not already in use by any of the HCP nodes.
Use an Ethernet cable to connect your computer to any unused port on

any one of the back-end switches in the HCP system.
Accessing the interfaces

Once your laptop computer is connected to the back-end network, you use the back-end IP address of a node to access the HCP Configuration menu on that node. To access the HCP System Management Console, you can use the back-end IP address of any available storage node. For example, for access to the System Management Console for the HCP system that includes a node with a back-end IP address of 10.1.1.16, you would enter this URL in your web browser address field:
https://10.1.1.16:8000
However, for you to access the Tenant Management Console, HCP needs to know the fully qualified hostname of the applicable tenant. This is true regardless of whether you access the Tenant Management Console directly by entering its URL in your web browser address field or indirectly by clicking on its URL in the System Management Console. Similarly, for you to access the Namespace Browser, HCP needs to know the fully qualified hostname of the applicable namespace. This is true regardless of whether you access the Browser directly by entering its URL in your web browser address field or indirectly by clicking on its URL in the Tenant Management Console. Because the back-end network does not support DNS, the browser cannot resolve these URLs to IP addresses. Therefore, for each applicable tenant or namespace, you need to provide a mapping of the tenant or namespace hostname to the back-end IP address of an available storage node in the HCP system. You specify these mappings in the hosts file on your laptop computer:
c:\windows\system32\drivers\etc\hosts
E2
Each line in the hosts file must be a mapping of a fully qualified hostname to a back-end IP address. For example, if the back-end IP address of one of the nodes in the HCP system named hcp-ma.example.com is 10.1.1.16, you would add this line to the hosts file to enable access to the Tenant Management Console for the Finance tenant in that system:
10.1.1.16 finance.hcp-ma.example.com
Similarly, if the Finance tenant has a namespace named AccountsReceivable, you would add this line to the hosts file to enable access to the Namespace Browser for that namespace:
10.1.1.16 accounts-receivable.finance.hcp-ma.example.com
You can use the same or different IP addresses in your hostname mappings. You can include comments in a hosts file either on separate lines or following a mapping on the same line. Each comment must start with a number sign (#). Blank lines are ignored. Tip: You may need to make frequent changes to the hosts file. For convenience, consider putting a shortcut to the file on your desktop.
E3
E4
Glossary
A
access protocol
See namespace access protocol.
active search system

The system that works with the enabled search facility to perform searches and return results to the HCP Search Console. The active search system also maintains an index of objects in searchable namespaces, which it uses for fast retrieval of search results.
alert
A graphic that indicates the status of some particular element of an HCP system in the System Management Console.
Appliance Operating System

The operating system installed on each HCP node.
B
bond
A pair of ports that share an IP address.
C
capacity
The total amount of storage space in HCP, excluding the space required for system overhead and the operating system. This is the amount of space available for all data in the repository, including the fixed-content data, metadata, and any redundant data required to satisfy namespace DPLs.
Glossary1
capacity balancing service
capacity balancing service

The HCP service that ensures that the percentage of space used remains roughly equivalent across all the storage nodes in the system.
CIFS
Common Internet File System. One of the protocols HCP uses to provide access to the contents of the default namespace. CIFS lets Windows clients access files on a remote computer as if they were part of the local file system.
compliance mode
The retention mode in which objects under retention cannot be deleted through any mechanism. This is the more restrictive retention mode.
compression service
The HCP service that compresses object data, thereby freeing space for the storage of additional objects.
content verification service

The HCP service that ensures the integrity of each object by checking that the object data still matches its cryptographic hash value.
core software
See HCP core software.
cross-mapping
With external storage, the mapping of two sets of logical volumes to two nodes such that each set maps to both nodes.
cryptographic hash value

A system-generated metadata value calculated by a cryptographic hash algorithm from object data. This value is used to verify that the content of an object has not changed.
custom metadata
One or more user-defined properties that provide descriptive information about an object. Custom metadata, which is normally specified as XML, enables future users and applications to understand and repurpose object content.
Glossary2
domain
D
data access path
With external storage, the path established by the mapping of a logical volume to a node.
data migration
The process of moving data off devices that are being retired from an HCP system into free storage on the remaining devices. The devices involved can be either storage nodes in an HCP RAIN system or storage arrays in an HCP SAIN system.
data outage
A situation in which one or more data objects are inaccessible (for example, due to multiple concurrent node failures).
data protection level (DPL)

The number of copies of an object HCP must maintain in the repository. Each namespace has its own DPL setting that applies to all objects in that namespace.
default namespace
A namespace that supports the HTTP, WebDAV, CIFS, NFS, SMTP, and NDMP protocols and does not require user authentication for data access. An HCP system can have at most one default namespace.
disposition service
The HCP service that automatically deletes expired objects.
distributor key
The name that determines the SSH keys required for access to the HCP nodes when theyre locked down.
DNS
See domain name system (DNS).
domain
A group of computers and devices on a network that are administered as a unit.
Glossary3
domain name system (DNS)
domain name system (DNS)

A network service that resolves domain names into IP addresses for client access.
DPL
See data protection level (DPL).
duplicate elimination
The process of transparently eliminating redundant data associated with two or more identical objects.
duplicate elimination service

The HCP service that transparently eliminates redundant data, thereby freeing space for the storage of additional objects.
dynamic DPL
A namespace data protection level that, at any given time, matches the system-level DPL setting.
E
enterprise mode
The retention mode in which these operations are allowed:
Privileged delete Changing the retention class of an object to one with a shorter duration Reducing retention class duration Deleting retention classes
F
fixed-content data
A digital asset ingested into HCP and preserved in its original form as the core part of an object. Once stored, fixed-content data cannot be modified.
Glossary4
HCP search facility
G
garbage collection service
The HCP service that deletes data and metadata left in the repository by incomplete operations, thereby freeing space for the storage of additional objects.
H
hash value
See cryptographic hash value.
HCP
See Hitachi Content Platform (HCP).
HCP core software

The software, other than the Appliance Operating System, thats installed on each HCP storage node to create an HCP system.
HCP management API

An HTTP REST interface to a subset of the administrative functions of an HCP system. Using this API, you can manage tenants, namespaces, retention classes, data access accounts, and tenant-level user accounts.
HCP metadata query API

See metadata query API.
HCP namespace
A namespace that requires user authentication for data access. An HCP system can have multiple HCP namespaces.
HCP node
See node.
HCP search facility

The search facility that interacts with HCP. The HCP search facility is available only in HCP systems that included it before being upgraded to release 4.1.
Glossary5
HCP service
HCP service
See service.
HCP Setup
The program you use to install, upgrade, and add nodes to an HCP system.
HCP software
The HCP core software together with the HCP search facility software. The HCP software does not include the Appliance Operating System.
HDDS
See Hitachi Data Discovery Suite (HDDS).
HDDS search facility

The search facility that interacts with Hitachi Data Discovery Suite. To use this facility, you need to first install and configure HDDS, which is a separate product from HCP.
Hitachi Content Platform (HCP)

A distributed object-based storage system designed to support large, growing repositories of fixed-content data. HCP provides a single scalable environment that can be used for archiving, business continuity, content depots, disaster recovery, e-discovery, and other services. With its support for multitenancy, HCP securely segregates data among various constituents in a shared infrastructure. Clients can use a variety of industry-standard protocols and various HCP-specific interfaces to access and manipulate objects in an HCP repository.
Hitachi Data Discovery Suite (HDDS)

A Hitachi product that enables federated searches across multiple HCP systems and other supported systems.
hotfix
An update to an HCP system that resolves a particular problem. Hotfixes are designed to resolve customer problems on a one-off basis.
HTTP
Hypertext Transfer Protocol. One of the protocols HCP uses to provide access to the contents of a namespace. Additionally, the HCP System Management, Tenant Management, and Search Consoles use HTTP to communicate with Console clients.
Glossary6
metadata
HTTPS
HTTP with SSL security. See HTTP and SSL.
I
index
See search index.
indexing service
The HCP service that builds and maintains the search index stored on the search nodes in HCP.
installation SSH key

The SSH key required to log into an HCP node from a laptop computer for the purpose of performing a fresh installation of HCP.
L
link
See replication link.
lockdown
The condition of an HCP node wherein login requires the use of SSH with specific keys.
logical volume
A unit of storage. For internal storage, a logical volume corresponds to a disk partition. For external storage, it corresponds to one or more segments of one or more disks.
M
management API
See HCP management API.
metadata
System-generated and user-supplied information about an object. Metadata is stored as an integral part of the object it describes, thereby making the object self-describing.
Glossary7
metadata query API
metadata query API

An HTTP REST interface that lets you search HCP for objects that meet specified metadata-based criteria. With this API, you can search not only for objects currently in the repository but also for information about objects that have been deleted or purged.
migration service
The HCP service that migrates data off selected storage nodes in an HCP RAIN system or selected storage arrays in an HCP SAIN system in preparation for retiring those devices.
multipathing
With external storage, multiple means of access to a logical volume from a single node.
N
namespace
A logical partition of the objects stored in an HCP system. A namespace consists of a grouping of objects such that the objects in one namespace are not visible in any other namespace. Namespaces are configured independently of each other and, therefore, can have different properties.
namespace access protocol

A protocol that can be used to transfer data to and from namespaces in an HCP system. HCP supports the HTTP protocol for access to HCP namespaces. HCP supports these protocols for access to the default namespace: HTTP, WebDAV, CIFS, NFS, SMTP, and NDMP.
NDMP
Network Data Management Protocol. The protocol HCP supports for backing up and restoring objects in the default namespace.
NFS
Network File System. One of the protocols HCP uses to provide access to the contents of the default namespace. NFS lets clients access files on a remote computer as if they were part of the local file system.
node
A server running HCP software and networked with other such servers to form an HCP system.
Glossary8
protection service
O
object
For a data object, an exact digital representation of data as it existed before it was ingested into HCP, together with the system and custom metadata that describes that data. A data object is handled as a single unit by all transactions and services, including shredding, indexing, versioning, and replication. For a directory or symbolic link in the default namespace, the digital representation of its metadata.
offline upgrade
An upgrade of an HCP system during which the system is automatically shut down and then restarted when the upgrade is complete. During an offline upgrade, the system is not accessible to users and applications.
online upgrade
An upgrade of an HCP system during which the system remains online and accessible to users and applications.
P
permission
The granted ability to access the HCP System Management Console, Tenant Management, or Search Console and to perform a specific activity or set of activities in that Console.
ping
A utility that tests whether an IP address is accessible on the network by requesting a response from it. Also, to use the ping utility.
policy
One or more settings that influence how transactions and services work on objects. Such a setting can be a property of an object, such as retention, or a property of a namespace, such as versioning.
protection service
The HCP service that ensures the stability of the repository by maintaining a set level of data redundancy, as specified by the data protection level (DPL) for each namespace.
Glossary9
protection set
protection set
A group of DPL storage nodes in which HCP tries to store all the copies of a given object.
protocol
See namespace access protocol.
Q
query API
See metadata query API.
R
RAIN
Redundant array of independent nodes. An HCP system configuration in which the nodes use internal or direct-attached storage.
region
An HCP-internal logical grouping of objects and certain types of system data (such as system log messages and usage statistics). HCP uses regions to distribute object metadata and system data among the storage nodes in the system.
replica
The target system to which the replication service copies objects and other information from the primary system during replication.
replication
The process of keeping selected tenants and namespaces in two HCP systems in sync with each other. Basically, this entails copying object creations, deletions, and metadata changes from one system to the other. HCP also replicates tenant and namespace configuration, data access accounts, retention classes, all compliance log messages, and all HCP tenant log messages.
replication link
A configurable association that specifies the HCP systems from and to which the replication service should copy objects and other data.
replication service
The HCP service that performs replication.
Glossary10
Search Console
repository
The aggregate of the namespaces defined for an HCP system.
retention mode
A namespace property that affects which operations are allowed on objects under retention. A namespace can be in either of two retention modes: compliance or enterprise.
retention period
The period of time during which an object cannot be deleted.
retention setting
The property that determines the retention period for an object.
role
A named collection of permissions that can be associated with an HCP user account, where each permission allows the user to perform some specific interaction or set of interactions with the HCP System Management Console, the Tenant Management Console, or, for default namespaces only, the Search Console. Roles generally correspond to job functions.
root SSH key

The SSH key required to log into an HCP node as the root user from a laptop computer.
S
SAIN
SAN-attached array of independent nodes. An HCP system configuration in which the nodes use SAN-attached storage.
scavenging service
The HCP service that ensures that objects in the HCP repository have valid metadata.
Search Console
The web application that provides interactive access to the search functionality of the active search system.
Glossary11
search facility
search facility
An interface between the search functionality provided by a system such as HDDS or HCP and the HCP Search Console. Only one search facility can be enabled at any given time.
search index
An index of the metadata and key terms in namespace objects. The active search system builds, maintains, and stores this index.
search node
An HCP node that runs the HCP search facility software and stores the search index thats built and maintained by HCP.
service
A background process that performs a specific function that contributes to the continuous tuning of the HCP system. In particular, services are responsible for optimizing the use of system resources and maintaining the integrity and availability of the data stored in the HCP repository.
service SSH key

The SSH key required to log into an HCP node as the service user from a laptop computer.
shredding
The process of deleting a data object and overwriting the locations where its bytes were stored in such a way that none of its data or metadata can be reconstructed. Also called secure deletion.
shredding service
The HCP service that shreds deleted objects that are marked for shredding.
SMTP
Simple Mail Transfer Protocol. The protocol HCP uses to receive and store email data directly from email servers.
SNMP
Simple Network Management Protocol. A protocol HCP uses to facilitate monitoring and management of the system through an external interface.
Glossary12
system service tools
SSH
Secure Shell. A network protocol that lets you log into and execute commands in a remote computer. SSH uses encrypted keys for computer and user authentication.
SSL
Secure Sockets Layer. A key-based Internet protocol for transmitting documents through an encrypted link.
SSL server certificate

A file containing cryptographic keys and signatures. When used with the SSL protocol, an SSL server certificate helps verify that the web site holding the certificate is authentic. An SSL server certificate also helps protect data sent to or from that site.
storage node
An HCP node that runs the complete HCP software (except the HCP search facility software) and stores the objects added to HCP.
subdomain
A subset of the computers and devices in a domain.
syslog
A protocol used for forwarding log messages in an IP network. HCP uses syslog to facilitate system monitoring through an external interface.
System Management Console

The system-specific web application that lets you monitor and manage HCP.
system metadata
System-managed properties that describe the content of an object. System metadata includes policies, such as retention and data protection level, that influence how transactions and services affect the object.
system service tools

A set of Python tools that let you act on all nodes in an HCP system with a single command.
Glossary13
tenant
T
tenant
An administrative entity created for the purpose of owning and managing namespaces and data access accounts. Tenants typically correspond to customers, business units, or individuals.
Tenant Management Console

The tenant-specific web application that lets you monitor and manage tenants and namespaces.
U
Unix
Any UNIX-like operating system (such as UNIX itself or Linux).
user account
A set of credentials that gives a user access to one or more of the System Management Console, Tenant Management Console, HCP management API, metadata query API, and Search Console.
user role
See role.
V
volume
See logical volume.
W
WebDAV
Web-based Distributed Authoring and Versioning. One of the protocols HCP uses to provide access to the contents of the default namespace. WebDAV is an extension of HTTP.
WORM
Write once, read many. A data storage property that protects the stored data from being modified or overwritten.
Glossary14
zero-copy failover
Z
zero-copy failover
The process of one node automatically taking over management of storage previously managed by another node that has become unavailable.
Glossary15
zero-copy failover
Glossary16
Index
A
access protocols, namespace 1-3 accessing HCP interfaces through back-end network E-1E-3 Active Directory server time synchronization 2-6 active search system 1-3 adding chassis IP addresses 9-7 nodes 5-25-13 SSH keys in Linux D-3D-4 SSH keys in Windows D-2D-3 storage 7-27-10 administrative interface for HCP search facility, accessing 10-6 advanced alerts 11-2 advanced options for installing HCP software 3-263-27 alerts advanced 11-2 upgrade completion 4-26 AMS-200 2-7 Appliance Operating System 1-10 See also operating system Appliance OS Installation Information form A-2 arc_nodes configuration file section C-2 backup during online upgrade 4-4 BIOS date 3-3 blade servers See CB 320 servers; nodes blade_servers configuration file parameter C-3
C
C components log files 11-611-7 logging levels, changing 11-911-10 canceling region-count changes 9-139-16 capablemsgversion configuration file parameter C-3 CB 320 servers See also nodes about 1-8 adding nodes 5-4 adding/removing chassis IP addresses 9-7 configuring for HCP 2-7 CD trays, ejecting 11-3311-34 chained writes, disabling 10-510-6 Change Password page 3-22 changing DNS settings 9-29-4 HTTP persistent connection timeout interval 9-10 install user password 3-133-14 logging levels 11-811-12 network settings 9-49-7 region count 9-139-16 security account password 3-22 serial numbers 9-99-10 time settings 9-89-9 chassis See CB 320 servers chassis_ip_csl configuration file parameter C-4 checking space for upgrading 4-74-13
B
back end access to HCP interfaces E-1E-3 IP addresses for new nodes 5-7 MTU, changing 9-49-7 network 2-3 node IP addresses, about 2-2 node IP addresses, changing 9-49-7 node IP addresses, setting 3-6 back-end network 1-6 backend_mtu configuration file parameter C-3
Index1
checkpoints, resetting for replication links

checkpoints, resetting for replication links 10-4 10-5 CIFS chained writes, disabling 10-510-6 support 1-3 client errors during online upgrades 4-3 clname configuration file parameter C-4 cluster_get examples 11-2711-29 syntax 11-2411-27 cluster_put examples 11-3111-32 syntax 11-2911-31 cluster_ssh examples 11-2111-24 syntax 11-1711-20 committing upgrades 4-254-27 Compute Blade 320 servers See CB 320 servers config_key configuration file parameter C-2 configuration file, HCP system modifying 3-323-35 parameters C-2C-9 viewing C-1 configuring CB 320 servers 2-7 HCP system 3-73-9, 3-163-19 connecting to back-end network E-2 Console pages Change Password 3-22 DNS Settings 9-39-4 HCP System Startup 3-22 login 3-21 Miscellaneous Settings 9-10, 9-10, 9-11 Network Settings 9-59-6, 9-7 Nodes 3-223-23, 4-25, 5-14, 6-18, 6-25, 7-107-11, 8-11, 9-17, 11-2 Overview 4-26, 10-3, 11-2 Search 8-11 Search Node 8-11, 11-2 Storage Node 8-11, 11-2 Time Settings 9-89-9 Upgrade 4-26 contact_info configuration file parameter C-4 core software See HCP core software creating JVM snapshot 11-1411-15 replication links 2-10 user accounts 2-10 cross-mapped nodes, adding 5-3 cross-mapping 1-91-10 current_time configuration file parameter C-4 currentmsgversion configuration file parameter C-4 custom metadata, storing with object data during online upgrades 4-3 customer support contact information about 2-8 setting 3-18
D
data access accounts about 1-3 creating/configuring during online upgrades 4-3 data migrations about 8-2 finalizing 8-28-9 upgrades during 4-2, 8-2 data outages during data migration 7-12 during online upgrade 4-4 database, recovering 6-4, 6-11 default namespace 1-2 default system configuration, resetting 3-9 default tenant 1-2 deleting encryption keys for log downloads 11-14 deletion records, enabling retention of 2-10 disabling chained writes 10-510-6 DPL one 9-119-12 encryption 3-19 protection service for storage addition 7-4 replication after installation 9-17 replication during installation 2-8, 3-18 services 10-210-3 zero-copy failover after installation 9-10 9-11 zero-copy failover for TrueCopy data migration 7-13 disk_flush configuration file parameter C-4 distributor key about 2-3 setting 3-17 distributor_key configuration file parameter C-4 DNS about 2-42-5 adding new nodes as master servers 5-3 dummy name for HCP system 2-5 settings, changing 9-29-4 settings, setting 3-17 DNS Settings page 9-39-4 dns_enabled configuration file parameter C-5
Index2
getProtectionSets.action page
dns_servers configuration file parameter C-5 domain name system See DNS downloading HCP internal logs encryption keys for 11-1311-14 procedure 11-1211-13 DPL configuring system-level setting 2-10 enabling/disabling DPL one 9-119-12 dummy DNS name for HCP system 2-5 HCP software installation 3-153-16, 4-19 4-22, 5-95-11, 6-76-9, 6-146-16, 6-226-23, 7-57-7 node list 3-273-31 filesystem configuration file parameter C-6 finalizing data migrations procedure 8-48-9 RAIN systems 8-28-3 requirements 8-38-4 SAIN systems 8-3 finding user IDs 11-5 fixed-content storage system 1-2 forms Appliance OS Installation Information A-2 Encryption Key A-5 HCP Node Addition Information A-6 HCP Software Installation Information A-3 A-4 forwarding SSH keys Linux D-3D-4 Windows D-2D-3 freeing space on nodes 4-144-16 front end IP addresses for new nodes 5-7 MTU, changing 9-49-7 network IP address, about 2-2, 2-3, 2-4 network IP address, changing 9-49-7 network IP address, setting 3-5, 3-17 node IP addresses, about 2-2 node IP addresses, changing 9-49-7 node IP addresses, setting 3-5 subnet mask 2-2 front-end network 1-6 frontend_mtu configuration file parameter C-6
E
ejecting CD trays 11-3311-34 enable_replication configuration file parameter C-5 enabling DPL one 9-119-12 encryption 3-19 Management API 2-10 replication after installation 9-169-17 replication during installation 2-8, 3-18 retention of deletion records 2-10 services 10-210-3 zero-copy failover after installation 9-10 9-11 encryption about 2-82-9 alert 11-2 configuration file parameter C-5 enabling/disabling 3-19 key, saving during installation 2-82-9, 3-20 keys for downloading logs 11-1311-14 Encryption Key form A-5 encryption_cipher configuration file parameter C-5 encryption_ivmode configuration file parameter C-6 encryption_key_length configuration file parameter C-6 external storage AMS-200 2-7 architecture 1-71-8 See SAN-attached storage WMS-100 2-7 external time source 2-52-6
G
gateway configuration file parameter C-6 gateway router IP address, about 2-2, 2-3, 2-4 IP address, changing 9-49-7 IP address, setting 3-17 gathering information about 2-1 HCP software installation 2-32-9 HCP upgrade 4-6 node addition 5-5 node recovery 6-36-4 operating system installation 2-22-3 post-installation setup 2-92-10 storage addition 7-3 getAdminView.action page 11-32 getMM.action page 11-33 getProtectionSets.action page 11-33
F
failed upgrades, restarting 4-274-28 fibre channel switches 1-8 files cluster.cfg 3-323-33, C-1C-9
Index3
getUUIDs.action page
getUUIDs.action page 11-33 encryption key during installation 2-82-9, 3-20 information for installing 2-32-9 installation files, retrieving 3-153-16, 4-194-22, 5-95-11, 6-76-9, 6-14 6-16, 6-226-23, 7-57-7 installation requirements 3-113-12 installing 3-123-20 installing, advanced options 3-263-27 verifying installation 3-203-25 HCP Software Installation Information form A-3 A-4 HCP system about 1-21-4 alerts 11-2 configuration, viewing C-1 configuring 3-73-9, 3-163-19 customer support contact information 2-8 DNS settings 2-42-5 encryption 2-82-9 hardware 1-41-8 hotfixes, installing 4-324-36 initial time 2-6 installation, verifying 3-203-25 installing 3-23-20 managing 10-1 monitoring during online upgrades 4-5 nodes 2-9 post-installation configuration 3-26 preparing for storage addition 7-4 preparing to upgrade 4-16 reconfiguring 9-19-2 replication 2-8 serial number 2-7 software 1-101-11 storage array migration 7-117-18 storage configuration 2-7 time source 2-52-6 time zone 2-6 troubleshooting 11-1 upgrade, verifying 4-244-25 upgrading 4-174-27 upgrading, about 4-24-3 HCP System Startup page 3-22 HDDS search facility 1-3 See also search facilities highest-numbered node 2-3 Hitachi Content Platform See HCP system Hitachi Device Manager integration, enabling 2-9 hostname prefix 9-29-4 hostname_prefix configuration file parameter C-6
H
hardware dependent HCP features 1-91-10 networking 1-6 nodes 1-5 overview 1-4 HCP back-end network 1-6 front-end network 1-6 hardware 1-61-8 HCP 300 1-4 HCP 500 1-4 HCP core software 2-1 See also HCP software HCP internal logs daily logs 11-7 downloading 11-1211-13 encryption keys for downloading 11-13 11-14 log files 11-611-7 log rotation 11-711-8 logging levels, about 11-811-9 logging levels, changing for C components 11-911-10 logging levels, changing for JVM 11-10 11-12 managing 11-511-6 HCP namespaces 1-2 HCP Node Addition Information form A-6 HCP search facility See also search facilities about 1-3 administrative interface, accessing 10-6 HCP Setup See also HCP Setup wizard about 3-7 adding storage 7-2, 7-87-10 logs 11-3 HCP Setup wizard See also HCP Setup adding nodes 5-115-13 configuration options 3-83-9, 3-173-19 configuring HCP system 3-163-19 installing HCP 3-163-20 menus B-1B-5 navigating 3-73-8 resetting configuration options 3-9 saving configuration settings 3-9 upgrading HCP 4-224-27 HCP software core 2-1
Index4
mdpl configuration file parameter

hotfixes, installing 4-324-36 HTTP persistent connection timeout interval, changing 9-10 support 1-3
J
JVM log files 11-7 logging levels, changing 11-1011-12 snapshot, creating 11-1411-15
I
indexing during online upgrade 4-4 install user password, changing 3-133-14 password, resetting 11-311-4 installation files, HCP software retrieving for HCP installation 3-153-16 retrieving for HCP upgrade 4-194-22 retrieving for node addition 5-95-11 retrieving for node recovery 6-76-9, 6-14 6-16, 6-226-23 retrieving for storage addition 7-57-7 installation SSH key D-1 installing HCP software 3-123-20 HCP software, advanced options 3-263-27 hotfixes 4-324-36 operating system 3-43-6 operating system for logical-volume recovery 6-12 operating system for OS recovery 6-6 operating system for search node recovery 6-21 operating system on new nodes 5-8 using modified configuration file 3-323-35 using node-list files 3-273-31 internal logs See HCP internal logs internal storage architecture 1-61-7 internal time source 2-5 IP addresses back end, about 2-2 back end, changing 9-49-7 back end, setting 3-6 front end, changing 9-49-7 front end, setting 2-2, 3-5 front-end network, about 2-4 front-end network, changing 9-49-7 front-end network, setting 2-2, 2-3, 3-5, 3-17 multicast, about 2-4 multicast, changing 9-49-7 multicast, setting 3-17 IP addresses for new nodes 5-7
K
kerberos_ssh configuration file parameter C-6 keys encryption 2-82-9 SSH D-1
L
links, resetting checkpoints 10-410-5 Linux, adding and forwarding SSH keys D-3D-4 listing encryption keys for log downloads 11-14 lockdown configuration file parameter C-7 logging in administrator interface for HCP search facility 10-6 HCP software installation 3-123-14 HCP upgrade 4-174-19 migration finalization 8-5 node addition 5-8 node recovery 6-6, 6-13, 6-21 storage addition 7-47-5 System Management Console 3-21 logical volumes adding 7-27-10 cross-mapping 1-91-10 multipathing 1-9 recovering 6-126-18 viewing on getUUIDs.action page 11-33 login page 3-21 logs See also HCP internal logs HCP Setup 11-3 server SEL 11-6, 11-8
M
Management API, enabling 2-10 managing HCP internal logs 11-511-6 HCP system 10-1 services 10-2 max_simul_disks configuration file parameter C-7 max_simul_nodes configuration file parameter C-7 mcast configuration file parameter C-7 mdpl configuration file parameter C-7
Index5
menus, HCP Setup wizard

menus, HCP Setup wizard B-1B-5 metadata 1-2 migrating data See also data migrations Tiered Storage Manager 7-18 TrueCopy 7-127-15 migration service 8-2 See also data migrations Miscellaneous Settings page 9-10, 9-10, 9-11 mm_filesystem configuration file parameter C-7 modifying HCP system configuration file 3-323-35 list of chassis IP addresses 9-7 monitoring HCP during online upgrades 4-5 MTU, changing 9-49-7 multicast IP address about 2-4 changing 9-49-7 setting 3-17 multipathing about 1-9 alert 11-2 storage, procedure 6-56-18 storage, verifying 6-18 node-list files 3-273-31 nodes about 1-5 adding 5-25-13 adding cross-mapped 5-3 adding in CB 320 servers 5-4 adding storage to 7-27-10 back-end IP address, about 2-2 CB 320 server blades 1-8 changing logging levels 11-811-12 downloading HCP internal logs 11-1211-13 freeing space 4-144-16 front-end IP address 2-2 highest numbered 2-3 hostname prefix 9-29-4 identifying during installation 2-9, 3-19, 3-273-31 IP addresses for new 5-7 IP addresses, changing 9-49-7 IP addresses, setting 3-53-6 locating 11-3311-34 recovering 6-2 removing 8-98-11 retiring in RAIN systems 8-28-3 search 1-5 shutting down 8-11 storage 1-5 system event log (SEL) 11-6, 11-8 updating to recognize new storage arrays 7-147-15 verifying addition 5-135-14 viewing on getAdminView.action page 11-32 Nodes page alerts 11-2 removing nodes 8-11 restarting HCP 9-17 verifying HCP installation 3-223-23 verifying HCP upgrade 4-25 verifying node addition 5-14 verifying storage addition 7-107-11 verifying storage node recovery 6-18, 6-25
N
Namespace Browser, access through back-end network E-2E-3 namespaces about 1-2 access protocols 1-3 creating/configuring during online upgrades 4-3 navigating HCP Setup wizard 3-73-8 NDMP 1-3 Network Settings page 9-59-6, 9-7 network settings, changing 9-49-7 networking about 1-6 installation information 2-4 replication 1-81-9 NFS chained writes, disabling 10-510-6 mounts during online upgrade 4-34-4 support 1-3 node addition about 5-25-3 information for 5-5 IP addresses for new nodes 5-7 procedure 5-75-13 verifying 5-135-14 node recovery search, procedure 6-216-25 search, verifying 6-25 storage, information for 6-36-4
O
objects about 1-2 storage 1-5 offline upgrades See also upgrading about 4-3 rolling back 4-294-32
Index6
restricting access to System Management Console

old_software_version configuration file parameter C-8 online upgrades See also upgrading about 4-34-5 considerations 4-34-5 monitoring HCP during 4-5 opening administrator interface for HCP search facility 10-6 operating system information for installing 2-22-3 installation requirements 3-23-3 installing 3-43-6 installing on new nodes 5-8 recovering 6-56-12 OS See operating system Overview page alerts 4-26, 11-2 Services section 10-3 rebuilding search index 8-88-9, 8-10 reconfiguring HCP system 9-19-2 SAN-attached storage 7-117-12 recovering logical volumes 6-126-18 operating system 6-56-12 search nodes 6-196-25 storage nodes 6-2 regionmapsize configuration file parameter C-8 regions about 9-12 count, about 9-129-13 count, canceling changes 9-139-16 count, changing 9-139-16 viewing current 11-33 viewing on getMM.action page 11-33 removing chassis IP addresses 9-7 nodes 8-98-11 replication about 1-4 enabling/disabling after installation 9-16 9-17 enabling/disabling during installation 2-8, 3-18 links, creating 2-10 modifying link during online upgrade 4-5 network connections 1-81-9 resetting link checkpoints 10-410-5 resuming after HCP upgrade 4-27 service 10-3 suspending for HCP upgrade 4-16 requirements finalizing data migrations 8-38-4 freeing space on nodes 4-14 HCP software installation 3-113-12 HCP upgrade 4-54-7 hotfix installation 4-33 node addition 5-45-7 offline upgrade rollback 4-30 operating system installation 3-23-3 search node recovery 6-196-20 storage addition 7-27-3 storage node recovery 6-36-5 upgrade prechecks 4-7 resetting default system configuration 3-9 install user password 11-311-4 replication link checkpoints 10-410-5 restarting failed upgrades 4-274-28 restricting access to System Management Console 2-10
P
Pageant D-2 passwords administrative interface for HCP search facility 10-6 install user, changing 3-133-14 install user, resetting 11-311-4 security account, changing 3-22 persistent connection timeout interval, changing 9-10 ping, disabling 2-9 policy_enabled configuration file parameter C-8 post-installation configuration 3-26 setup information 2-92-10 prechecks for upgrading 4-74-13 preparing HCP system for storage addition 7-4 HCP system for upgrade 4-16 protection service disabling for storage addition 7-4 during online upgrade 4-4 reenabling after storage addition 7-10 protection sets, viewing current 11-33 protocols, namespace access 1-3 PuTTY 3-11, D-2D-3
R
RAIN systems about 1-4 architecture 1-61-7
Index7
resuming replication after HCP upgrade

resuming replication after HCP upgrade 4-27 retention classes, creating/configuring during online upgrades 4-3 retiring devices about 8-1 nodes 8-28-3 storage arrays 8-2, 8-3 retrieving HCP software installation files HCP installation 3-153-16 HCP upgrade 4-194-22 node addition 5-95-11 node recovery 6-76-9, 6-146-16, 6-22 6-23 storage addition 7-57-7 reviewing SAN-attached storage configuration 3-33-4 roles 1-11 rolling back offline upgrades about 4-29 procedure 4-304-32 resources for 4-30 root SSH key D-1 running system service tools 11-16 upgrade prechecks 4-74-13 adding search nodes 5-3 rebuilding 8-88-9, 8-10 search node recovery 6-19 storage 1-5 Search Node page removing nodes 8-11 shutting down nodes 8-11 Search Node page alerts 11-2 search nodes See also search facilities; search index about 1-5 adding 5-3 recovering 6-196-25 removing 8-10, 8-11 retiring 8-1 Search page 8-11 search_nodes configuration file section C-3 security account about 1-11 changing password 3-22 SEL 11-6, 11-8 serial number about 2-7 changing 9-99-10 setting 3-18 verifying 3-21 serial_number configuration file parameter C-8 service activities 1-111-15, 2-92-10 Service menu, verifying HCP installation 3-23 3-25 service shell 3-93-11 service SSH key D-1 services about 1-4 disabled during online upgrades 4-4 enabling/disabling 10-210-3 managing 10-2 migration 8-2 starting/stopping 10-210-3 shared storage 2-7 shutting down nodes 8-11 single_node configuration file parameter C-8 SMTP 1-3 snapshot of JVM, creating 11-1411-15 SNMP enabling 2-10 logging, enabling 2-10 software_version configuration file parameter C-9 space for upgrading 4-74-13 SSH keys about D-1 adding and forwarding in Linux D-3D-4
S
SAIN systems See also SAN-attached storage about 1-4 architecture 1-71-8 fibre channel switches 1-8 SAN-attached storage See also SAIN systems array migration 7-117-18 configuration review 3-33-4 retiring arrays 8-2, 8-3 shared 2-7 virtualizing 7-157-17 saving system configuration settings 3-9 scp 11-13 screen command 3-10 Search Console See also search facilities about 1-3 use during online upgrades 4-4 search facilities See also search index; search nodes about 1-31-4 enabling 2-10 search index See also search facilities; search nodes about 1-3
Index8
upgrading
adding and forwarding in Windows D-2D-3 opening install shell 3-10 SSH, enabling/disabling 2-10 SSL server certificate, new 2-10 starter account 1-11 starting services 10-210-3 stopping services 10-210-3 storage adding 7-27-10 reviewing SAN-attached configuration 3-3 3-4 shared SAN-attached 2-7 type, setting 3-18 types 1-5, 2-7 verifying addition 7-107-11 storage addition information for 7-3 preparing HCP system 7-4 procedure 7-37-10 verifying 7-107-11 storage arrays See SAN-attached storage Storage Node page alerts 11-2 removing nodes 8-11 shutting down nodes 8-11 storage nodes about 1-5 recovering 6-2 retiring in RAIN systems 8-1 retiring in SAIN systems 8-1 verifying recovery 6-18, 6-25 storage_type configuration file parameter C-9 subnet back end, about 2-2 back end, changing 9-5 mask, front end 2-2 masks, changing 9-49-7 masks, setting 3-5 suspending replication for HCP upgrade 4-16 syslog logging, enabling 2-10 system health, verifying after installation 3-23 3-25 System Management Console about 1-11 access through back-end network E-2 alerts 11-2 logging in 3-21 restricting access 2-10 verifying HCP installation 3-213-23 verifying HCP upgrade 4-244-25 verifying node addition 5-135-14 verifying node recovery 6-18, 6-25 verifying storage addition 7-107-11 system service tools about 11-1511-16 cluster_get 11-2411-29 cluster_put 11-2911-32 cluster_ssh 11-1711-24 output 11-17 running 11-16 target nodes 11-1611-17 target system 11-16 system status pages, viewing 11-3211-33 system-level DPL setting 2-10, 9-119-12
T
Tenant Management Console about 1-11 access through back-end network E-2E-3 tenants about 1-2 creating/configuring during online upgrades 4-3 Tiered Storage Manager about 7-11 migrating data 7-18 time settings, about 2-52-6 settings, changing 9-89-9 settings, setting 3-18 Time Settings page 9-89-9 timeserver configuration file parameter C-9 timezone configuration file parameter C-9 troubleshooting 11-1 TrueCopy about 7-11 migrating data 7-127-15 type configuration file parameter C-2
U
updating nodes to recognize new storage arrays 7-147-15 Upgrade page 4-26 upgrading about 4-2 committing upgrades 4-254-27 completion alert 11-2 during data migration 4-2, 8-2 HCP system 4-174-27 information for 4-6 in-progress alert 11-2 offline 4-3
Index9
upgrading, online
online 4-34-5 prechecks 4-74-13 preparing HCP system 4-16 requirements 4-54-7 restarting failed upgrades 4-274-28 rolling back offline upgrades 4-294-32 space for 4-74-16 uploading encryption keys for log downloads 11-14 URLs administrator interface for HCP search facility 10-6 System Management Console 3-21 user accounts about 1-11 creating 2-10 creating/configuring during online upgrades 4-3 security 1-11 user roles 1-11 using HCP Setup wizard 3-73-9 zero-copy failover about 1-91-10 alerts 11-2 disabling for logical volume recovery 6-12 disabling for storage reconfiguration 7-13 enabling/disabling after installation 9-10 9-11 during online upgrade 4-4 reenabling after logical-volume recovery 6-18 reenabling after storage reconfiguration 7-15
V
verifying HCP installation 3-203-25 HCP upgrade 4-244-25 hotfix installation 4-36 node addition 5-135-14 offline upgrade rollback 4-32 serial number 3-21 storage addition 7-107-11 storage node recovery 6-18, 6-25 viewing current node status 11-32 current protection sets 11-33 current regions 11-33 HCP system configuration C-1 logical volume information 11-33 system status pages 11-3211-33 virtual terminal session 3-93-11 virtualizing SAN-attached storage 7-157-17
W
WebDAV 1-3 Windows, adding and forwarding SSH keys D-2 D-3 WMS-100 2-7 WORM 1-2
Z
zero_copy_failover configuration file parameter C-9
Index10
Hitachi Data Systems Corporate Headquarters 750 Central Expressway Santa Clara, California 95050-2627 U.S.A. Phone: 1 408 970 1000 www.hds.com info@hds.com Asia Pacific and Americas 750 Central Expressway Santa Clara, California 95050-2627 U.S.A. Phone: 1 408 970 1000 info@hds.com Europe Headquarters Sefton Park Stoke Poges Buckinghamshire SL2 4HD United Kingdom Phone: + 44 (0)1753 618000 info.eu@hds.com
FE-98ARC018-07

Installing and Maintaining An H CP System

Hochgeladen von

Dokumentinformationen

Originalbeschreibung:

Originaltitel

Copyright

Verfügbare Formate

Dieses Dokument teilen

Dokument teilen oder einbetten

Freigabeoptionen

Stufen Sie dieses Dokument als nützlich ein?

Sind diese Inhalte unangemessen?

Copyright:

Verfügbare Formate

Installing and Maintaining An H CP System

Hochgeladen von

Copyright:

Verfügbare Formate

Hitachi Content Platform

Installing and Maintaining an HCP System

Gathering installation information .......................................................2-1

Contents Installing and Maintaining an HCP System

Internals . . . . . . . . . . . . . . . . . . . . . Security . . . . . . . . . . . . . . . . . . . . . . HCP nodes . . . . . . . . . . . . . . . . . . . . Post-installation configuration information

.... .... .... ....

2-6 2-8 2-9 2-9

Installing an HCP system ...................................................................3-1

Upgrading an HCP system .................................................................4-1

Contents Installing and Maintaining an HCP System

Adding nodes to an HCP system........................................................5-1

Contents Installing and Maintaining an HCP System

Maintaining HCP nodes......................................................................6-1

Managing storage in an HCP SAIN system........................................7-1

Contents Installing and Maintaining an HCP System

Retiring devices ..................................................................................8-1

Reconfiguring HCP system software..................................................9-1

Contents Installing and Maintaining an HCP System

10 Managing an HCP system................................................................10-1

11 Troubleshooting an HCP system......................................................11-1

Forms ................................................................................................ A-1

Contents Installing and Maintaining an HCP System

HCP Setup wizard menus ................................................................. B-1

Accessing HCP interfaces through the back-end network ................ E-1

Contents Installing and Maintaining an HCP System

Contents Installing and Maintaining an HCP System

Preface Installing and Maintaining an HCP System

Chapter 10, Managing an HCP system

Chapter 11, Troubleshooting an HCP system Appendix A, Forms

Appendix B, HCP Setup wizard menus

Preface Installing and Maintaining an HCP System

Preface Installing and Maintaining an HCP System

Administering HCP This book explains how to use an HCP system to

Managing a Tenant and Its Namespaces This book contains complete

Managing the Default Tenant and Namespace This book contains

Replicating Tenants and Namespaces This book covers all aspects of

Preface Installing and Maintaining an HCP System

HCP Management API Reference This book contains the information

Using a Namespace This book describes the properties of objects in

Searching Namespaces This book describes the HCP Search Console.

Preface Installing and Maintaining an HCP System

Installing an HCP System This book provides the information you

Third-Party Licenses and Copyrights This book contains copyright

HCP-DM Third-Party Licenses and Copyrights This book contains

Installing an HCP 500 System Final On-site Setup This book

Assembling and Configuring an HCP 300 System This book contains

Installing an HCP 300 System Final On-site Setup This book

Maintaining HCP 300 System Hardware This book is the hardware

Preface Installing and Maintaining an HCP System

Preface Installing and Maintaining an HCP System

Preface Installing and Maintaining an HCP System

HCP overview Installing and Maintaining an HCP System

Introduction to Hitachi Content Platform

Introduction to Hitachi Content Platform

HCP overview Installing and Maintaining an HCP System

Introduction to Hitachi Content Platform

HCP overview Installing and Maintaining an HCP System

HCP overview Installing and Maintaining an HCP System

Nodes and storage

HCP overview Installing and Maintaining an HCP System

HCP overview Installing and Maintaining an HCP System

The table below describes the cables in this figure: