Installing or Replacing a Flash Cache Module or PAM

You can install the caching modules256-GB and 512-GB Flash Cache modules and 16-GB PAMs in an existing storage system as a first-time installation, as an addition to a storage system that already has caching modules, as an upgrade from PAMs to Flash Cache modules, or as replacements when a caching module is no longer functioning.

Installation requirements
Before you install or replace a caching module, your system must meet certain criteria, depending on your situation. If your system does not meet these criteria, you should contact NetApp technical support. The System Configuration Guide at http:/NOW/knowledge/docs/hardware/hardware_index.shtml contains the most up-to-date information about supported Data ONTAP versions, supported storage systems, and slot assignments for your caching module.

Your system must be running a version of Data ONTAP that supports the caching module you are installing. The system on which you are installing the caching module must be supported. You must check the System Configuration Guide to determine which PCIe slots are supported for your system and your system must have an available PCIe slot for each caching module you are installing. For HA pair configurations, you must install an equal number of caching modules in each node. For first-time caching module installations, you must have the software license key and download instructions that came with your order so that you can enable the software license. You can also access this information at http://now.netapp.com/licenses.

When upgrading 16-GB PAMs to 256-GB or 512-GB Flash Cache modules, you must have purchased the WAFL external software license that supports Flash Cache modules. There are two different WAFL external software licenses. One is used with PAMs, and one is used with Flash Cache modules.

When installing additional caching modules that are a different type than are currently installed, you must verify that your system supports the mixed caching module types. For example, some storage systems support mixing 256-GB and 512-GB modules.
Note

You cannot mix PAMs with Flash Cache modules.

All other components in the system must be functioning properly; if not, you should contact technical support.

Installing or Replacing a Flash Cache Module or PAM

215-03515_D0 Copyright 2010 NetApp, Inc. All rights reserved.

Installing or replacing a caching module

Installing or replacing a caching module involves opening your system, running diagnostics, and if you are installing a caching module for the first-time or upgrading PAMs to Flash Cache modules, enabling the software license and configuration options. Detailed instructions for Step 1 through 5 are available at the following site, organized by platform: http://now.netapp.com/NOW/knowledge/docs/hardware/hardware_index.shtml.

The Hardware Overview that came with your storage system contains instructions about shutting down, opening, closing, and rebooting your system in order to access the PCIe slots. The Replacing PCI Cards and Risers flyer contains instructions about removing and installing a PCIe card.

1. Shut down the controller in which you are replacing or adding a caching module. 2. Open your system. 3. Remove the failed caching module (if necessary). 4. Install the new caching module. Note

If you are installing a PAM, some PAMs might look different because of the arrangement of the DRAM. The difference is based on DRAM technology, but the module functionality is the same and the modules are compatible.
5. Close and boot your system. 6. Press Ctrl-c to stop the boot process. 7. Run diagnostics on the new caching module.

See Running diagnostics on page 3.

8. Complete the installation process: IF you are installing an additional caching module in a system with preexisting caching modules,

you have completed the installation process and are done with this flyer.
IF you are replacing a failed caching module, go to Step 11. IF you are installing a caching module for the first time or upgrading the 16-GB PAMs to 256-GB

or 512-GB Flash Cache modules, complete Steps 9 and 10.

9. Enable the WAFL external cache software.

See Enabling the WAFL external cache software license and functionality on page 6.
10. Set the software configuration options.

See Enabling the WAFL external cache configuration options on page 7.

11. If you replaced a caching module, return the failed part to NetApp, following the RMA instructions

shipped with the kit. Contact NetApp technical support at 888-463-8277 (North America), 00-800-44-NETAPP (Europe), or +800-800-80-800 (Asia/Pacific) if you need the RMA number or additional help with the replacement procedure.

Installing or replacing a caching module

Running diagnostics
After installing a new or replacement caching module, you should run diagnostics. Depending on your platform model, you use the SYSDIAG tool or system-level diagnostics to run diagnostics.

Running SYSDIAG tool diagnostics

Complete the following steps to run diagnostics on platforms earlier than 32xx and 62xx.
Note You can stop the diagnostics test at any time without harm to the system by pressing Ctrl-c. 1. From the boot menu, select the Maintenance mode boot option to bring up the Maintenance mode

prompt.
2. Enter the following command at the prompt:
halt

3. Enter the following command at the Loader prompt to bring up the diagnostics menu:
boot_diags

4. Enter the applicable command at the prompt: IF you have a PAM, enter the following command:
iomem

IF you have a Flash Cache module, enter the following command:

fcache

5. When prompted, enter the following selection to start the module selection process:
90

6. When prompted, enter the slot number of each module you want to test. 7. When prompted, enter the following selection to verify your configuration:
51

Result: The output for each slot you designated should be PASSED.

If the result is something other than PASSED, see the Diagnostics Guide for a list of error messages and corrective actions at http://now.netapp.com/NOW/knowledge/docs/hardware/NetApp/diag/diag.pdf.
8. If you have a Flash Cache module, complete the following substeps; otherwise, go to Step 7.

The memory size of the caching module dictates the amount of time a full memory test takes. For example, running a full memory test on a Flash Cache module can take about one hour, so you want to control the test time by setting the test address range.
a. When prompted, enter the following selection:
81

b. Answer the prompts to set the address range.

Installing or Replacing a Flash Cache Module or PAM

c. When prompted, enter the following selection to verify the address range you designated for

the Flash Cache module:

80

9. When prompted, enter the following selection to run the comprehensive test on the new caching

module:
1

Result: The output for each slot you designated should be PASSED.

If the result is something other than PASSED, see the Diagnostics Guide for a list of error messages and corrective actions at http://now.netapp.com/NOW/knowledge/docs/hardware/NetApp/diag/diag.pdf.
10. Exit diagnostics by entering the following command:
exit

11. Return the system to normal operation: IF your system is an HA pair, run the cf giveback command (if in a 7-Mode system) or storage failover giveback command (if in a Cluster-Mode system) from the partner node console. IF your system is a stand-alone configuration, run the boot_ontap command at the prompt to boot the system.

Running system-level diagnostics

Complete the following steps to run diagnostics on 32xx and 62xx platforms.
Note You can stop the diagnostics test at any time without harm to the system by entering the command sldiag device stop. 1. Complete the applicable action depending on where the node halted during the boot process: IF the node halted at the Loader prompt, go to Step 2. IF the node halted at the boot menu, complete the following substeps: a. Select the Maintenance mode option from the displayed menu. b. Enter the following command at the prompt:
halt

c. Go to Step 2. 2. Enter the following command at the Loader prompt:

boot_diags

Result: The Maintenance mode prompt (*>) appears. Note

You must run this command from the Loader prompt for system-level diagnostics to function properly. The boot_diags command starts special drivers designed specifically for system-level diagnostics.

Running diagnostics

3. Enter the following command at the Maintenance mode prompt:

sldiag

For details about the sldiag command, see the sldiag man page.
4. Clear the status logs by entering the following command:
sldiag device clearstatus

5. Verify that the log is cleared by entering the following command:

sldiag device status

Result: The following default response is displayed:

SLDIAG: No log messages are present.

6. Run the test for the Flash Cache module you installed by entering the following command at the

Maintenance mode prompt:

sldiag device run -dev fcache

7. View the status of the test by entering the following command:

sldiag device status

Result: Your storage system provides the following output while the tests are still running:
There are still test(s) being processed.

After all the tests are complete, the following response appears by default: *> <SLDIAG:_ALL_TESTS_COMPLETED>
8. Verify that no hardware problems resulted from the addition or replacement of hardware

components on your system by entering the following command:

sldiag device status -dev fcache -long -state failed

Result: If there are hardware problems, the prompt is followed by the status of the test failures. If

there are no hardware problems, only the prompt is displayed.

9. If the system-level diagnostics tests resulted in test failures, go to Step 9; otherwise, complete the

following substeps:
a. Clear the status logs by entering the following command:
sldiag device clearstatus

b. Verify that the log is cleared by entering the following command:

sldiag device status

Result: The following default response is displayed:

SLDIAG: No log messages are present.

c. Exit Maintenance mode by entering the following command:

halt

d. Enter the following command at the firmware prompt to reboot the storage system:
boot

Installing or Replacing a Flash Cache Module or PAM

e. If your system is an HA pair, run the cf giveback command (if in a 7-Mode system) or
storage failover giveback command (if in a Cluster-Mode system) from the partner node

console. You have completed system-level diagnostics.

10. If the system-level diagnostics tests resulted in some failures, determine the cause of the problem

by completing the following substeps:

a. Exit Maintenance mode by entering the following command:
halt

b. Perform a clean shutdown and disconnect the power supplies. c. Verify that you have observed all the considerations identified for running system-level

diagnostics, that cables are securely connected, and that hardware components are properly installed in the storage system.
d. Reconnect the power supplies and power on the storage system. e. Rerun the system-level diagnostics tests.

Enabling the WAFL external cache software license and functionality

For first-time installations of a caching module, or for upgrading PAMs to Flash Cache modules, you must enable the WAFL external cache software license for your caching module type. One WAFL external cache software licenses is used with PAMs, and one is used with Flash Cache modules.
1. See the software license key and download instructions that came with your order, or go to

http://now.netapp.com/licenses.
2. Enable the WAFL external cache software license by entering the following command: Note For 7-Mode, run this command from the system console prompt. For HA pair configurations, run this command from each node. For Cluster-Mode, run this command from the nodeshell of the target node.
license add license key

Result: WAFL external cache functionality is enabled automatically. 3. Verify that WAFL external cache functionality is enabled by entering the following command: Note For 7-Mode, run this command from the system console prompt. For HA pair configurations, run this command from each node. For Cluster-Mode, run this command from the nodeshell of the target node.
options flexscale.enable

Result: The output should show that the functionality is on.

Enabling the WAFL external cache software license and functionality

Enabling the WAFL external cache configuration options

WAFL external cache configuration options enable you to optimize performance for the systems particular workload. Depending on the workload, the following four options can apply:

Caching normal user data blocks Caching low-priority user data blocks Caching only system metadata Integrating FlexShare buffer cache policies with WAFL external cache options
Note Systems running Cluster-Mode do not support FlexShare.

To enable WAFL external cache configuration options, complete the following steps:
Note You can set the WAFL external cache configuration options during module installation or later. However, it is recommended that you set options at the time of installation to ensure optimal performance for the systems workload. 1. Read about the WAFL external cache configuration options in the Data ONTAP System

Administration Guide. You can access this document at http://now.netapp.com/NOW/knowledge/docs/docs.cgi.

Note Knowing your application and volume configuration helps you decide what configuration options to choose. 2. Follow the applicable option enabling procedures in the Data ONTAP System Administration

Guide.
Note For 7-Mode HA pair configurations, you must configure options on each node. For Cluster-Mode, you must configure options from the nodeshell of the target node.

Installing or Replacing a Flash Cache Module or PAM