Tivoli Storage Manager for AIX Administrator's Guide

6/14/2013

Tivoli Storage Manager for AIX Administrator's Guide

Migration of Files in a Storage Pool Hierarchy
The server provides automatic migration to maintain free space in a primary storage pool. The server can migrate data from one storage pool to the next storage pool in the hierarchy. This process helps to ensure that there is sufficient free space in the storage pools at the top of the hierarchy, where faster devices can provide the most benefit to clients. For example, the server can migrate data stored in a random access disk storage pool to a slower but less expensive sequential access storage pool. You can control:

When migration begins and ends

You use migration thresholds to control when migration begins and ends. Thresholds are set as levels of the space that is used in a storage pool, expressed as a percent of total space available in the storage pool. For a disk storage pool, the server compares the threshold with a calculation of the amount of data stored in the pool as a percent of the actual data capacity of the volumes in the pool. For a sequential access storage pool, the server compares the threshold with a calculation of the number of volumes containing data as a percent of the total number of volumes available to the pool.

How the server chooses files to migrate

By default, the server does not consider how long a file has been in a storage pool or how long since a file was accessed before choosing files to migrate. Optional parameters allow you to change the default. You can ensure that files remain in a storage pool for a minimum amount of time before the server migrates them to another pool. To do this, you set a migration delay period for a storage pool. Before the server can migrate a file, the file must be stored in the storage pool at least as long as the migration delay period. For disk storage pools, the last time the file was accessed is also considered for migration delay. Migration processing differs for disk storage pools versus sequential access storage pools. If you plan to modify the default migration parameter settings for storage pools or want to understand how migration works, you should read the following sections: Migration for Disk Storage Pools Migration for Sequential Access Storage Pools

Migration for Disk Storage Pools

When you define or update a storage pool, you can set migration thresholds to specify when the server should begin and stop migrating data to the next storage pool in the storage hierarchy. Migration thresholds are defined in terms of a percentage of total data capacity for the disk storage pool. You can use the defaults for the migration thresholds, or you can change the threshold values to identify the maximum and minimum amount of space for a storage pool. See How the Server Selects Files to Migrate and Choosing Appropriate Migration Threshold Values for more information about migration thresholds. You can control how long files must stay in a storage pool before they are eligible for migration by setting a migration delay for a storage pool. See Keeping Files in a Storage Pool. If you decide to enable cache for disk storage pools, files can temporarily remain on disks even after migration. You may want to set migration thresholds lower when you use cache. See Minimizing Access Time to Migrated Files and Using Cache on Disk Storage Pools for information about using the cache.

How the Server Selects Files to Migrate

When data in a storage pool uses a percentage of the pool's capacity that is equal to the high migration threshold, the server migrates files from the pool to the next storage pool. The server selects the files to migrate as follows: 1. The server checks for the client node that has backed up or migrated the largest single file space or has archived files that occupy the most space. 2. For all files from every file space belonging to the client node that was identified, the server examines the number of days since the files were stored in the storage pool and last retrieved from the storage pool. The server compares the number (whichever is less) to the migration delay that is set for the storage pool. The server migrates any of these files for which the number is more than the migration delay set for the storage pool. 3. After the server migrates the files for the first client node to the next storage pool, the server checks the low migration threshold for the storage pool. If the amount of space that is used in the storage pool is now below the low migration threshold, migration ends. If not, the server chooses another client node by using the same criteria as described above, and the migration process continues. The server may not be able to reach the low migration threshold for the pool by migrating only files that have been stored longer than the migration delay period. When this happens, the server checks the storage pool characteristic that determines whether migration should stop even if the pool is still above the low migration threshold. See Keeping Files in a Storage Pool for more information. If multiple migration processes are running (controlled by the MIGPROCESS parameter of the DEFINE STGPOOL command), the server may choose the files from more than one node for migration at the same time.
http://www.urz.uni-heidelberg.de/UnixCluster/Hinweise/Hilfe/System/Adsm/ibmdoc.tsm41/html/aix/guide/anragd87.htm 1/5

Tivoli Storage Manager for AIX Administrator's Guide

6/14/2013

For example, Table 21 displays information that is contained in the database that is used by the server to determine which files to migrate. This example assumes that the storage pool contains no space-managed files. This example also assumes that the migration delay period for the storage pool is set to zero, meaning any files can be migrated regardless of time stored in the pool or the last time of access.

Table 21. Database Information on Files Stored in DISKPOOL Client Node

TOMC CAROL PEASE

Backed-Up File Spaces and Sizes

TOMC/C TOMC/D CAROL PEASE/home PEASE/temp 200MB 100MB 50MB 150MB 175MB

Archived Files (All Client File Spaces)

55MB 5MB 40MB

Figure 20. The Migration Process and Migration Thresholds

Figure 20 shows what happens when the high migration threshold defined for the disk storage pool DISKPOOL is exceeded. When the amount of migratable data in DISKPOOL reaches 80%, the server performs the following tasks: 1. Determines that the TOMC/C file space is taking up the most space in the DISKPOOL storage pool, more than any other single backed-up or space-managed file space and more than any client node's archived files. 2. Locates all data belonging to node TOMC stored in DISKPOOL. In this example, node TOMC has backed up or archived files from file spaces TOMC/C and TOMC/D stored in the DISKPOOL storage pool. 3. Migrates all data from TOMC/C and TOMC/D to the next available storage pool. In this example, the data is migrated to the tape storage pool, TAPEPOOL. The server migrates all of the data from both file spaces belonging to node TOMC, even if the occupancy of the storage pool drops below the low migration threshold before the second file space has been migrated. If the cache option is enabled, files that are migrated remain on disk storage (that is, the files are cached) until space is needed for new files. For more information about using cache, see Using Cache on Disk Storage Pools. 4. After all files that belong to TOMC are migrated to the next storage pool, the server checks the low migration threshold. If the low migration threshold has not been reached, then the server again determines which client node has backed up or migrated the largest single file space or has archived files that occupy the most space. The server begins migrating files belonging to that node. In this example, the server migrates all files that belong to the client node named PEASE to the TAPEPOOL storage pool. 5. After all the files that belong to PEASE are migrated to the next storage pool, the server checks the low migration threshold again. If the low migration threshold has been reached or passed, then migration ends.

Choosing Appropriate Migration Threshold Values

Setting migration thresholds for disk storage pools ensures sufficient free space on faster speed devices, which can lead to better performance. Choosing thresholds appropriate for your situation takes some experimenting, and you can start by using the default values. You need to ensure that migration occurs frequently enough to maintain some free space but not so frequently that the device is unavailable for other use.
Choosing the High-Migration Threshold

To choose the high-migration threshold, consider:

http://www.urz.uni-heidelberg.de/UnixCluster/Hinweise/Hilfe/System/Adsm/ibmdoc.tsm41/html/aix/guide/anragd87.htm

2/5

Tivoli Storage Manager for AIX Administrator's Guide

6/14/2013

The amount of storage capacity provided for each storage pool The amount of free storage needed for users to store additional files, without having migration occur If you set the high-migration threshold too high, the pool may be just under the high threshold, but not have enough space to store an additional, typical client file. Or, with a high threshold of 100%, the pool may become full and a migration process must start before clients can back up any additional data to the disk storage pool. In either case, the server stores client files directly to tape until migration completes, resulting in slower performance. If you set the high-migration threshold too low, migration runs more frequently and can interfere with other operations. Keeping the high-migration threshold at a single value means that migration processing could start at any time of day, whenever that threshold is exceeded. You can control when migration occurs by using administrative command schedules to change the threshold. For example, set the high-migration threshold to 95% during the night when clients run their backup operations. Lower the highmigration threshold to 50% during the time of day when you want migration to occur. By scheduling when migration occurs, you can choose a time when your tape drives and mount operators are available for the operation.
Choosing the Low-Migration Threshold

To choose the low-migration threshold, consider: The amount of free disk storage space needed for normal daily processing. If you have disk space to spare, you can keep more data on the disk (a larger low threshold). If clients' daily backups are enough to fill the disk space every day, you may need to empty the disk (a smaller low threshold). If your disk space is limited, try setting the threshold so that migration frees enough space for the pool to handle the amount of client data that is typically stored every day. Migration then runs about every day, or you can force it to run every day by lowering the high-migration threshold at a time you choose. You may also want to identify clients that are transferring large amounts of data daily. For these clients, you may want to set up policy (a new copy group or a new policy domain) so that their data is stored directly to tape. Using a separate policy in this way can optimize the use of disk for the majority of clients. Whether you use cache on disk storage pools to improve how quickly some files are retrieved. If you use cache, you can set the low threshold lower, yet still maintain faster retrieval for some data. Migrated data remains cached on the disk until new client data pushes the data off the disk. Using cache requires more disk space for the database, however, and can slow backup and archive operations that use the storage pool. If you do not use cache, you may want to keep the low threshold at a higher number so that more data stays on the disk. How frequently you want migration to occur, based on the availability of sequential access storage devices and mount operators. The larger the low threshold, the shorter time that a migration process runs (because there is less data to migrate). But if the pool refills quickly, then migration occurs more frequently. The smaller the low threshold, the longer time that a migration process runs, but the process runs less frequently. You may need to balance the costs of larger disk storage pools with the costs of running migration (drives, tapes, and either operators or automated libraries). Whether you are using collocation on the next storage pool. When you use collocation, the server attempts to store data for different clients or client file spaces on separate tapes, even for clients with small amounts of data. You may want to set the low threshold to keep more data on disk, to avoid having many tapes used by clients with only small amounts of data.

Keeping Files in a Storage Pool

For some applications, you may want to ensure that files remain in the storage pool where they were initially stored by the server for a certain period of time. For example, you may have backups of monthly summary data that you want to keep in your disk storage pool for quicker access until the data is 30 days old. After the 30 days, the server can then move the file off into a tape storage pool. You can delay migration of files for a specified number of days. The number of days is counted from the day that a file was stored in the storage pool or retrieved by a client, whichever is more recent. You can set the migration delay separately for each storage pool. When you set the delay to zero, the server can migrate any file from the storage pool, regardless of how short a time the file has been in the storage pool. When you set the delay to greater than zero, the server checks whether the file has been in the storage pool for at least the migration delay period before migrating the file.

Note: If you want the number of days for migration delay to be counted based only on when a file was stored and not when it was retrieved, use the NORETRIEVEDATE server option. See Administrator's Reference for more information on the server
option. If you set migration delay for a pool, you need to decide what is more important: either ensuring that files stay in the storage pool for the migration delay period, or ensuring that there is enough space in the storage pool for new files. For each storage pool that has a migration delay set, you can choose what happens as the server tries to move enough data out of the storage pool to reach the low migration threshold. If the server cannot reach the low migration threshold by moving only files that have been stored longer than the migration delay, you can choose one of the following:

http://www.urz.uni-heidelberg.de/UnixCluster/Hinweise/Hilfe/System/Adsm/ibmdoc.tsm41/html/aix/guide/anragd87.htm

3/5

Tivoli Storage Manager for AIX Administrator's Guide

6/14/2013

Allow the server to move files out of the storage pool even if they have not been in the pool for the migration delay (MIGCONTINUE=YES). This is the default. Allowing migration to continue ensures that space is made available in the storage pool for new files that need to be stored there. Have the server stop migration without reaching the low migration threshold (MIGCONTINUE=NO). Stopping migration ensures that files remain in the storage pool for the time you specified with the migration delay. The administrator must ensure that there is always enough space available in the storage pool to hold the data for the required number of days. If you allow more than one migration process for the storage pool and allow the server to move files that do not satisfy the migration delay time (MIGCONTINUE=YES), some files that do not satisfy the migration delay time may be migrated unnecessarily. As one process migrates files that satisfy the migration delay time, a second process could begin migrating files that do not satisfy the migration delay time to meet the low migration threshold. The first process that is still migrating files that satisfy the migration delay time might have, by itself, caused the storage pool to meet the low migration threshold.

Minimizing Access Time to Migrated Files

Caching is a method of minimizing access time to files on disk storage, even if the server has migrated files to a tape storage pool. However, cached files are removed from disk when the space they occupy is required. The file then must be obtained from the storage pool to which it was migrated.

Note: The use of cache has some disadvantages. See Using Cache on Disk Storage Pools.
To ensure that files remain on disk storage and do not migrate to other storage pools, use one of the following methods: Do not define the next storage pool. A disadvantage of using this method is that if the file exceeds the space available in the storage pool, the operation to store the file fails. Set the high-migration threshold to 100%. When you set the high migration threshold to 100%, files will not migrate at all. You can still define the next storage pool in the storage hierarchy, and set the maximum file size so that large files are stored in the next storage pool in the hierarchy. A disadvantage of setting the high threshold to 100% is that once the pool becomes full, client files are stored directly to tape instead of to disk. Performance may be affected as a result.

Migration for Sequential Access Storage Pools

You can set up migration thresholds for sequential access storage pools. However, you probably will not want the server to perform this type of migration on a regular basis. An operation such as tape-to-tape migration has limited benefits compared to disk-to-tape migration, and requires at least two tape drives. Migrating data from one sequential access storage pool to another may be appropriate in some cases, for example, when you install a tape drive that uses a different type of tape and want to move data to that tape. To control the migration process, you can set migration thresholds and a migration delay for each storage pool.

Note: You can migrate data from a sequential access storage pool only to another sequential access storage pool. You cannot
migrate data from a sequential access storage pool to a disk storage pool. If you need to move data from a sequential access storage pool to a disk storage pool, use the MOVE DATA command. See Moving Files from One Volume to Another Volume.

How Tivoli Storage Manager Migrates Data from Sequential Access Storage Pools
The server begins the migration process when the number of volumes containing data as a percentage of the total volumes in the storage pool reaches the high migration threshold. The server migrates data from sequential storage pools by volume, to minimize the number of mounts for volumes. The server performs the following processing for migration: 1. The server first reclaims volumes that have exceeded the reclamation threshold. Reclamation is a server process of consolidating data from several volumes onto one volume. (See Reclaiming Space in Sequential Access Storage Pools.) 2. After reclamation processing, the server compares the space used in the storage pool to the low migration threshold. 3. If the space used is now below the low migration threshold, the server stops processing. If the space used is still above the low migration threshold, the server determines which volume is the least recently referenced volume. 4. If the number of days since data was written is greater than the migration delay, the server migrates the volume. Otherwise, the server does not migrate this volume. 5. The server repeats steps 3 and 4 until the storage pool reaches the low migration threshold. Because migration delay can prevent volumes from being migrated, the server can migrate data from all eligible volumes yet still find that the storage pool is above the low migration threshold. If you set migration delay for a pool, you need to decide what is more important: either ensuring that data stays in the storage pool for as long as the migration delay, or ensuring there is enough space in the storage pool for new data. For each storage pool that has a migration delay set, you can choose what happens as the server tries to move enough data out of the storage pool to reach the low migration threshold. If the server cannot reach the low migration threshold by migrating only volumes that meet the migration delay requirement, you can choose one of the following:

http://www.urz.uni-heidelberg.de/UnixCluster/Hinweise/Hilfe/System/Adsm/ibmdoc.tsm41/html/aix/guide/anragd87.htm

4/5

Tivoli Storage Manager for AIX Administrator's Guide

6/14/2013

Allow the server to migrate volumes from the storage pool even if they do not meet the migration delay criteria (MIGCONTINUE=YES). This is the default. Allowing migration to continue ensures that space is made available in the storage pool for new files that need to be stored there. Have the server stop migration without reaching the low migration threshold (MIGCONTINUE=NO). Stopping migration ensures that volumes are not migrated for the time you specified with the migration delay. The administrator must ensure that there is always enough space available in the storage pool to hold the data for the required number of days.

Selecting Migration Criteria for Sequential Access Storage Pools

When defining migration criteria for sequential access storage pools, consider: The capacity of the volumes in the storage pool The time required to migrate data to the next storage pool The speed of the devices that the storage pool uses The time required to mount media, such as tape volumes, into drives Whether operator presence is required If you decide to migrate data from one sequential access storage pool to another, ensure that: Two drives (mount points) are available, one in each storage pool. The access mode for the next storage pool in the storage hierarchy is set to read/write. For information about setting an access mode for sequential access storage pools, see Defining or Updating Primary Storage Pools. Collocation is set the same in both storage pools. For example, if collocation is set to yes in the first storage pool, then collocation should be set to yes in the next storage pool. When you enable collocation for a storage pool, the server attempts to keep all files belonging to a client node or a client file space on a minimal number of volumes. For information about collocation for sequential access storage pools, see Keeping a Client's Files Together: Collocation. You have sufficient staff available to handle any necessary media mount and dismount operations. More mount operations occur because the server attempts to reclaim space from sequential access storage pool volumes before it migrates files to the next storage pool. If you want to limit migration from a sequential access storage pool to another storage pool, set the high-migration threshold to a high percentage, such as 95%. For information about setting a reclamation threshold for tape storage pools, see Reclaiming Space in Sequential Access Storage Pools. There is no straightforward way to selectively migrate data for a specific node from one sequential storage pool to another. If you know the volumes on which a particular node's data is stored, you can use the MOVE DATA command to move all files from selected volumes to the new storage pool. See Moving Files from One Volume to Another Volume.

Migration and Copy Storage Pools

Copy storage pools are not part of the hierarchy for migration. Files are not migrated to or from copy storage pools. The only way to store files in copy storage pools is by backing up primary storage pools (the BACKUP STGPOOL command). Migration of files between primary storage pools does not affect copy storage pool files. Copy storage pool files do not move when primary storage pool files move. For example, suppose a copy of a file is made while it is in a disk storage pool. The file then migrates to a primary tape storage pool. If you then back up the primary tape storage pool to the same copy storage pool, a new copy of the file is not needed. The server knows it already has a valid copy of the file.

[ Top of Page | Previous Page | Next Page | Table of Contents | Index ]

http://www.urz.uni-heidelberg.de/UnixCluster/Hinweise/Hilfe/System/Adsm/ibmdoc.tsm41/html/aix/guide/anragd87.htm

5/5