Veritas Netbackup 6.X Troubleshooting Techniques For Unix (Lessons)

Veritas NetBackup 6.
x
Troubleshooting Techniques
for UNIX (Lessons)
100-002372-A
COURSE AND LAB Copyright © 2008 Symantec Corporation. All rights reserved. Symantec,
DEVELOPERS the Symantec Logo, and Veritas are trademarks or registered trademarks of
Lisa Goldring Symantec Corporation or its affiliates in the U.S. and other countries.
Paul Johnston Other names may be trademarks of their respective owners.
Sandy Tipper THIS PUBLICATION IS PROVIDED "AS IS" AND ALL EXPRESS OR
ADVISORY BOARD IMPLIED CONDITIONS, REPRESENTATIONS AND WARRANTIES,
MEMBERS INCLUDING ANY IMPLIED WARRANTY OF MERCHANTABILITY,
Albrecht Scriba FITNESS FOR A PARTICULAR PURPOSE OR NON-
Carly Jacques INFRINGEMENT, ARE DISCLAIMED, EXCEPT TO THE EXTENT
Chris Amidei THAT SUCH DISCLAIMERS ARE HELD TO BE LEGALLY
Christian Rabanus INVALID. SYMANTEC CORPORATION SHALL NOT BE LIABLE
FOR INCIDENTAL OR CONSEQUENTIAL DAMAGES IN
Dave Little
CONNECTION WITH THE FURNISHING, PERFORMANCE, OR USE
David Rogers OF THIS PUBLICATION. THE INFORMATION CONTAINED
Don Anderson HEREIN IS SUBJECT TO CHANGE WITHOUT NOTICE.
Freddie Gilyard
No part of the contents of this book may be reproduced or transmitted in
Graeme Gofton any form or by any means without the written permission of the publisher.
Joseph Gallagher
Veritas NetBackup 6.x Troubleshooting Techniques
Kleber Saldanha
Mauricio Julian Paredes Symantec Corporation
Mike Williams 20330 Stevens Creek Blvd.
Cupertino, CA 95014
Ray Katos
Robert Owen http://www.symantec.com
Roy Freeman
Satoko Saito
Stephen Williams
Sue Rich
Suzanne Trigg
Tomer Gurantz
TECHNICAL REVIEWERS
AND CONTRIBUTORS
Rich Armstrong
James Dandeneau
Wm. M. Drazkowski
Scott Frohreich
John Gerhardson
Kevin Holtz
Dave Little
Steve Schwarze
Debbie Wilmot
Table of Contents
Course Introduction
Lesson 1: Ensuring a Successful Upgrade

Topic 1: Preparing for an Upgrade.......................................................................... 1-3
Topic 2: Upgrading from NetBackup 5.1 to NetBackup 6.x................................... 1-16
Lesson 2: Troubleshooting Methods and Tools

Topic 1: Troubleshooting Situations and Methodology ........................................... 2-3
Topic 2: Gathering Information Using the nbsupport Utility ................................ 2-9
Topic 3: Review of Troubleshooting Tools ............................................................ 2-11
Lesson 3: NetBackup Process Flow

Topic 1: Core Process Functions ............................................................................ 3-3
Topic 2: Process Communications........................................................................ 3-18
Topic 3: Backup Process Flow.............................................................................. 3-22
Topic 4: Restore Process Flow ............................................................................. 3-26
Lesson 4: Using Debug Logs

Topic 1: Log Management ...................................................................................... 4-3
Topic 2: Capturing Debug Logs ............................................................................ 4-17
Topic 3: Viewing Debug Logs ............................................................................... 4-24
Lesson 5: Understanding the NetBackup Database

Topic 1: The NetBackup Relational Database (NBDB)........................................... 5-3
Topic 2: The Enterprise Media Manager (EMM) Domain...................................... 5-10
Topic 3: Client Backup Process Flow Through the EMM Server .......................... 5-23
Topic 4: Catalog Backup and Recovery................................................................ 5-30
Topic 5: Maintaining the NBDB ............................................................................. 5-54
Lesson 6: Troubleshooting Devices

Topic 1: Preliminary Device Troubleshooting.......................................................... 6-3
Topic 2: Environmental Device Troubleshooting Approach .................................... 6-5
Topic 3: NetBackup Device Troubleshooting Approach........................................ 6-20
Lesson 7: Troubleshooting Media

Topic 1: Media-Related Status Codes .................................................................... 7-3
Topic 2: Troubleshooting Frozen and Suspended Media ..................................... 7-17
Lesson 8: Troubleshooting Network Issues

Topic 1: NetBackup and TCP/IP ............................................................................. 8-3
Topic 2: Physical Network Failures ......................................................................... 8-6
Topic 3: NetBackup Configuration Errors................................................................ 8-9
Topic 4: Isolating Network Errors .......................................................................... 8-20
Table of Contents i
Copyright © 2008 Symantec Corporation. All rights reserved.
Lesson 9: Troubleshooting Performance Issues
Topic 1: NetBackup Performance Overview ........................................................... 9-3
Topic 2: Isolating Bottlenecks ................................................................................. 9-9
Topic 3: Addressing Bottlenecks........................................................................... 9-19
Glossary
Index
ii Veritas NetBackup 6.x Troubleshooting Techniques

Course Introduction
Course Objectives
After completing this course, you will be able to:

• Detect problems using tools, such as the NetBackup
reports, nbsupport, and administrative commands.
• Identify the functions of processes and databases that
drive NetBackup operations.
• Troubleshoot a variety of media, device, and network-
related problems.
• Administer backup and restore operations at a higher
rate of success.
Course Objectives
After completing this course, you will be able to:
• Upgrade from NetBackup 5.x to NetBackup 6.0 or NetBackup 6.5.
• Detect problems using tools, such as the NetBackup reports, nbsupport, and
administrative commands.
• Understand the functions of services, daemons, and processes that drive
NetBackup operations.
• View and manage NetBackup debug logs.
• Back up, recover, and troubleshoot the EMM database.
• Rapidly and accurately isolate the root cause of a backup failure.
• Troubleshoot a wide variety of media, device, and network-related problems.
• Isolate and address problems with backup performance.
• Run backups and restores with a higher rate of success and efficiency.
Intro–2 Veritas NetBackup 6.x Troubleshooting Techniques

Course Prerequisites
Before attending this course, you should have:

• Knowledge of backup, restore, and shared storage
concepts
• Completed the NetBackup 6.x Administration course, or
equivalent experience
• This includes working knowledge of:
– Storage units
– Media management
– Policy configuration
– NetBackup architecture
• Two years experience in Windows or UNIX system
administration
Course Prerequisites
Before attending this course, you should have:
• Knowledge of backup, restore, and shared storage concepts
• Completed the NetBackup 6.x Administration course, or equivalent experience
This includes working knowledge of:
– Storage units and devices
– Media management
– Policy attributes and schedules
– Restore and import operations
– NetBackup reports
– Common NetBackup command-line utilities, such as bpexpdate and
available_media
• Two years experience in Windows or UNIX system administration, including
knowledge of:
– System Logs
– Tape device configuration
– Networking
– File management
Course Introduction Intro–3

Additional Course Prerequisites
Additionally, you should know:

• How to start and stop NetBackup services or daemons
and processes
• How to back up and restore data using both server and
user-directed methods
• How to restore data using a different media server
• Backup and restore strategies
• Common tuning options
Additional Course Prerequisites

Specifically, you should know how to:
• Start and stop NetBackup daemons or services and processes.
• Back up and restore using both the server and user-directed methods.
• Restore data using a different media server, given a scenario with a media
server failure.
• Implement backup and restore (disaster recovery) strategies, such as the
principles of offsite tape storage, scheduling concepts, difference between full
and incremental backups, data storage retention requirements, and factors that
affect speed of backup and restore.
• Describe tuning options and where they are implemented, such as data buffer
size, multiplexing concepts, and multistreaming concepts.

Synopsis of the Lessons in the
Course
• Course Introduction
• Lesson 1: Ensuring a Successful Upgrade
• Lesson 2: Troubleshooting Methods and
Tools
• Lesson 3: NetBackup Process Flow
• Lesson 4: Using Debug Logs
• Lesson 5: Understanding the NetBackup
Database
• Lesson 6: Troubleshooting Devices
• Lesson 7: Troubleshooting Media
• Lesson 8: Troubleshooting Network Issues
• Lesson 9: Troubleshooting Performance
Issues
Synopsis of the Lessons in the Course

The lessons in this course include:
In this lesson, you will learn about the upgrade process from NetBackup 5.x to
NetBackup 6.0 and NetBackup 6.5. You will learn about the prerequisites and
best practices before performing an upgrade, the mechanics of the upgrade
process, and common problems associated with upgrade installations.
• Lesson 2: Troubleshooting Methods and Tools
In this lesson, you will learn about NetBackup error detection tools. You will
review the NetBackup reports and nbsupport utility to collect error
detection information. You will learn how to use e-mail notification and
NetBackup Operations Manager (NOM) to monitor NetBackup or report on
problems.
In this lesson, you will review the low-level functions of the master server,
media server, client, and the Enterprise Media Manager (EMM). You will
identify the functions of key services and processes involved in backup and
restore jobs.
In this lesson, you will review how to enable Unified and Legacy debug
logging. You will also learn how to view and manage these log files.

• Lesson 5: Understanding the NetBackup Database
In this lesson, you will learn the concepts of the NetBackup Enterprise Media
Manager (EMM) Database, and how to manage and maintain it. You will learn
about the objects stored in the database, the mechanics behind retrieving and
processing this information, and disaster recovery procedures for the database.
In this lesson, you will learn how to verify that all hardware and software
components are compatible with NetBackup. You will verify tape and pass-
through drivers, operating system device access, manual manipulation of the
robot, mounts, unmounts, and device replacement. You will also use
NetBackup tools and techniques to verify NetBackup device access and
configuration.
In this lesson, you will learn methods to troubleshoot media-related problems
with backups. You will examine common error codes associated with media
problems and how to troubleshoot them. You will learn about conditions that
prevent media from being used for a backup, and where to find information to
isolate a media problem.
• Lesson 8: Troubleshooting Network Issues
In this lesson, you will learn how to detect and correct physical network errors,
NetBackup and network configuration errors, and other errors, such as port
shortages, multiple network interface issues, and firewall configuration issues.
• Lesson 9: Troubleshooting Performance Issues
In this lesson, you will learn how to isolate and address poorly performing
backup and restore operations. You will learn best practice methods for
performance measuring and testing. You will also learn the underlying cause
for many performance problems, and some common methods for addressing
poor performance.

Typographic Conventions Used in This Course
The following tables describe the typographic conventions used in this course.
Typographic Conventions in Text and Commands
Convention Element Examples

Courier New, Command input, To display the robot and drive configuration:
bold both syntax and tpconfig -d
examples
To display disk information:
vxdisk -o alldgs list
Courier New, • Command output In the output:
plain • Command protocol_minimum: 40
names, directory protocol_maximum: 60
names, file protocol_current: 0
names, path Locate the altnames directory.
names, user
Go to http://www.symantec.com.
names,
passwords, URLs Enter the value 300.
when used within Log on as user1.
regular text
paragraphs.
Courier New, Variables in To install the media server:
Italic, bold or command syntax, /cdrom_directory/install
plain and examples:
To access a manual page:
• Variables in
man command_name
command input
are Italic, plain. To display detailed information for a disk:
• Variables in vxdisk -g disk_group list
command output disk_name
are Italic, bold.
Typographic Conventions in Graphical User Interface Descriptions
Convention Element Examples

Arrow Menu navigation paths Select File—>Save.
Initial capitalization Buttons, menus, windows, Select the Next button.
options, and other interface Open the Task Status
elements window.
Remove the checkmark
from the Print File check
box.
Quotation marks Interface elements with Select the “Include
long names subvolumes in object view
window” check box.

Lesson 1
Ensuring a Successful Upgrade
Lesson Introduction
Tools
Database
• Lesson 8: Troubleshooting Network
Issues
Issues
Lesson Topics and Objectives
After completing this lesson, you will be

Topic
able to:
Topic 1: Preparing for an Identify the prerequisites and considerations
Upgrade for upgrading to NetBackup 6.x.
Topic 2: Upgrading from List the general procedure for upgrading

NetBackup 5.1 to NetBackup from NetBackup 5.1 to NetBackup 6.x.
6.x
1–2 Veritas NetBackup 6.x Troubleshooting Techniques

Upgrade Difficulties
A successful NetBackup upgrade depends upon:
1
• Reading available documentation
– Release notes
– Installation manuals
– Latest TechNotes
• Verifying prerequisites
– Minimum required disk space
– Minimum required memory
– Minimum previous NetBackup version
• Deployment planning
– Rollout and staffing schedule
– Complete catalog backup of the previous installation
– A plan to back out
– Procedure to remove and install the new version, including the latest maintenance pack (MP)
or Release Update
• Postinstallation testing
– Catalog backup of the new installation
– Sample client backups
– Sample client restores
Topic 1: Preparing for an Upgrade

After completing this topic, you will be able to identify the prerequisite
considerations for upgrading to NetBackup 6.x.
Upgrade Difficulties
An automated upgrade of a complex or custom environment may be difficult. If
the Global Device Database and the Volume Pool Databases are located centrally
on the same host prior to the upgrade, merging these catalogs into the Enterprise
Media Manager (EMM) Database is simplified. If you are using supported Disk
Staging Storage Units (DSSUs), the upgrade is quite direct. Difficulties arise when
custom disk staging is implemented, involving Disk Storage Units that use cron
jobs to initiate the duplication of backup images to media. When upgrading from
NetBackup 5.x to 6.x, install the latest 6.x maintenance pack or release update
before running nbpushdata -add to merge legacy 5.x catalog entries into the
EMM database properly.
In a large environment, it is impractical to upgrade every NetBackup server and
client simultaneously, so mixed versions must coexist during the enterprise-wide
rollout. The upgrade process supports a rolling upgrade for the days or weeks it
may take, as long as the EMM database knows that previous-version media servers
are still in active duty (by using nbpushdata -modify_5x_hosts).
Lesson 1 Ensuring a Successful Upgrade 1–3

Upgrade Prerequisites
Database Resolve any issues before attempting an

consistency upgrade.
Failing backups Resolve any problems with failing

in 5.x backups before attempting an upgrade.
No NBAR or Remove NBAR and GDM before starting

GDM the upgrade.
Capture global and client attribute

Global and
settings prior to upgrading to NetBackup
client settings
6.5.
6
Upgrade Prerequisites
Before performing an upgrade, ensure that:
• There are no 5.x database inconsistencies.
NetBackup commands and the NetBackup Consistency Check (NBCC) utility
are available to perform database checks.
• Your current 5.x backups run correctly.
If backups are failing in NetBackup 5.x, they will also fail after an upgrade to
NetBackup 6.x.
• NetBackup Advanced Reporter (NBAR) and Global Data Manager (BDM)
have been removed.
NetBackup Advanced Reporter and Global Data Manager are not compatible
with NetBackup 6.0 and must be removed from all servers before you can
upgrade. NOM is not an upgrade but provides new functionality that replaces
their roles. See TechNote 281578, How to preserve Veritas NetBackup
Advanced Reporter (NBAR) data before upgrading to NetBackup 6.0.
• Global and client settings have been captured to a text file.
An upgrade to NetBackup 6.5 may lose some customized configuration
settings for global and client attributes, resetting them back to the defaults.
Therefore, you should save these settings before the upgrade as a safety
precaution.

To capture the global and client attribute settings, use the bpconfig and the
bpclient commands and send their output to a text file. The bpconfig
and the bpclient commands are located in /usr/openv/netbackup/
bin/admincmd (UNIX) or
install_path\Veritas\NetBackup\bin\admincmd (Windows).
1
First run the bpconfig -L > global_attributes.txt command:
Then run the bpclient -All -L > client_attributes.txt
command.
After these two text files are created, move them to a new location outside of
the NetBackup directory structure. Retain them until after the upgrade is
completed and the environment is running correctly.
For more information about capturing global and client attribute settings prior
to a NetBackup 6.5 upgrade, see TechNote 294899.

Consistency Issues or Catalog
Inconsistencies
Resolve any known preexisting problems.
If you want to … Then use …
Scan the catalog and report bpdbm -consistency 2

inconsistencies,
Reclaim disk space by deleting bpimage -cleanup –allclients

expired images,
Discard completed jobs from the bpdbjobs -clean

jobs database,
Consistency Issues or Catalog Inconsistencies

First and foremost, resolve any known preexisting problems. Then, ensure that
there are no database inconsistencies. The commands to run are:
• bpdbm -consistency 2
This command scans the catalog and reports inconsistencies. In some versions,
this command also moves corrupt images into the db.corrupt directory,
and in some versions, it also freezes affected media. See TechNote 282017, In
certain versions of Veritas NetBackup, the dbcheck or bpdbm
-consistency commands may move corrupt images into a directory named
db.corrupt, and TechNote 265806, How to prevent backups and restores
from starting while running the bpdbm -consistency command or the
NetBackup Consistency Checker, for details.
• bpimage -cleanup -allclients
This command reclaims disk space by deleting expired images from the
catalog. See TechNote 236274, The Veritas NetBackup image database does
not clean up expired images on the expiration date, for further details.
• bpdbjobs -clean
This command discards completed jobs from the jobs database. A time period
parameter, such as -keep_hours 3, is required.

The NetBackup Consistency Checker
(NBCC)
1
1. Open a case
with support.
2. Download
and run NBCC. Your environment is
ready for the upgrade.
3. Send the
output to Run fix scripts.
support.
Contact consulting to
help with the upgrade.
The NetBackup Consistency Checker (NBCC)

At least two weeks before the upgrade, perform the following:
1 Contact Symantec Technical Support and open a case with them.
2 Follow the instructions from Technical Support to download and run the
NetBackup Consistency Checker (NBCC).
3 Send the output from the NBCC to Technical Support, who will analyze your
output, providing the following possible results:
– Your environment is OK, and ready for you to begin the upgrade.
– Your environment is almost ready for you to begin the upgrade. Run the
recommended fixes before starting the upgrade.
– Preparing your environment for upgrade is complicated. Contact the
Symantec consulting organization to assist you with your upgrade.
See TechNote 267965, How to install and use the Symantec Veritas NetBackup
Catalog Consistency Utility (NBCC), for more information.
Note: It is strongly recommended that you run NBCC before performing the
upgrade, and that you obtain assistance during the upgrade.

Inventorying the Servers
Server Inventory Checklist:

;List all master and media servers.
;Identify the NetBackup version for each server.
;Identify the Global Device Database host:
tpautoconf -get_gdbhost
;Identify the Volume Database host:
tpconfig –d
;Validate the SERVER entries in the bp.conf file (UNIX) or
the Registry (Windows).
Inventorying the Servers

Create a document that lists the master and all media servers in the configuration.
If you are not upgrading all media servers to NetBackup 6.x, identify the version of
NetBackup that will be running on each media server. In particular, identify the
Global Device Database host server and the Volume Database host servers. To do
this:
• Run the tpautoconf -get_gdbhost command on each media server to
determine the Global Device Database host.
• Run the tpconfig -d command on each media server to determine the
corresponding Volume Database host.
Ensure that all SERVER entries in the bp.conf file (UNIX) or in the servers list
in the Registry (Windows) are valid.

Network Issues
• Ensure that there are no connectivity issues.
1
• Hostnames or fully qualified domain names (FQDNs) and reverse
name lookup are required.
train5 train5.symantec.com
Short Name FQDN
IP Address
192.168.27.105
10
Network Issues
Ensure that there are no connectivity issues between the servers. Not only must
each server be reachable, but name resolution must be consistent and include
hostnames or fully qualified domain names (FQDNs) and reverse name lookup.
Resolve any DNS problems before you proceed.
Alternatively, use a hosts file on every master and media server containing the
IP address, FQDN, and short name of every master and media server to prevent
any DNS issues.
Hint: If it takes a long time to restart NetBackup on the master server or to run the
nbpushdata or vmoprcmd commands, it is possible that a name resolution
issue is causing delays and eventual timeouts. Resolve the name resolution issue,
and, if necessary as a workaround, remove all SERVER entries from the bp.conf
file (UNIX) or from the Registry (Windows) for each unreachable media server.
When you are ready to reintegrate these servers, upgrade their software, if
applicable, add their SERVER entries, and run the appropriate nbpushdata
commands.
If you have a firewall between any of the media servers and the master server,
ensure that all required ports are open.
Note: Network Address Translation (NAT) is not supported.

Maintenance Packs or Release Updates
Always check the http://entsupport.symantec.com Symantec Support
Web site for the latest maintenance packs (MPs) or Release Updates. Maintenance
packs are cumulative, so you only need to apply the highest version. Always read
the associated notes to determine whether there are outstanding issues, or whether
the pack addresses your current problems.
When you are upgrading to version 6.x from 5.x, apply the latest MP or Release
Update immediately after installing the NetBackup software, before restarting the
services.

Clusters
Virtual
1
NetBackup
Server
NetBackup
Clients
12
Clusters
The virtual name and IP address of the cluster is the server name used by
NetBackup, so the actual node running the software is not important. Only the
active node communicates with other NetBackup servers (such as for the
nbpushdata -add command).
Ensure that any required patches to the cluster software are applied prior to
upgrading to NetBackup 6.x. See TechNote 278307, Veritas Storage Foundation
4.1 and 4.2 HA for Windows - Patch for Enterprise Agent NetBackup to use
AgentFile and AgentDirectory attributes, as an example for VCS 4.1 or 4.2 on
Windows.
After you have run the upgrade procedure, you must run nbpushdata on the
active node of a cluster. Running nbpushdata pushes the data to the EMM
database from the existing shared database files and from the local database files
on the inactive nodes. To do this, nbpushdata obtains a list of all of the nodes in
the cluster, uses bpcd to obtain a copy of each local database file from all inactive
nodes, and stages these files on the active node. The data from the staged files is
then pushed to the EMM database.
Note for VCS Clusters

For Windows 2000 environments, perform NetBackup upgrade installations from
the system console, not from a Remote Terminal Services session.

Mixed Environments
Clustered nodes must run consistent

Clusters
OSs, SP levels, and NBU versions.
Shared Storage Mixed versions are not recommended,

Option (SSO) but they are supported.
Version 5.x nbpushdata -modify_5x_hosts

Media Servers
13
Mixed Environments
Mixed environments are supported as a migration tool, but they are not meant to be
permanent.
Clusters
Mixing server versions is not supported in a clustered environment. All clustered
nodes must run the same operating system, service pack level, and version of
NetBackup.
Shared Storage Option (SSO)

Sharing drives between NetBackup 6.x and NetBackup 5.x media servers is
supported only as a migration tool. In version 6.x the EMM server is responsible
for granting an SSO resource, but in version 5.x the media server itself has more
involvement in SSO tape drive selection. This difference may cause slower drive
selection in larger environments. For best results, upgrade all 5.x SSO media
servers as soon as possible.
Version 5 Media Servers

If your migration plan requires that some media servers temporarily remain at
version 5.x, run the nbpushdata -modify_5x_hosts command on the
master servers that control these media servers.

Supported Server Upgrades to
NetBackup 6.5
1
NBU 6.0
NBU 5.0 MP4 NBU 6.5
NBU 5.1
NBU 5.0 (pre-

NBU 4.5
MP4)
114
Supported Server Upgrades to NetBackup 6.5

NetBackup only supports server upgrades from NetBackup 5.0 MP4 (or later),
NetBackup 5.1, or NetBackup 6.0 to NetBackup 6.5.
If you currently have a pre-NetBackup 5.0 MP4 version installed, perform an
intermediate upgrade to NetBackup 5.0 MP4+ or 5.1 first, and then upgrade to
NetBackup 6.5.
The NetBackup 6.5 Upgrade Portal at
http://seer.entsupport.symantec.com/docs/290185.htm
lists many documents and resources to help you perform a successful upgrade.
For example, the NetBackup 5.x to 6.5 Upgrade Guide at
http://seer.entsupport.symantec.com/docs/290141.htm
includes:
• Information on where to locate minimum system and environment
requirements
• A checklist of items and procedures that should be verified and backed up prior
to beginning an upgrade
• A detailed walk-through of the upgrade process, which provides the
appropriate upgrade information in existing NetBackup documents and
published TechNotes

Upgrading to Mixed Version
Environments
6.5
Media 6.0 6.5
6.0
6.5
Master 6.0
6.0
Media 6.0
6.0
5.1
Media 5.1
5.1
5.0
MP4
Media 5.0 5.0
MP4
MP4
6.5
EMM 6.0
5.0
MP4
115
2
3
4
Upgrading to Mixed Version Environments

NetBackup has supported mixed patch-level versions for many releases. Mixed
patch-level support enables NetBackup servers to run a mixture of NetBackup
major releases and patches in the same environment. For example, this release of
NetBackup allows everything in the following customer environment to be
managed by one master server:
• One or more master servers running NetBackup 6.5 (The EMM server and all
its master servers must also run NetBackup 6.5.)
• A media server running NetBackup 6.5
• Any other media server running either NetBackup 6.0, 5.1, or 5.0 MP4 or later
• Any number of clients running NetBackup 6.0, 5.1, or 5.0 MP4 or later
The general rules for a mixed-server environment are as follows:
• In a mixed environment such as this, the master server and the EMM server
must be running the latest version of NetBackup in use in that configuration.
• A master server can interoperate with a media server that is running a level of
NetBackup from one major release earlier.
• Back-level servers (those servers that are running NetBackup 5.x) cannot be
used as a Volume Database host, global device database host, or vmd/DA host.
These capabilities are all on the EMM database host as part of the EMM server.
Always upgrade NetBackup 5.x servers that are performing these roles first.
Only then can media servers, and finally client machines, be upgraded.
• Media server backward compatibility extends only one major release prior to
the master server. No media server may have a numerically later version than

the master server. (The media servers must be running equal or earlier levels of
NetBackup.)
• Client backward compatibility extends only one major release prior to the
master server. This includes database clients.
• All components (the master server, media server, client, console, and agent) on
1
an individual system must run the same version.
• Backup images created under an older version of NetBackup are always
recoverable with a newer version of NetBackup.
Upgrade Guidelines
Follow these guidelines when upgrading to mixed-version environments:
• No client can be greater than its media server.
• No media server can be greater than its master server.
• All of the master servers and the EMM server must run 6.5.

Upgrade Overview
• Upgrade the NetBackup software in the following order:

1. Master servers
2. Media servers
3. Remote administration consoles
4. Clients
5. Add-on products
• Run the nbpushdata command in the following order:
1. 5.x Global Device Database Host
2. Master servers
3. Volume Database Hosts
4. Media servers
17
Topic 2: Upgrading from NetBackup 5.1 to NetBackup 6.x

After completing this topic, you will be able to list the general procedure for
upgrading from NetBackup 5.x to 6.x.
Upgrade Overview
When upgrading from one version of NetBackup to another, install the NetBackup
software in the order shown on the slide.
Note: This is not the sequence you use to run the nbpushdata command.
The nbpushdata command moves data from your current database files (a
subset of the NetBackup catalog) into a newly created EMM database. The slide
shows the sequence of when and on what systems you run the nbpushdata
command.
For more information on upgrading to NetBackup 6.x and running nbpushdata
refer to the Veritas NetBackup Upgrade Portal at
http://seer.entsupport.symantec.com/docs/290185.htm.

General NetBackup 6.x Upgrade
Process
1
1. Prepare,
Back up the 2. Populate
upgrade, patch
NBU catalogs. the EMM DB.
Master Servers.
3. Upgrade the 4. Upgrade 5. Prepare,

6. Populate
NBU Remote any add-on upgrade, patch
the EMM DB.
Admin console. components. Media servers.
8. Activate the
7. Upgrade the Back up the
policies and run
clients. NBU catalogs.
a test backup.
7
8
9
1
2
3
4
5
6
18
General NetBackup 6.x Upgrade Process

This slide shows a general process for upgrading your environment to NetBackup
6.x.
You can upgrade directly to NetBackup 6.x from version 5.1 (GA) or 5.0
(Maintenance Pack 4 or greater). If you have an earlier version you must first
upgrade to a supported 5.x version throughout the enterprise.
You can upgrade from NetBackup Server to NetBackup Enterprise Server with the
appropriate license, but you cannot convert a media server to a master server.
As with earlier versions of NetBackup, you must upgrade master servers first, then
their media servers, and then their clients. However, in version 6.x, you must
populate the EMM database, transferring the catalog data from former version 5.x
master servers to the EMM server before you upgrade your media servers. The
original Global Device Database Host becomes the EMM server and maintains the
EMM database. The upgrade is not complete until the EMM database is populated
using the nbpushdata command on each upgraded machine.
Each of the steps shown in the slide is discussed in more detail on the following
pages.

Step 1: Prepare, Upgrade, and Patch
the Master Servers
1. Back up the catalogs.
2. Deactivate all policies.
3. In a clustered environment, take NetBackup offline.
4. On Solaris only, uninstall the old version of NetBackup.
5. Upgrade the software:
a. Stop all services.
b. Run the installation.
6. Apply the latest maintenance packs.
19
Step 1: Prepare, Upgrade, and Patch the Master Servers

1 Back up the catalogs.
2 On the master servers, deactivate all policies (in the GUI, or with the
bpplinfo policy_name -modify -inactive command) and wait
for any jobs to complete.
3 In a clustered environment, take NetBackup offline appropriately:
– For MSCS clusters: Take all of the NetBackup group resources offline
except for the disk.
– For VCS clusters: Take the NetBackup resource offline and freeze the
NetBackup Group.
4 On Solaris only, uninstall the old version of NetBackup.
5 Upgrade the software on all master servers. On each master server:
a Stop all services.
b Run the NetBackup 6.x installation program.
On Windows, a silent installation of PBX occurs automatically. On UNIX,
PBX must be installed before NetBackup can be installed. This may be
performed in one of two ways:
› Run the NetBackup install script. You are prompted to load the CD
that contains the PBX installics script, or to type the path to the
PBX installics script.
› Run the PBX installics script first, and then run the NetBackup
install script.
6 Apply the latest maintenance packs.

Step 2: Populate the EMM Database
1. Ensure that the daemons or services and processes
1
are running.
2. Run nbpushdata -add on the upgraded systems in the
following order:
a. The host that was the 5.x Global Device Database host
b. All upgraded master servers
c. All upgraded 5.x Volume Database hosts
3. Restart the services.
4. Run nbpushdata –modify_5x_hosts on all 6.x master
servers.
20
Step 2: Populate the EMM Database
The nbpushdata Command

The nbpushdata command is used to populate the EMM database with
information from the 5.x databases and files. When the nbpushdata command
is run, the following 5.x databases and files are moved into the EMM database:
globDB poolDB stunit_groups

ltidevs volDB Some entries from bp.conf,
vm.conf, and the Registry
robotic_def mediaDB Media and devices-related touch files
ruleDB storage_units
The nbpushdata command is run after the 5.x to 6.x upgrades are complete and
all maintenance packs have been applied on the master server. This is important
because maintenance packs may include nbpushdata-related changes.
1 To run nbpushdata, the daemons or services and processes must be running
on the EMM server and on the systems where the nbpushdata command is
being run.
2 In most NetBackup 5.x environments, the master server, the Global Device
Database host, and the Volume Database host are the same machine, and
nbpushdata -add only needs to be run once on the master server.
However, if the master server, the Global Device Database host, and the

Volume Database host are split across separate hosts, run the nbpushdata -
add command on each host in the following order:
a Run nbpushdata -add from the upgraded host that was the 5.x Global
Device Database host.
b Run nbpushdata -add from all upgraded master servers.
c Run nbpushdata -add from all upgraded 5.x Volume Database hosts.
3 Restart the daemons or services and processes.
4 Run nbpushdata -modify_5x_hosts on all 6.x master servers. This
makes some minor changes to the media servers’ databases, but it does not
push any data into the EMM database.
The nbpushdata command creates an nbpushdata directory at
/usr/openv/netbackup/logs (UNIX) or
install_path\VERITAS\NetBackup\logs (Windows). In 6.0, this
directory contains the log file for the nbpushdata command. In 6.5 this
directory contains a README.txt file that explains where the nbpushdata log
file is located. If the nbpushdata directory or log file cannot be created, the
nbpushdata command fails.
Note: The 5.x media server must be up and reachable. The nbpushdata
-modify_5x_hosts command modifies the 5.x servers so that they can
operate in a mixed environment.
If mistakes are made during the upgrade, for example, running nbpushdata
-add in the wrong order or aborting the command accidentally, work with
Technical Support to clear and repopulate the EMM database. Technical Support
may ask you to run commands similar to the following:
• nbpushdata -remove host_name
This command removes information populated into the EMM database when
nbpushdata -add was run from host_name.
• nbemmcmd -deletehost host_name
This command deletes references to host_name from the EMM database.
• nbpushdata -add
This command repopulates the EMM database.
Recovering from Problems

Review the following TechNotes if you experience problems with the
nbpushdata -add command after the media server upgrade:
• 281789: How to verify the required NetBackup 6.0 daemons or services are up
and running on a master server
and running on a media server
and running on a client

Steps 3-4: Upgrade the NBU Remote Admin
Console System and Any Add-on Components
1. Stop all services.
1
2. Run the installation.
5. Upgrade any add-on components.
6. Install the latest maintenance packs for the add-on
components.
21
Steps 3-4: Upgrade the NetBackup Remote Administration Console

System and Any Add-on Components
If you use the NetBackup Remote Administration console, upgrade the software
on that system as follows:
1 Stop all services.
2 Run the NetBackup 6.x installation program.
4 Restart the services.
Next, upgrade and patch any add-on components. Add-on components that may
need to be upgraded include Bare Metal Restore (BMR) and NetWare Media
Server.

Steps 5-6: Prepare, Upgrade, and
Patch the Media Servers
1. On Solaris only, uninstall the old version of NetBackup.
2. Upgrade the software:
a. Stop all services.
b. Run the installation.
5. Populate the EMM database:
nbpushdata -add
22
Steps 5-6: Prepare, Upgrade, and Patch the Media Servers

1 On Solaris media servers, uninstall the old version of NetBackup using the
pkgrm VRTSnetbp command.
2 Upgrade the software on all media servers that you are migrating to NetBackup
6x. On each media server:
a Stop all services.
b Run the NetBackup 6.x installation program.
On Windows, a silent installation of PBX occurs automatically. On UNIX,
PBX must be installed before NetBackup can be installed. This may be
performed in one of two ways:
› Run the NetBackup install script. You are prompted to load the CD
that contains the PBX installics script, or to type the path to the
PBX installics script.
› Run the PBX installics script first, and then run the NetBackup
install script.
4 Restart the daemons or services and processes.
5 Run the nbpushdata -add command on the upgraded media servers to
populate the EMM database.

Step 7: Upgrade the Clients
1. Perform local or remote client upgrades.
1
2. Install the latest maintenance packs.
23
Step 7: Upgrade the Clients

Upgrade any clients as needed:
• Local or remote upgrades are supported. Multiple clients can be installed
remotely.
• For Windows clients, perform a local or remote client installation, and then
install the latest maintenance pack.
• For UNIX clients, it is possible to push the client software (including the latest
maintenance pack) directly from a UNIX master or media server. Alternatively,
perform a local client installation, followed by a local installation of the latest
maintenance pack.
For specific details, see the OS-specific NetBackup Installation Guide.
NetBackup 5.x clients can coexist in and are supported with a NetBackup 6.x
master server, EMM server, and media server environment. It is not possible to
upgrade from NetBackup 4.5 directly to NetBackup 6.x.

Step 8: Complete and Test the Upgrade
1. Back up the catalogs.

2. Activate the policies.
3. Run test backup jobs.
24
Step 8: Complete and Test the Upgrade

To test the upgrade, activate the policies and run test backup jobs.
Note: You must back up your NetBackup catalogs before and after any NetBackup
upgrade, including maintenance packs. Catalog backups are only readable
by the same version of software that created them.

Resolving Configuration Issues
1
If you want to … Then enter …
Obtain the name of the EMM server that a tpautoconf -get_gdbhost

media server is configured to use,
Correct the name of the EMM server that a tpautoconf –set_gdbhost

media server is configured to use,
List the hostname and server type for nbemmcmd –listhosts

each server that the EMM server can
detect,
Add a media server to the EMM database, nbemmcmd –addhost
Have the EMM server request information nbemmcmd -getemmserver

from the known systems,
25
Resolving Configuration Issues

If there are problems with the upgrade, they may be the result of a
misconfiguration. The most likely misconfiguration occurs when the server
relationships are not defined correctly. For example:
• The EMM server does not recognize one or more of the NetBackup servers.
• The servers do not recognize the (correct) EMM server.
• A master server does not recognize one or more of its media servers.
• A media server does not recognize the (correct) master server.
TechNote 279038, How to determine what EMM server a media server is
configured to use, describes how to detect and resolve these problems.
• Run tpautoconf -get_gdbhost on a media server to obtain the name of
the EMM server the media server is configured to use. Run
tpautoconf -set_gdbhost EMM_server to correct the name.
• Run nbemmcmd -listhosts on any server to list the hostnames that its
EMM server detects and the kind of server it associates with each hostname.
Run nbemmcmd -addhost to add a media server to the EMM database.
• Run nbemmcmd -getemmserver to have the EMM server request the
hostname, server type, and the associated EMM server from each of the known
systems.
• Master and media servers recognize each other as they did in earlier versions,
through the SERVERS entries in the bp.conf file (UNIX) or the Registry
(Windows), where the first entry defines the master, and the remaining entries
are media servers.

EMM Database Issues
Missing, obsolete, misspelled, or otherwise incorrect entries in the EMM database,
in the NetBackup server’s bp.conf file (UNIX), or in the Registry (Windows)
create similar problems. Names are case-sensitive, and they should be in lower
case.
One example of a problem arising from a bad SERVER list is described in
TechNote 277242, Unable to connect to the master server using a NetBackup
Administration Console from a remote host. Connectivity to the master server is
known to be good and the console hostname appears on the server list of the
master server. This issue can occur when the server list on the master server
contains invalid server entries for other hosts, or if the master server has
connection issues with other hosts in the server list.
NetBackup Upgrade Portals

There are two upgrade portals to assist you with upgrading to NetBackup 6.0 or
NetBackup 6.5.
• The NetBackup 6.0 Upgrade Portal is documented in TechNote 285223 and is
located at:
• The NetBackup 6.5 Upgrade Portal is documented in TechNote 290185 and is
located at:

Lesson Summary
• Key Points
1
– In this lesson, you reviewed the main reasons for problems when
upgrading to NetBackup 6.x.
– You learned about some of the prerequisite considerations to
performing an upgrade.
– You also reviewed the general procedure for upgrading from NetBackup
5.x to NetBackup 6.x.
• Reference Materials
– NetBackup Troubleshooting Guide
– NetBackup Installation Guide
– NetBackup Commands
– The support Web site at: http://entsupport.symantec.com for
maintenance packs, release updates, and TechNotes
– TechNotes 236274, 265806, 267965, 277242, 278307, 279038, 281578,
281789, 282017, 282159, 282162, 285223, 290185, 294899
26
Lab 1: Ensuring a Successful Upgrade
In this lab, you will:

• Prepare to upgrade from NetBackup 5.1 to NetBackup 6.0
or to NetBackup 6.5.
• Upgrade from NetBackup 5.1 to NetBackup 6.0 or to
NetBackup 6.5.
• Test the NetBackup 6.0 or 6.5 configuration.
• For Lab Details, see Appendix A.

• For Lab Solution, see Appendix B.
Labs and solutions for this lesson are located on the following pages:
• Appendix A provides step-by-step lab instructions. See “Lab 1 Details:
Ensuring a Successful Upgrade,” page A-6.
• Appendix B provides complete lab instructions and solutions. See “Lab 1
Solution: Ensuring a Successful Upgrade,” page B-6.

Lesson 2
Troubleshooting Methods and Tools
Lesson Introduction
Tools
Database
Issues
Issues
After completing this lesson,

Topic
you will be able to:
Topic 1: Troubleshooting • Identify situations that require
Situations and troubleshooting.
Methodology • Describe a scientific methodology for
troubleshooting NetBackup.
Topic 2: Gathering Gather troubleshooting information by

Information Using the using the nbsupport utility.
nbsupport Utility
Topic 3: Review of Identify NetBackup tools used to

Troubleshooting Tools troubleshoot problems.

Which Situations Require
Troubleshooting?
Are backups and restores functioning

Functionality
correctly?
Are environmental issues affecting

Environment
2
NetBackup?
Is backup and restore performance

Performance
optimal?
Topic 1: Troubleshooting Situations and Methodology

After completing this topic, you will be able to:
• Identify situations that require troubleshooting.
• Describe a scientific methodology for troubleshooting NetBackup.
Which Situations Require Troubleshooting?

The following types of situations may require troubleshooting:
• Issues with functionality
– Are your backup jobs running according to their schedules?
– Do backup jobs fail or are they incomplete?
– Do restore jobs fail or are they incomplete?
– Are restores possible for all of your disaster recovery scenarios?
• Issues with the environment
– Can you eliminate NetBackup as the source of the problem?
– Did you test functionality outside of NetBackup, if possible?
– Are there name resolution, connectivity, or routing network issues?
– Do the system logs show network, controller, or disk errors?
• Issues with performance
– Considering other factors (disk I/O, network speed, application load,
downtime window), are client backups performing optimally?
– Are the storage units performing near their theoretical top speeds?
– Are the media servers performing near their theoretical top speeds?
Lesson 2 Troubleshooting Methods and Tools 2–3

Using the Scientific Method
Step Description
Observe Observe the situation to generate a description of a
symptom or group of symptoms.
Hypothesize Formulate a hypothesis to explain the symptoms.
Predict Make a prediction regarding the existence of other

symptoms or to quantify the results of new
observations.
Experiment Perform experiments to validate the predictions by

using multiple testers following the same
procedure.
Using the Scientific Method

The scientific method is comprised of four steps, as follows:
1 Observe
Ask “How should my NBU domain be running?” This step can include:
– Reading the NetBackup Troubleshooting Guide for a functional overview
– Creating a diagram showing the components in the backup domain,
including the EMM Server, master servers, media servers, and clients
– Reviewing policy coverage and schedules
– Using NOM and the Administration console to view media, capacity, and
performance reports
2 Hypothesize
Ask “What do I think is happening?” This step includes gathering evidence
using:
– The Activity Monitor
– NetBackup logs, system logs, and the Event Viewer
– Reports from elements outside of NetBackup (SAN, LAN, switches,
routers, firewalls, devices)
3 Predict
Ask “How can I isolate what is working and what is not working?” This step
can include:
– Using a “divide and conquer” strategy
– Limiting the external variables

– Attempting to control the environment
– Working on the problem using a subset of the total environment
4 Experiment
Ask “How should I perform the tests?” This step can include:
– Developing a test case with limited scope
– Running the test case and observe the results
– Comparing the results to the predictions
– Correcting failing elements

If the Problem Still Exists…
Repeat the scientific method.
1
Refine and
repeat the
observation.
4 2
Experiment by Formulate a
retesting. new
hypothesis.
3
Make a new
prediction.
Repeating the Scientific Method

If you have performed experiments to test your prediction, and the problem still
exists, repeat the scientific method:
1 Refine and repeat your observations.
2 Generate a new hypothesis based on new observations.
3 Develop a new prediction.
4 Experiment by retesting to prove or disprove your hypothesis.

Resolution or Workaround?
Resolution Workaround
Advantage Is usually a permanent Requires less time to

solution develop
2
Disadvantage Requires more time to Is usually a temporary
develop solution
Example Replace faulty tape drive Backup to disk
Comparing Resolutions with Workarounds

A scientific troubleshooting method may result in a problem resolution, or in some
cases, only a workaround.
• A resolution can take longer to implement and can result in more downtime,
but a resolution is usually permanent.
• A workaround may take less time to implement and result in less downtime,
but a workaround is usually temporary.
Choosing between these two options often involves short- and long-term
supportability of the resolution or workaround.
In the example in the table on the slide, in both the resolution and the workaround,
the backup succeeds, so the data is still protected; however, the resolution allows
for the desired portability and offsite capabilities of tape.

Utilities Used for Gathering
Information
• The NetBackup Configuration Validation Utility (NCVU)
– Obsolete
– Last supported in NBU 5.1
• support
– In the goodies directory
– Superceded by nbsupport and will be retired in the next major
release (7.0)
• nbsupport
– In the support directory
– Supported in NBU 4.5, 5.x, 6.0
• The NetBackup Support Utility (NBSU)
– In the support directory
– Supported in NetBackup 6.5 and higher
Utilities Used for Gathering Information

This slide summarizes the utilities in NetBackup that are used to retrieve
Operating System (OS), network, and NetBackup information:
The obsolete utilities NCVU and support are not covered in this course. The
nbsupport utility is covered later in this lesson. For more information about
NBSU, see the NetBackup Support Utility (NBSU) appendix.

2
Windows
Solaris
Topic 2: Gathering Information Using the nbsupport Utility

After completing this topic, you will be able to gather troubleshooting information
by using the nbsupport utility.
The nbsupport Utility

nbsupport is a utility that produces a compilation of reports designed to
summarize the configuration and status of a NetBackup host. This function is
similar to that of the support utility, but it provides additional information.
The nbsupport utility is automatically installed in
/usr/openv/netbackup/bin/support (UNIX) or
install_path\NetBackup\bin\support (Windows) as part of
NetBackup, since version 6.0 MP2. For prior versions, you must download the
OS-specific archive from the NetBackup Support Web site
(entsupport.symantec.com) and extract the files manually on each system
for which you want to generate reports.
Some of the reports included in the nbsupport output include:
• OS hosts file
• OS network configuration
• Storage Unit list
• Activity Monitor contents
• Output from the available_media script

Using nbsupport

Collect reports related to a nbsupport -media -detail high
media server at a high level of
detail,
Collect reports related to a nbsupport –master –detail medium

master server at a medium
level of detail,
List the touch files that can nbsupport -list_exception_files

disable specific reports,
12
Using nbsupport
nbsupport is a command-line-based utility. The output nbsupport produces
is determined by the parameters it is given. The two primary parameters are:
• Host type
The host type specifies the type of NetBackup system from which
nbsupport will be run. The host type is specified by using
-master, -media, or -client. The host type also determines the type of
reports that NetBackup attempts to generate for the host.
When running nbsupport, determine where you need nbsupport to run.
One problem may warrant collecting nbsupport output from just one
NetBackup system, such as the master server; another problem may warrant
output from several NetBackup systems, such as a media server and two failing
clients.
• Level of detail
The level of detail determines, at a high level, how many reports should be run.
The available options are -detail {low | medium | high}.
In addition, nbsupport can be configured to exclude specific reports from the
output by creating touch files in the appropriate folder. This may be desirable on a
master or media server where a significant amount of time is required to generate a
report.
The slide on this page shows some examples of using the nbsupport utility.

2
Topic 3: Review of Troubleshooting Tools
After completing this topic, you will be able to identify NetBackup tools used to
troubleshoot problems.
Review: Detecting Problems Using the Activity Monitor

The Activity Monitor provides real-time status of NetBackup jobs, daemons, or
services, and processes; it is often the first indicator that a problem has occurred
within your NetBackup environment.
Job information is also available using the bpdbjobs -all_columns
command. Refer to the NetBackup Commands reference manual for column
descriptions. Service, Daemon and Process information is available using the
following commands:
• UNIX: bpps -x
• Windows: bpps

Review: Using the Job Details Dialog Box
From the job list, double-click a job to display the Job Details dialog box. The Job
Details dialog box contains:
• The Job Overview tab
• The Detailed Status tab
Access the Detailed Status tab for the most helpful error detection information.
• The Troubleshooter button
The Troubleshooting Wizard

Under the Detailed Status tab, click the Troubleshooter button to activate the
Troubleshooting wizard. This wizard is available for jobs that return an error status
code. The Troubleshooting wizard provides an explanation about the error and
recommends corrective actions to perform.
In addition, you can always start the Troubleshooting wizard from the Help menu.
You can also type bperror -S status_code -r at a command prompt to
decode a status code and receive troubleshooting recommendations.

Review: The Support Web Site
Knowledge
Knowledge
Base
Base Search
Search
Common
Common
support
support links
links
2
Hot
Hot
Topics
Topics
http://entsupport.symantec.com
18
Review: Accessing the Symantec Support Web Site

To search the Symantec Support Web site:
1 Navigate to http://entsupport.symantec.com.
2 Click the quick link for NetBackup Enterprise Server within the Support for
Top Products section.
This Web page provides links to the most commonly used NetBackup support
links. In addition, from this Web page you can search the NetBackup
Knowledge Base.
3 Enter status code status_code_number in the Search field.
The search produces links to TechNotes related to the specified status code.

Review: Troubleshooting Reports
You can run various reports from the NetBackup Administration Console and from
the command line for verifying, managing, and troubleshooting NetBackup
operations. NetBackup reports show status or problem information for NetBackup
servers or clients.
The following table identifies the equivalent command-line commands for the
listed reports:
Report Command-line Eqivalent

Status of Backups bperror -backstat -U
Client Backups bpimagelist -L
Problems bperror -problems -U
All Log Entries bperror -all -U
Images on Media bpimmedia -U
Media Logs bperror -media -U

Report Command-line Eqivalent
Tape Reports
Images on Tape bpimmedia -tape -U
Tape Logs bperror -tape -U
Tape Contents bpmedialist -mcontents
-owner media_server -ev media_id
Tape Summary bpmedialist -summary -brief
Tape Written bpimagelist -media -U
2
Tape Lists bpmedialist -mlist
Disk Reports
Images on Disk bpimmedia -disk -U
Disk Logs bperror -disk -dt 0 -U
Disk Storage Unit Status N/A
Disk Pool Status N/A
Most of the reporting commands accept the following parameters:
Parameter Description
-hoursago Use -hoursago instead of the -d (start time) and
number_of_hours -e (end time) switches.
-U Use -U for “user readable.”
-l Use -l for long-winded.
-L Use -L for really long-winded.

Review: Detecting Problems Using NOM
Using NOM, you can diagnose problems, identify potential issues, or just review
the operational status of multiple NetBackup master and media servers at many
locations—all from a centralized location.The screen on this page is displayed by
selecting Monitoring—>Overview—>Summary.

Review: Global E-mail Notification
E-mail • Successful jobs

notification • Failed jobs
events • Backup environment maintenance
Configuring • Configure the command-line e-mail

global e-mail interface.
2
notification
• Configure the nbmail.cmd script.
(Windows only)
Specifying • The Global Attributes window

e-mail • The bpconfig command
addresses • The bpadm utility (UNIX only)
22
Review: Detecting Problems Using E-mail Notification

You can detect errors through e-mail notification. With global e-mail notification,
e-mails are sent for all NetBackup events, such as successful jobs, failed jobs, and
backup environment maintenance. Global e-mail notification generates many
e-mails.
NOM also has the ability to send e-mails and SNMP traps, which are triggered by
specific alert conditions.
In UNIX only, there is a template script called mail_bp_reports, which is
located in /usr/openv/netbackup/bin/goodies. Use the
mail_bp_reports script to have reports e-mailed automatically.

Review: Notify Scripts on Component
Systems
Master Server Media Server Client
session_start_notify
parent_start_notify
bpstart_notify
diskfull_notify
Per bpend_notify
Job
backup_notify
backup_exit_notify
restore_notify
parent_end_notify
userreq_notify dbbackup_notify*
session_notify
mail_dr_info*
23
Review: Notify Scripts

Notify scripts are another method used to receive notification for specific events or
errors. Unlike global e-mail notification, notify scripts enable you to filter the
amount of information that you receive.
There are notify scripts that run on the master server, the media server, and the
clients. For e-mail notification of successful or failed jobs, the most useful scripts
are those on the master server or the client.
*The dbbackup_notify script is only run after a cold catalog backup. The
mail_dr_info script is only run after a hot catalog backup. The
mail_dr_info script is discussed later in the course.

Lesson Summary
• Key Points
In this lesson, you learned about situations that require troubleshooting, a
standard troubleshooting methodology, and how to gather information using
nbsupport. You also reviewed NetBackup tools used for troubleshooting.
– NetBackup System Administrator’s Guide
2
– NetBackup Operations Manager (NOM) Getting Started Guide
– http://entsupport.symantec.com
– http://msdn.microsoft.com
– http://www.blat.net
24
Lab 2: Troubleshooting Methods and

Tools
• Generate nbsupport reports.
• Configure notify scripts.
• Use notify scripts to send e-mails identifying successful,
partially successful, and failed backups.
• Optional (NetBackup 6.5 only): Generate NBSU reports.

• For Lab Solutions, see Appendix B.
Troubleshooting Methods and Tools,” page A-16.
Solution: Troubleshooting Methods and Tools,” page B-18.

Lesson 3
NetBackup Process Flow
Lesson Introduction
Tools
Database
Issues
Issues
After completing this lesson, you

Topic
will be able to:
Topic 1: Core Process • Identify the function of the master server,
Functions EMM server, media server, and clients.
• Define the roles of key individual
processes.
Topic 2: Process Identify the communication methods used

Communications between NetBackup processes.
Topic 3: Backup Process Flow Summarize the process flow that occurs
during an automatic backup operation.
Topic 4: Restore Process Flow Summarize the process flow that occurs
during a restore operation.

NetBackup Major Processes Overview
EMM EMM Media Server

Database
and Engine
nbrb* bpbrm
nbemm bptm/bpdm
nbproxy
Master Server
NBU Catalogs
Master Server Client
Jobs bprd
Catalog vnetd/bpcd
nbpem*
nbproxy
bpbkar(32)
tar(32)
3
bpjobd
nbjm*
nbgenjob* (6.0 only) * Part of the IRM
bpdbm
5
Topic 1: Core Process Functions

• Identify the function of the master server, Enterprise Media Manager (EMM)
server, media server, and clients.
• Define the roles of key individual processes.
NetBackup Major Processes Overview

NetBackup uses a multitiered architecture. This architecture includes:
• Master servers
• Enterprise Media Manager
• Media servers
• Clients
NetBackup consolidates all of the media, volume, and device databases once
spread across multiple media servers now into a single relational database. This
improves the scalability of the Enterprise Media Manager or EMM.
Lesson 3 NetBackup Process Flow 3–3

Process Types
PBX-based Legacy
processes: processes:
• nbpem • bprd
• nbjm • bpdbm
• nbrb • bpjobd
• nbemm • bpbrm
• nbvault • bptm
• nbsvcmon • bpdm
• nbnos • bpcd
• nbsl • bpbkar(32)
• nbgenjob (6.0 • tar(32)
only)
6
Process Types
Private Branch Exchange (PBX) is a communication mechanism in NetBackup 6.x
that allows for reduced network port usage. PBX processes use the
pbx_exchange process for all incoming communications. Certain features in
NetBackup, such as Service Monitoring and Unified Logging, are supported only
with these PBX-based processes. PBX processes exist on the master server, EMM
server, and media server. Processes relating to BMR and NDMP are not shown in
the slide.
Legacy processes are processes that do not receive communications from
pbx_exchange. In NetBackup 6.x, if a legacy process requires direct
communication with another process over the network, it may receive the
communication through vnetd (default). Legacy processes exist on all
NetBackup systems.

NetBackup Process Communication
Paths
Master Server Enterprise Media Manager Server
Jobs
Database
nbrb nbemm
bprd bpjobd
nbpem nbjm
EMM
nbgenjob Database
and Engine
Media Server
bpbrm bptm/bpdm
bpdbm
Client vnetd bpcd
3
Master Server bpbkar
Catalogs Notes: nbproxy is not shown
nbgenjob is N/A in 6.5 7
NetBackup Process Communication Paths

This slide shows the connections between the major processes of the Master
Server, the EMM Server, the Media Server and the Client.
Master Server Processes

The following table lists some of the NetBackup master server processes:
Name Function Started by Invokes Description

bprd Request Always bpdbm Listens for and responds to
manager available unscheduled client requests
bpdbm Database bprd nbjm, Maintains NetBackup
manager nbpem catalogs
bpjobd Activity bpdbm N/A Manages the jobs catalog
monitor (primarily)
nbpem Policy bprd nbproxy, Manages the running of
Execution nbjm jobs
Manager
nbjm Job Manager bprd bpbrm Initiates jobs
nbproxy Proxy service IRM services N/A Queries bpdbm catalogs
nbgenjob Generic Job nbjm N/A Supports multiple data
(6.0 only) Process streams, snapshots, and
gathering BMR
information.

Master Server Catalogs
The NetBackup 6.0 Master Server Catalogs include the following information:
• The Images database
• The Backup Policies database
• The Configuration database or global attributes
• The Retention Levels database
• The Client database
• The Catalog backup configuration database
This is the legacy configuration information if a previous version of
NetBackup was installed and a catalog backup was configured.
Enterprise Media Manager (EMM) Server Processes

The following table lists some of the EMM server processes:

nbemm Enterprise nbrb n/a Queries EMM database
Media
Manager
nbrb Resource Always nbemm Resource allocation for
Broker available all NetBackup activities
that require resources
nbproxy Proxy service IRM services N/A Queries NetBackup
catalogs managed by
bpdbm
Enterprise Media Manager Database

The NetBackup EMM database contains the following:
• Volume and pool information
• The active drive status
• The SSO hosts reservation database to support Device Allocation (DA)
• The external files
• The device mappings file
• The NetBackup media error database
• The storage_units file, previously kept on the master server
• The stunit_groups file, previously kept on the master server
• Specific vm.conf entries
• Specific bp.conf entries
• Specific touch files relating to media and device selection
TechNote 278996, Certain bp.conf entries, vm.conf entries, and touch files
are now set using nbemmcmd in Veritas NetBackup 6.0, identifies which entries
are now housed in the EMM database.

Media Server Processes
The following table lists some of the NetBackup media server processes:

(bp)inetd* Listener bprd bpcd Provides a listening
service for
connection requests
to vnetd/bpcd
bpcd Communicatio (bp)inetd* bpbrm Launches processes
n service on remote hosts
bpbrm Backup and nbjm bptm/bpdm Starts actual backup
restore of data by
manager requesting bpcd to
activate bpbkar
Receives backup
metadata from
3
bpbkar and passes
it to bpdbm
bptm Tape manager bpbrm (Child) Manages transfer of
bpdm Disk manager bptm backup images
between the client
bpdm
and the storage
device (tape or disk)
* inetd = UNIX, bpinetd = Windows, xinetd = Linux

Client Processes
The following table lists some of the NetBackup client machine processes:

bpbkar(32) Backup and bpbrm N/A Generates a backup
archive (through image and sends it
manager bpcd) to bptm/bpdm
Sends metadata to
bpbrm
bpcd Communicatio (bp)inetd* bpbkar(32) Authenticates the
n service initiator of a
connection (base-
level) and forks a
process on behalf of
that caller
(bp)inetd* Listener bpbrm bpcd Listening services
Note: Some Solaris
and Linux systems
use xinetd.
tar (32) Compression bpbrm N/A The NetBackup
agent version of tar,
which supports the
multiplexing,
compression, and
encryption
capabilities of
NetBackup, is used
to restore images.
* inetd = UNIX, bpinetd = Windows, xinetd = Linux

Enterprise Media Manager (EMM)
Processes
Part
Part of
of the
the •• Stores
Stores and
and
IRM
IRM manages
manages
media
media and
and
nbrb nbemm device
device
information
information
•• Provides
Provides an
an
interface
interface to
to
access
access this
this
information
information
EMM Database
and Engine
3
8
Enterprise Media Manager (EMM) Processes

The Enterprise Media Manager Server is a new tier within the NetBackup 6.x
Architecture. The Enterprise Media Manager may be installed on the Master
Server system, which is the default, or on a Media Server system. In all cases, the
system on which the EMM Server is installed must have network connectivity to
all server systems in the NetBackup environment.
The Enterprise Media Manager (EMM) Server is centralizes the management and
allocation of resources required for all NetBackup operations, including backup,
restore, duplication, vaulting, import, and catalog backup.
The Enterprise Media Manager (EMM) process, nbemm, runs on the EMM Server
and performs two primary functions:
• Storing and managing media and device information in a centralized, relational
database
• Providing an interface to access, manage, and modify this information
The Enterprise Media Manager and nbemm are discussed in more detail in the
Understanding the NetBackup Database lesson in this course.

Intelligent Resource Manager (IRM)
Processes
• Master server:
– NetBackup Policy Execution Manager (nbpem)
• Handles job scheduling
• Maintains the Next due jobs work list
– NetBackup Job Manager (nbjm)
• Manages the operation of scheduled jobs
• Performs resource allocation requests for all job types
• EMM server:
NetBackup Resource Broker (nbrb)
• Provides centralized resource allocation for an entire server group
• Coordinates the available resources to satisfy all job requests
Intelligent Resource Manager (IRM) Processes

The Intelligent Resource Manager (IRM) is a collection of three daemons or
services. The Intelligent Resource Manager (IRM) enables policy-based and
prioritized scheduling of all jobs in the NetBackup resource allocation scheduler to
improve automation and prevent nonbackup processes from inadvertently
acquiring drives from high-priority processes. Two of these components (nbpem
and nbjm) perform master server operations, while the third (nbrb) is integrated
with EMM. All IRM components require the Veritas PBX software to receive
communications from other processes.
The functions of nbpem, nbjm, and nbrb are:
• Policy Execution Manager (nbpem): nbpem manages the scheduling of jobs
by maintaining a Next due jobs work list. To create this list nbpem
preprocesses the policy configuration on startup (or whenever policies are
modified), determines which jobs need to run, and assigns a timer for each job.
When a timer expires, nbpem notifies nbjm to initiate the job.
• Job Manager (nbjm): nbjm processes all scheduled backup jobs at the request
of nbpem. Processing by nbjm permits the job to queue and become visible in
the activity monitor. Other job types, such as restore and import, are only
processed by nbjm for resource allocation requests to nbrb.
• Resource Broker (nbrb): nbrb provides centralized resource management
for all jobs in a NetBackup server group. nbrb identifies available resources
from the EMM database to satisfy requests from nbjm.

The IRM Work List: Scheduled Backups
Master Server
GUI / CLI
3
Policy Change bpjobd
2 bpdbm nbpem 6 nbjm

1
Next Due Job Work List

Master Server Client Policy Schedule Due Status
Catalogs Time
5 train7 db-back full 9:00 PM Not
Due Due
train1 servers full 9:30 PM Not Due
3
4 train5 unix-client incr 9:30 PM
10:00 Not Due
710
1
2
3
4
5
6
The IRM Work List: Scheduled Backups

The slide shows how the IRM work list is manipulated during scheduled backups.
1 On initialization or wake up nbpem builds a work list of due jobs in memory
on the master server. This initial work list consists of scheduled backup jobs
only, and it is based on the policy and schedule configuration data received
through bpdbm. After the work list is built, nbpem maintains the work list on
an as-needed basis for specific policies. The slide shows a simplified example
of the Next due job work list.
2 When a change to a policy is made using the GUI or a command line utility,
bpdbm is notified to update the master server catalogs.
3 If the change impacts scheduling, nbpem is notified automatically of the new
configuration.
4 nbpem updates the work list to reflect the change after the Policy Update
Interval is reached (10 minutes by default). In this example, the backup
window for client train5 was modified, and the due timer for the job
changed from 10:00 P.M. to 9:30 P.M.
5 Every item in the work list is associated with a timer that indicates when the
job is due to run. The work list is sorted by job due time. When the timer
expires, the job status is changed from not due to due.
6 nbpem checks the work list for due jobs every minute. When due jobs are
found, nbpem sends a job try for each due job to the nbjm process.
7 nbjm notifies bpjobd to add the job to the jobs database, and the job appears
in the Activity Monitor in the queued state.

IRM Work List: Manual Backups
Master Server
bprd
bprd 2 bpjobd
child
3
1 6
bpdbm nbpem 5 nbjm

Manual backup
request
Next Due Job Work List

Master Server Client Policy Schedule Due Status
Catalogs Time
4 train3
train1 unix-client
servers full 9:00 PM Due Due
Not
train2
train7
train2 db-back
servers full 9:30 PM
9:00 Not Due
train7 db-back full 9:30 PM Not Due
611
1
2
3
4
5
IRM Work List: Manual Backups

The slide shows how the IRM work list is manipulated during manual backups.
1 A request for a manual (unscheduled) backup operation is sent to bprd.
2 bprd spawns a child process.
3 The child bprd process passes the job start parameters to nbpem and then
exits.
4 nbpem enters the job into the work list as a run once job that is due to start
immediately.
5 nbpem sends a job try to nbjm.
6 nbjm notifies bpjobd to add the job to the jobs database and the job appears
in the Activity Monitor in the queued state.

Using Scheduling Utilities

Update the nbpem work list, nbpemreq –updatepolicies
Determine due jobs for a specific nbpemreq –predict

time, –date mm/dd/yyyy hh:mm:ss
View the raw nbpem work list, nbpemreq –tables screen
Prevent new jobs from entering bprdreq -terminate

the queue,
Resume allowing new jobs to • UNIX:

enter the queue, bprd
• Windows:
3
net start “NetBackup Request
Manager”
12
Using Scheduling Utilities

This slide describes some of the common functions you can perform using the
nbpemreq and bprd utilities.
• nbpemreq -updatepolicies
This command initiates nbpem to reread the policy configuration and update
the work list. nbpem checks for changes every 10 minutes by default.
• nbpemreq -predict -date mm/dd/yyyy hh:mm:ss
This command displays policies that are eligible to run at the time specified.
The -date mm/dd/yyyy hh:mm:ss or the -dateu unix_ctime
option must be specified. A job is displayed for any time specified within its
backup window.
• nbpemreq -tables screen
This command dumps the work list to the screen. The display includes the
policy and the schedule for each job, and the specific time the job is due to run.
A simpler version, nbpemreq -due is now available in NetBackup 6.5.
• bprdreq -terminate
Terminating bprd prevents any new manual jobs from entering the queue. In
addition, a command is passed to nbpem that prevents any job scheduling
from occurring until bprd comes back online.
• bprd (UNIX) or net start "NetBackup Request Manager"
(Windows)
This command starts bprd, which receives requests for manual jobs. This
command also notifies nbpem to resume job scheduling.

Resource Allocation for Backup Jobs
Master Server EMM Server
EMM
5 Database
and Engine
bpjobd
Media and
5 drive Physical nbemm
information resources
Job try
4
nbjm nbrb 3 MDS DA
1
bpdbm 2
Logical
resources
Master Server
Catalogs
13
Resource Allocation for Backup Jobs

The slide shows how resources are allocated for backup jobs.
1 Resource requests for all jobs in NetBackup begin with a resource request from
nbjm to nbrb.
2 The nbrb process engages bpdbm for logical resources or limitations on
items, such as MPX groups and the maximum jobs allowed per client or policy.
3 nbrb queries the MDS function of nbemm for available physical resources,
such as storage units, drive paths, and media.
The nbemm process includes two independent functions: Media and Device
Selection (MDS) and Device Allocator (DA). MDS determines the available
media and device resources for any tape-based operation in NetBackup. The
Device Allocator (DA) manages the support of the Shared Storage Option
(SSO).
4 nbrb provides nbjm with the specific resources to be used for the job, based
upon the available resources reported from bpdbm and nbemm.
5 nbjm activates the job and instructs bpjobd to change the Activity Monitor
job state from queued to active.

Media Mount
Master Server Media and drive
information
nbjm 1
Media Server 2
Establish client
connection
vmd bpbrm 3 A
Move media
Robot Arm txxcd 4
7 ltid 5 bptm
Tape 8 txxd 6
Drive Load and
3
scan drive
avrd
14
Media Mount
The slide shows the process flow during a media mount.
1 The nbjm process on the master server receives information on which specific
drive and media are available for the operation.
2 Explicit instructions on which resources to use are passed from nbjm to
bpbrm, the backup and restore manager, on the media server.
3 Before proceeding with the mount request, bpbrm establishes a socket
connection with the client system.
4 After the connection is established, and before backup data flow begins,
bpbrm sends instructions to mount a specific drive and media to bptm, the
tape manager.
5 bptm forwards the request to the device manager daemon (ltid).
6 ltid calls the robotic drive daemon, for example txxd.
7 txxd calls the robotic control daemon, for example txxcd, on the robot
control host to issue SCSI commands to mount the media.
8 After the media is mounted, txxd scans the drive to verify that the correct tape
is loaded.
vmd is used for remote media and device management and avrd is used for bar
code and recorded label recognition.
Now nbjm is notified, and the backup data flow from the client to the media
server begins.

Backup Data Flow
Master Server Client
Master Server bpdbm

Catalogs Client
Data
1
4
Media Server Establish client
connection
A
bpbrm 2 bpbkar
Metadata
bptm
4
parent
2
3
bptm
child Backup image
15
Backup Data Flow

The slide shows processes that are involved in moving the data during a backup
operation.
1 After the media mount, bpbkar, the backup and archive program, begins
reading and packaging the backup data on the local client.
2 An image of the backup data is streamed to the bptm child process on a
remote media server, and metadata about the backup image is streamed
simultaneously to bpbrm.
3 On the media server, the bptm child moves the incoming client data to the
bptm parent by using a shared memory buffer.
4 The parent process writes this data to tape. At the same time, the bpbrm
process relays the client metadata to bpdbm on the master server so that it may
be written to the Image database.

The NetBackup Service Monitor
• Automatically restarts PBX-based services that terminate

abnormally
• Enabled by default
• Configured by using the nbsvcmon.conf file
– UNIX: /usr/openv/netbackup/nbsvcmon.conf
– Windows: install_path\NetBackup\nbsvcmon.conf
• Default configuration:
– Monitor interval: 60 seconds
– Restart limit: 3
– Service monitoring: Enabled
• Disabled automatically for NetBackup cluster configurations
3
16
The NetBackup Service Monitor

The NetBackup Service Monitor, nbsvcmon, runs on the master server, EMM
server, and media servers to monitor and recover any PBX-based daemon or
service that terminates abnormally.
nbsvcmon default settings are shown on the slide. Edit the nbsvcmon.conf
file to adjust the default settings. All values must be specified in hexadecimal, as
follows:
• "MONITOR_INTERVAL"=dword:0000003c
The MONITOR_INTERVAL is the period between checks for monitored
daemons or services. If a monitored service is not running, nbsvcmon restarts
it.
• "RESTART_LIMIT"=dword:00000003
RESTART_LIMIT is the maximum number of times a daemon or service can
be restarted by nbsvcmon. Restarting nbsvcmon resets this counter.
• "MONITOR_ON"=dword:00000001
MONITOR_ON enables or disables the automatic restart of daemons or services
by nbsvcmon.
The remaining parameters in the nbsvcmon.conf file are managed by
individual daemons or services, and they should not be modified.

Private Branch Exchange (PBX)
Network
Port 1556
PBX-Based PBX-Based
pbx_exchange
Process Process
Legacy
Process
18
Topic 2: Process Communications

After completing this topic, you will be able to identify the communication
methods used between NetBackup processes.

Veritas Private Branch Exchange (PBX) allows all PBX-based processes to receive
communications on a single port. PBX is a required software component for all 6.x
master, media, and EMM servers and cannot be disabled.
The pbx_exchange process listens on port 1556 (default) to receive requests for
all PBX-based processes, even if they are local to the system. Port 1557 is also
used by PBX for registering local services (not shown in the slide). Only port 1556
is used for connections over the network.
In the slide, the incoming request contains an identifier for the process being
sought, which PBX uses to establish direct communication between the source and
destination processes.

The Network Daemon (vnetd)
Network
Port 13724
PBX-Based
(bp)inetd vnetd
Process
Legacy Legacy
bpcd
Process Process
3
19
The Network Daemon (vnetd)

The NetBackup Network Daemon (vnetd) is very similar in function to PBX.
vnetd reduces port usage between NetBackup systems by listening on port 13724
for connections to any legacy process it supports on a given system.
vnetd exists on NetBackup client systems. This allows for a one-way socket
connection from media server to client in order to establish communications. A
return socket connection from the client to the media server is not required for 6.x
clients.
vnetd is used by all legacy NetBackup daemons or services that do not use PBX
and that must listen for incoming connections from the network.

bpcd and (bp)inetd
Network
Port 13724
PBX-Based
(bp)inetd vnetd
Process
Legacy Legacy
bpcd
Process Process
20
bpcd and (bp)inetd

The NetBackup Communications Daemon (bpcd) performs forward and reverse
name lookups and hostname authentication. In addition, bpcd initiates other
processes based on requests from remote systems. bpcd is present on the master
server, media server, EMM server, and client. For example, on a client system,
bpcd initiates bpbkar (UNIX) or bpbkar32 (Windows) or tar (UNIX) or
tar32 (Windows) based on backup and restore requests from bpbrm on the
media server.
By default bpcd does not receive direct network connections on port 13782.
Rather, an initial connection is made through inetd and vnetd on port 13724,
and then that connection is passed to bpcd.
The inetd (UNIX) or bpinetd (Windows) process is simply a listener for
incoming connections from the network, and it initiates the vnetd (default) or
bpcd process as required.

nbproxy
nbpem nbjm nbrb
nbproxy nbproxy
bpdbm
Master Server
3
Catalogs
21
nbproxy
The nbproxy process is used on master and EMM server systems to
communicate with bpdbm for access to the master server catalogs. nbproxy acts
as an adapter between the IRM and EMM processes to communicate with bpdbm.
This communication is necessary because the bpdbm process is single-threaded
and cannot receive direct communications from the multithreaded IRM processes.
In this way, the IRM processes can access the master server in a synchronized,
consistent way. The policy configurations, Image database, and global settings for
the server group are made available to IRM through nbproxy.
Processes that use nbproxy in this way are nbpem, nbjm, and nbrb. A separate
instance of nbproxy runs persistently for all three processes.

Backup Operation Process Flow
(1 of 4)
6
Jobs bpjobd 2 nbjm 3 nbrb

Database
1 5
EMM
Master Server bpdbm nbpem nbemm Database
Catalogs and Engine
4
Media Server Client

bpbrm bpbkar Data
bptm 4 Note: Actual flow is Client

nbrbÆnbjmÆnbproxyÆbpdbm
23
Topic 3: Backup Process Flow

This topic provides an overview of the flow of a single stream backup operation to
tape. Child processes and processes relating to NetBackup communications, such
as vnetd and PBX, are not shown.
After completing this topic, you will be able to summarize the process flow that
occurs during an automatic backup operation.
Backup Operation Process Flow (1)

1 On the master server, nbpem communicates with nbjm to schedule a job
using the job start function.
2 nbjm initiates a backup job by communicating with bpjobd and the job is
added to the job list in the jobs database. The job is now visible in the Activity
Monitor in a queued state, waiting for resources to be allocated.
3 nbjm communicates with nbrb on the EMM server to request the necessary
resources.
4 From the EMM server, nbrb accesses resource consumption constraints from
bpdbm, the database manager, on the master server. Resource consumption
constraints include configured policies and other attribute information such as
max jobs per policy and max jobs per client.
5 nbrb secures physical resources from nbemm. Resources include storage
units, tape drives, and media IDs.
6 After resources are acquired, nbrb notifies nbjm that the resources have been
allocated.

(2 of 4)
Jobs bpjobd nbjm nbrb

Database
EMM
Catalogs and Engine
Media Server Client

bpbrm 8 bpbkar Data
10
9
3
bptm Client
24

7 nbjm communicates with bpjobd and the job becomes active. On the master
server, nbjm communicates with the media server to start bpbrm. nbjm
passes the start job arguments to bpbrm. These arguments include the tape and
drive to be used for the backup.
8 On the media server, bpbrm establishes a connection with bpbkar on the
client.
9 bpbrm also starts bptm and passes it instructions to mount the media to a
specific drive.
If the backup is using a disk storage unit, bpbrm calls bpdm, the disk
manager, at this step.
10 When the tape mount completes, bpbrm notifies bpbkar that it may begin
transferring the data for the backup.

(3 of 4)
Jobs bpjobd nbjm nbrb

Database
EMM
Catalogs and Engine
11a
Media Server Client
bpbrm 11a bpbkar Data
11b
bptm 11b
Client
25

11 bpbkar begins sending two concurrent data streams as follows:
a One stream contains metadata about the backup.
bpbkar sends the metadata to bpbrm on the media server. bpbrm then
sends the metadata to bpdbm on the master server, where it is written to
the Image database.
b One data stream contains the actual image data to be backed up.
bpbkar sends the actual image to bptm on the media server, where it is
committed to tape.

(4 of 4)
13
Jobs bpjobd nbjm 14 nbrb
Database
15
EMM
Catalogs 12 and Engine
Media Server Client

bpbrm bpbkar Data
3
bptm Client
26

12 After the data transfer is complete, bpbrm notifies nbjm that the data transfer
has completed.
13 nbjm notifies bpjobd to change the job in the Activity Monitor to a Done
state.
14 nbjm notifies nbrb that the job is complete, initiating the release of
resources.
15 nbjm notifies nbpem of the job completion so that nbpem may update the
Next due time for the job or remove the job from the work list.

Restore Operation Process Flow
(1 of 4)
Master Server User request
EMM Server
1
3
Jobs bpjobd bprd
Database
nbjm nbrb
2
EMM
Master Server bpdbm nbemm Database
Catalogs and Engine
4
Media Server Client
Data
bptm
tar
bpbrm Client
28
Topic 4: Restore Process Flow

This topic provides an overview of the flow of a restore operation from tape. Child
processes and processes relating to NetBackup communications, such as vnetd
and PBX, are not shown.
After completing this topic, you will be able to summarize the process flow that
occurs during a restore operation.
Restore Operation Process Flow (1)

1 A user request to initiate a restore job is sent to bprd.
2 bprd contacts bpdbm to identify the files and the client targeted for the
restore, and to identify media information.
3 bprd communicates with bpjobd to initiate the restore job. The restore job is
now visible in the Activity Monitor.
4 bprd initiates bpbrm on the media server that owns the tape that contains the
requested backup image.

(2 of 4)
Jobs bpjobd bprd

Database
10
nbjm 8 nbrb
9
EMM
Catalogs and Engine
Media Server Client

Data
bptm
tar
3
6
5
bpbrm Client
29

5 bpbrm establishes a connection with the tar process on the client system.
6 bpbrm initiates bptm on the local media server.
If the backup image is located on a disk storage unit, bpbrm calls bpdm at this
step.
7 bptm contacts nbjm on the master server to request the required resources.
8 nbjm contacts nbrb to determine if the resources are available.
9 nbrb queries nbemm to reserve the required resources.
10 nbrb returns a confirmation on the required resources to nbjm

(3 of 4)
Jobs bpjobd bprd

Database
nbjm nbrb
EMM
Catalogs and Engine
11
Media Server Client

Data
bptm
12 tar 13
bpbrm Client
30

11 nbjm confirms the resource allocation to bptm.
12 bptm mounts the media, reads the backup image from tape, and begins
streaming the requested data to the tar process on the client.
13 tar commits the data to the storage destination on the client as it receives the
data from bptm.

(4 of 4)
17
Jobs bpjobd bprd
Database
16
nbjm 15 nbrb
EMM
Catalogs and Engine
14
Media Server Client

Data
bptm
tar
3
bpbrm Client
31

14 When the data transfer completes, bptm un-mounts the tape and notifies
nbjm that the job is complete.
15 nbjm sends instructions to nbrb to deallocate the media and drive resources.
16 nbjm notifies bprd that the restore operation is complete.
17 bprd sends instructions to bpjobd to change the state of the job to "Done" in
the Activity Monitor.

Lesson Summary
• Key Points
In this lesson, you reviewed the functions of the master server, EMM
server, media server, client, and their daemons or services and
processes. You looked at the communication methods used between
NetBackup processes. You also followed the processes through an
automatic backup operation and a restore operation.
– NetBackup Troubleshooting Guide: Appendix A, Functional Overview
– http://entsupport.symantec.com/docs/282015 for Details on the
VERITAS NetBackup Backup and Restore Process Flow
– TechNote 278996
32
Lab 3: NetBackup Process Flow

• Configure the NetBackup Service Monitor.
• Manage job activity.
• Identify relevant processes for specific failures.

NetBackup Process Flow,” page A-25
Solution: NetBackup Process Flow,” page B-29

Lesson 4
Using Debug Logs
Lesson Introduction
Tools
Database
Issues
Issues

Topic
will be able to:
Topic 1: Log Management • Enable legacy logging.
• Use the vxlogcfg command to manage
unified and robust log settings.
Topic 2: Capturing Debug Use the vxlogmgr command to prepare

Logs unified logs for sending to NetBackup
Technical Support.
Topic 3: Viewing Debug Logs • Extract relevant data from legacy logs.
• Use the vxlogview command to format
data from raw unified logs.

Legacy vs. Unified Logging
Process Name Legacy Unified Process Name Legacy Unified
bprd X bpdm X
bpdbm X ltid X
bpjobd X vmd X
nbpem X txxd/txxcd X
nbjm X vnetd X
nbproxy X bpcd X
nbgenjob X bpbkar X
nbsvcmon X tar X
pbx_exchange X bpmount X
nbrb X bpbackup X
nbemm X bprestore X
bpbrm X bparchive X
bptm X bplist X
5
Topic 1: Log Management
4
• Enable legacy logging.
• Use the vxlogcfg command to manage unified and robust log settings.
Legacy vs. Unified Logging

The chart on this slide lists NetBackup 6.x processes and which type of debug
logging they utilize. Not all NetBackup processes are listed, nor are all listed logs
covered in this course.The two basic types of debug logging in NetBackup 6.x are:
• Legacy logging
Legacy debug logs provide a record of NetBackup activity related to specific
legacy processes.
• Unified logging
Unified logging was introduced with NetBackup 6.0. Unified logging is simply
a standard format for log file names and messages that is planned to become
common across all Symantec products.
Keep in mind that legacy logging is still required with NetBackup 6.x. Unified
logging has not superseded legacy logging, particularly in the case of the bp
commands.
Debug log files can grow very large, especially if the verbose level is set to log
maximum information. As a best practice, enable debug logs only when
troubleshooting or when directed by Technical Support. After a problem has been
corrected, disable the debug logs.
Lesson 4 Using Debug Logs 4–3

Legacy-Style Debug Logging
bpbrm
Media bpdm
bptm
EMM vnetd
bprd bpcd
Master
bpdbm
bpdbjobs
vnetd
bpcd
Clients
admin
bpcd bpbkar
vnetd tar
Legacy-Style Debug Logging

When using NetBackup legacy debug logs, consider the following:
• A debug log is created by a process only if the subdirectory for that process has
been created. The subdirectory must exist before the process starts if logging is
to occur.
• A process creates one debug log file per day. Legacy debug logs have names in
the form log.mmddyy (UNIX) or mmddyy.log (Windows).
• NetBackup retains debug logs for the number of days specified by the Duration
to Keep Logs global attribute (28 days by default). After that, NetBackup
deletes the files.
• Log files can grow very large. Enable these files only if unexplained problems
exist. Delete the logs and the directory when they are no longer needed.
The Location of Common Legacy Logs

Legacy logs are created for the master server, media servers, and clients.
The graphic in the slide denotes which logs reside on the different machines within
a NetBackup environment. When these directories exist, NetBackup creates log
files in the directory for the associated process.
The management of the legacy-style debug logs is handled differently and
independently between NetBackup and Media Manager.

Enabling Legacy NetBackup Logging
Step Action
1 Create directories under:

– UNIX:
/usr/openv/netbackup/logs
– Windows:
install_path\NetBackup\logs
2 In the Host Properties, set the logging levels.
3 Restart daemons or services to activate the logs.
Enabling Legacy NetBackup Logging
4
Perform the following steps to enable the NetBackup debug logs:
1 Create directories for the NetBackup logs.
For those processes that use legacy logging, create a log directory for each
process to be logged as follows:
– UNIX: /usr/openv/netbackup/logs/process_name
– Windows: install_path\NetBackup\logs\process_name
The mklogdir script can be used to create all the legacy logging directories.
However, this script also enables all daemons or services and processes to be
logged the next time that they are restarted, causing excessive disk I/O
operations and disk space usage. Use caution with this script and tailor it
accordingly.
Run the mklogdir script as follows:
– UNIX: /usr/openv/netbackup/logs/mklogdir
– Windows: install_path\NetBackup\logs\mklogdir.bat

2 Set the verbose levels for the NetBackup logs.
The Logging properties apply to currently selected master servers, media
servers, and clients. The available properties differ between a server and a
client. Verbose levels are set on the system where logging occurs.
Verbose levels are set in Host Properties, in the bp.conf file (UNIX), or in
the Registry (Windows).
To access the Host Properties from within the NetBackup Administration
Console, double-click the name of the host or right-click the name and select
Properties. The Master Server properties dialog box is displayed. Select
Logging in the left pane.
You can increase the amount of information logged by adjusting the logging
levels. The verbose level (0 through 5) identifies how much information is
logged.
3 Activate the logs.
To activate the logs, within the Activity Monitor, click either the Daemons
(UNIX), Services (Windows), or Processes tab to select the daemon, process,
or service you want to restart. A persistent daemon or service, such as bprd or
bpdbm, must be stopped and restarted to activate logging. A transient process,
such as bpcd or bptm, logs to its directory the next time the process runs.
– On UNIX systems, use the netbackup stop command to shut down
NetBackup daemons, and use the netbackup start command to start
NetBackup daemons.
– On Windows systems, use the bpdown command to shut down NetBackup
services, and use the bpup command to start NetBackup services.

Media and Device Management
Debug Logs
• UNIX: /usr/openv/volmgr/debug
• Windows: install_path\Volmgr\debug
Create the … folder To enable logging for …

daemon vmd (Volume Manager service)
reqlib The process requesting a vmd service
tpcommand The tpconfig and tpautoconf commands
ltid ltid (Device Manager service)
robots SCSI robotic daemons
10
Media and Device Management Debug Logs
4
Media Manager logging is a form of legacy logging that may be requested by
NetBackup Technical Support. The processes logged by media manager logging
play a lesser role in NetBackup 6.x than in previous versions due to the
introduction of the EMM server.
Media Manager debug logs are enabled by creating a directory with a name, as
shown in the slide.
Additional debug and informational messages from the robot and drive processes
(txxcd and txxd) are also logged to the /var/adm/messages file (UNIX)
and the Event Viewer application log (Windows).
• The txxd/txxcd processes write to log files in the robots directory.
• The vmd daemon writes to log files in the daemon directory.
• The tpcommand log directory contains debug log files used by the
tpconfig and the tpautoconf commands.

Enabling Media and Device
Management Logging
• Add a VERBOSE entry in the vm.conf file:
– UNIX: /usr/openv/volmgr/vm.conf
– Windows: install_path\Volmgr\vm.conf
MM_SERVER_NAME = pc1train07
MEDIA_ID_BARCODE_CHARS = 0 8 1:2:3:4:5:6
VERBOSE VERBOSE entry
• Upon startup, ltid reads the vm.conf file. If ltid finds

the VERBOSE entry, it passes the VERBOSE parameter to
the other Media Manager processes (vmd, txxd, txxcd,
and avrd).
11
Enabling Media and Device Management Logging

The verbosity level for Media Manager logs is simply enabled or disabled
(default). When Media Manager logging is enabled, all enabled Media Manager
daemons or services and processes create verbose debug logs.
To enable verbose Media Manager logging, add the word VERBOSE to the
vm.conf file, as shown on the slide.
Media Manager logs system errors, warnings, and debug information as follows:
• Errors are logged with LOG_ERR.
• Warnings are logged with LOG_WARNING.
• Debug information is logged with LOG_NOTICE.
Refer to the syslogd man page(UNIX) for the locations of system log messages.
For media manager logs, NetBackup creates one log per day in each of the debug
directories with file names of the form log_name.date.log.
By default, legacy-style debug logs created by Media Manager processes are not
cleaned up automatically; they accumulate indefinitely unless administrative
action is taken. To clean up Media Manager debug logs, place the entry
DAYS_TO_KEEP_LOGS = xx in the vm.conf file on the Media Server system
where debug logging has been enabled. xx represents the number of days that log
files accumulate before cleanup.
For example, if xx is set to 7 days, a Media Manager debug log file is
automatically deleted and removed from the Media Manager system after the
debug log file is eight days old.

Unified-Style Debug Logging
• Is new to NetBackup 6.x

• Will become the standard for most
Veritas products nbemm
• Logs IRM, EMM, PBX, BMR, nbrb Media
and NOM nbsvcmon
nbsvcmon nbsl
• Allows for greater
control over EMM
logging
nbpem Master
nbjm
nbsvcmon
nbsl
Clients
12
Unified-Style Debug Logging
4
Unified logging was introduced with NetBackup 6.0. Unified logging is simply a
format for log file names and messages that is planned to become standard across
most Symantec products.
By default, unified logging is enabled on all NetBackup 6.x master servers and
EMM servers. Unified logging primarily covers components new to NetBackup
6.x. These components include the Intelligent Resource Manager (IRM), the
Enterprise Media Manager (EMM), Private Branch Exchange (PBX), Bare Metal
Restore (BMR), and NetBackup Operations Manager (NOM).
Unified logging cannot be completely disabled; however, the level of detail can be
adjusted by configuring its verbose levels without requiring a restart of the
daemons or services.
Raw unified logs are located in the /usr/openv/logs directory (UNIX) or the
install_path\NetBackup\logs folder (Windows).
Note: Legacy logging is still required with NetBackup 6.x. Unified logging has not
superseded legacy logging, particularly in the case of the bp commands.

Unified Logging Standard Naming
Format
productID-originatorID-hostID-date-rotation.log
Sample Unified log file name:

51216-111-3474696384-060710-0000000051.log
Variable Description
productID productID identifies which VERITAS product wrote the
log.
originatorID originatorID identifies which process, service, script, or

other software wrote the log.
hostID hostID is the host that created the log file. Unless the log
file was moved, this is the host where the log resides.
Date date is in year, month, day format (YYMMDD).

rotation.log rotation.log is a numbered instance of a log file for the
originator. The number increments each time log rollover
occurs.
13
Unified Logging Standard Naming Format

Unified logs are named using the standard format, as follows:
productID-originatorID-hostID-date-rotation.log
Product IDs
NetBackup uses two Product IDs, as follows:
• NetBackup (NB) is Product ID 51216.
• Infrastructure Core Services (VxICS) is Product ID 50936.
Originator IDs
The following table is a partial list of originator IDs and short names:
Originator ID Short Name Originator ID Short Name

103 PBX 118 nbrb
111 nbemm 132 nbsl
116 nbpem 143 mds
117 nbjm 144 da
A comprehensive list of Originator IDs for NetBackup can be found in

/usr/openv/netbackup/nblog.conf (UNIX) or
install_path\NetBackup\nblog.conf (Windows).

DebugLevel for
IRM Processes
Setting Unified-Style Debug Log Verbosity
4
Use the NetBackup Host Properties to change the unified logging diagnostic
message level for the nbrb, nbpem, and nbjm logs on a master or EMM server.
Only diagnostic level messages are changed through Host Properties, not debug or
application messages. Unified logging message types are discussed later in this
lesson.
The diagnostic logging level has six possible numeric values, 0 - 5, representing
the amount of detail to be logged. In addition, an individual unified log may be set
to No Logging (disabled) or Same as global.
Use the vxlogcfg command to configure unified logs for processes other than
nbrb, nbpem, and nbjm, or for diagnostic level messaging.
The vxlogcfg Command

The following command configures OID 117 (nbjm) to log debug messages at
verbose level 5:
vxlogcfg -a -p 51216 -o 117 -s DebugLevel=6
The six message levels are represented by 0 through 5 within the NetBackup
console, and by 1 through 6 when using the vxlogcfg command. Setting a
message level to 0 using vxlogcfg disables logging for that message type.
PBX log configuration and verbosity is different than the NetBackup VxUL
logging. Refer to TechNote 279929 How to change the logging level and other
unified log settings for the Veritas Private Branch Exchange process for details.

Log File Rollover
• Rollover mode (based on file size or

Log rollover time)
attributes:
• The number of logs to keep per process
• Rollover is always on.

For unified logs:
• Individual logs may be configured.
• Enable rollover mode by enabling

Robust Logging.
For legacy logs:
• All legacy logs are configured using
OID 112.
16
Log File Rollover

Robust logging simply means that rollover mode and the number of log files can
be configured. Log rollover is required on unified logs and can be enabled on
certain legacy logs using host properties. These logs include:
bpbkar bpcd bpdm bprd

bpbrm bpdbm bptm
Robust debug logging is also referred to as legacy logging file rotation. When
robust logging is enabled, rather than each NetBackup process creating a single log
file of unpredictable size per process per day, each process is limited on the size to
which its debug log file can grow before starting a new log file (rollover mode).
Additionally, robust logging controls the number of logs (per process) that can
exist at one time. After the limit is reached, robust logging deletes the oldest file.
Robust logging does not affect Media and Device Management debug logs.
Robust log file names include an incremental number designation:
MMDDYY_NNNNN.log, where NNNNN is an incrementing number from 00001 to
99999.
For specific unified logs, use vxlogcfg to customize rollover mode and the
number of log files. Legacy logs cannot be configured individually; to configure
the robust settings for all legacy logs, specify originator ID 112.

4
Enabling Robust Logging
Robust logging is not the default behavior for logs. Explicitly enable robust
logging by marking the Enable robust logging check box in the Logging pane of
the Master Server Properties, Media Server Properties, or Client Properties dialog
boxes as shown in the graphics.
By default robust logging retains three files per process at 5MB each. Change
these values using the vxlogcfg command.
The following commands configure robust logging to retain 10 files at 15MB per
process.
vxlogcfg -a -p 51216 -o 112 -s MaxLogFileSizeKB=15360
vxlogcfg -a -p 51216 -o 112 -s NumberOfLogFiles=10

Using the vxlogcfg Command
The following are examples of using the vxlogcfg command to

manage unified and robust log settings:
• List the configuration options for originator ID 111 (nbemm):
vxlogcfg –l –p 51216 –o 111
• Configure all unified logs for diagnostic and debug verbose level
6:
vxlogcfg –a –p 51216 –o ALL –s DebugLevel=6
–s DiagnosticLevel=6
• Configure log rollover for all unified and robust logs based on file
size, with a maximum size of 20480 KB:
vxlogcfg –a –p 51216 –o ALL –s RollOvermode=FileSize
–s MaxLogFileSizeKB=20480
• Review NB product default log configuration settings:
vxlogcfg -l -p 51216 -o Default
19
Using the vxlogcfg Command

The vxlogcfg command is used to view and configure unified and robust log
settings. These include the verbose logging level, the location of the logs, and
parameters for log rollover. The following is a partial list of vxlogcfg
parameters with examples.
List or Change Log Parameters (specify one):
-l Lists configuration settings
-a Modifies the product’s unified logging settings, in conjunction with
other vxlogcfg options
Product and Originator ID (required for both the list and change functions):
Parameter Example Description

-p -p 51216 Specifies the product ID
This is typically 51216, the NetBackup product ID.
-o -o 116 Specifies the originator ID of the process to be
viewed or configured
Use ALL to affect all unified logs, including rollover
settings for robust logs..

Configuration Parameters (used only with the -a option):

-s keyname=value -s DebugLevel=1 Specifies a configuration
parameter to modify, and the new
value
A partial list of key names follows.
Keys and Values (used only with the -s option):
Key Name Allowable Values Description

LogDirectory new_directory Configures a new destination
For example for the specified logs
/usr/opt/nbulogs
or F:\nbulogs
DebugLevel 0–6 Specifies the verbose logging
level for debug messages
DiagnosticLevel 0–6 Specifies the verbose logging
level for diagnostic messages
RolloverMode FileSize, Determines the mechanism
LocalTime, used for unified and robust
4
Periodic, None logs to roll over to the next file
MaxLogFileSize 1–4294967295 Maximum size (in Kb) a log
can grow before rollover
occurs, if FileSize is the
rollover mode
RolloverPeriodIn 1–2147483648 Period of time (in seconds) a
Seconds log file is used before rollover
occurs, if Periodic is the
rollover mode
RolloverAtLocal 00:00–23:59 Time of day a log rollover
Time occurs, if LocalTime is the
rollover mode
NumberOfLogFiles 1–4294967295 Maximum number of log files
that can exist before the oldest
file is removed
LogRecycle True or False Enables or disables automatic
purging of old log files,
keeping the latest
NumberOfLogFiles
Note: A restart of the daemons or services is not required when enabling verbose
debug logs or when changing vxlogcfg options.

Sample vxlogcfg Output
To review NB product default log configuration settings, enter the following
command:
vxlogcfg -l -p 51216 -o Default
Output similar to the following is produced:
Configuration settings for originator 112, of product 51,
216...
LogDirectory = C:\PROGRA~1\VERITAS\NetBackup\logs\
DebugLevel = 6
DiagnosticLevel = 1
DynaReloadInSec = 0
LogToStdout = False
LogToStderr = False
LogToOslog = False
RolloverMode = FileSize
LogRecycle = False
MaxLogFileSizeKB = 5120
RolloverPeriodInSeconds = 43200
RolloverAtLocalTime = 0:00
NumberOfLogFiles = 3
OIDNames =
L10nLib = C:\PROGRA~1\VERITAS\NetBackup\bin\vxextwincat_1
.dll
L10nResourceDir = C:\PROGRA~1\VERITAS\NetBackup\resources
NtEventLogSourceName = Application
NtEventLogCategory = 0
LogFileSDDL = (null)

Debug Log Contents
• Debug logs are rarely captured as a first step in

troubleshooting.
• Debug logs provide low-level details on NetBackup
operations.
• Reproduce any failure with appropriate log settings in
place.
21
Topic 2: Capturing Debug Logs
4
After completing this topic, you will be able to use the vxlogmgr command to
prepare unified logs for sending to NetBackup Technical Support.
Debug Log Contents

Debug logs are generally engaged as a last resort. You can diagnose and resolve
most problems without the need for information from the debug logs. Unless
logging was already in place at the time of the failure, it is necessary to reproduce
an issue after configuring the logs.
The debug logs do, however, contain data that cannot be found elsewhere. Most
valuable when troubleshooting are the low-level errors and operational logic,
which are not visible using the job logs or reports.

Log Collection Considerations
Understanding the problem specifics and the operations of

NetBackup are essential for efficient log collection:
• Which NetBackup hosts are involved in the job failure?
• Which NetBackup tiers are affected? Is the job failure a
Master-, Media-, or Client-specific issue?
• Apply the NetBackup process flow to the nature of the
problem.
• Are system resources available for verbose log
processing and storage?
22
Log Collection Considerations

In general, it is better to collect too much information than too little. Debug
logging can significantly impact processing power and disk space usage, making
an “enable everything” approach undesirable in many cases. Performance factors
are particularly important on the master server, and on heavily loaded systems.
Consider the following for effective and efficient log collection:
• Specific NetBackup systems involved in the job failure
Determine the specific NBU systems involved in the job failure using the
Activity Monitor job log, and use this information to determine which NBU
systems should perform debug logging.
• Affected NetBackup tiers
Investigate the status code and other circumstances surrounding the job failure
to determine if the issue is related to the master server, media server, EMM
server, or client. Use this information to determine which systems should
perform debug logging.
• NetBackup process flow
CPU usage and space consumption increase with higher verbose logging
levels, and as more processes are logged. Use this information to determine
which debug logs to enable, or for which logs to increase the verbose level.
• System resources for log processing and storage
Log processing may substantially slow individual stream performance on a
client, or slow overall operations on a NBU server. Do not overload a system
already strained for resources. The unified log location may be changed.

Log Collection Example: Status Code
96
Message: Unable to allocate new media for backup, storage unit
has none available
nbemm Media bpbrm

nbrb
bptm
EMM
Master
nbjm
Clients:
Logging is not relevant
23
Log Collection Example: Status Code 96
4
This slide shows an example of where to create logs and which logs to create for a
given problem. The problem identified is a status code 96, indicating that
NetBackup does not have sufficient media to complete the backup operation.
1 Based on knowledge of this type of failure, and the roles of the various
NetBackup tiers, first eliminate the need to collect logs on the client side.
Failure to locate sufficient media is clearly a server-side problem.
2 Based on knowledge of process flow, determine which specific debug logs
should be enabled, or which logs should be examined first. In this example,
nbjm, nbrb, nbemm, MDS, bpbrm, and bptm play some role in media
allocation, or communicating with processes involved in media allocation.
For example, although bptm does not directly determine media availability, it
does report on failed media operations that may result in frozen media.
3 Determine which specific NetBackup hosts are involved in the failure.
In this example, the master server is able to successfully perform backups to
media, indicating that the master and EMM servers are not the most likely
sources of this problem. Focus your investigation on the media server involved
in the failing backup.

Providing Logs to Technical Support
• Source unified log files in unformatted

Raw unified log state
format:
• .log file in logs directory
Collecting raw • Manually from the logs directory

unified logs: • vxlogmgr command
Technical Support • Highest verbose setting

requirements: • Renamed legacy logs
24
Providing Logs to Technical Support
Raw Unified Log Format

The raw format of a unified log is simply the source log file in its unformatted
state. This data is available as .log files in the
/usr/openv/logs directory (UNIX) or the
install_path\NetBackup\logs folder (Windows).
NetBackup Technical Support almost always requests unified logs in their raw
format. This allows Technical Support the greatest flexibility when sifting through
the data, and helps avoid the need to resend data in a different format.
Collecting Raw Unified Logs

Collected raw logs by sending the actual unified log files that exist in the
NetBackup logs directory, or by extracting the raw data using the vxlogmgr
command. The alternative to collecting the raw logs is to create a text output of the
log data using the vxlogview command, which is discussed later in this lesson.
Renaming Legacy Logs

The default name for a legacy NetBackup log is simply the date the log was
written. It can be very helpful to Technical Support if the logs are renamed to
include the NetBackup role, hostname, and originating process. For example:
pc1train07.mediaserver.bptm.06172006.log

Using the vxlogmgr Command
The following are examples of using the vxlogmgr command to copy

raw unified logs:
• Copy all raw unified log entries from the last 12 hours:
– UNIX: vxlogmgr –c –t 12:00:00 –f /usr/tmp/logs
– Windows: vxlogmgr –c –t 12:00:00 –f C:\temp
• Copy all raw unified log entries from a specific time span:
– UNIX: vxlogmgr –c –b '07/28/07 9:34:00 AM'
–e '07/28/07 11:59:00 AM' –f /usr/tmp/logs
– Windows: vxlogmgr –c –b "07/28/07 9:34:00 AM"
–e "07/28/07 11:59:00 AM" –f C:\temp
• Copy raw log entries from originator ID 117 (nbjm) over the last
two days:
– UNIX: vxlogmgr –c –o 117 –n 2 –f /usr/tmp/logs
– Windows: vxlogmgr –c –o 117 –n 2 –f C:\temp
25
Using the vxlogmgr Command
4
The vxlogmgr command is used to manage unified log files. Log file
management includes actions such as copying, moving, or deleting log files. The
following is a partial list of parameters associated with the copy function (-c) of
the vxlogmgr command, with examples.
Originator ID

-o -o 116 Gathers logs for the specified originator ID
Time Period

-b -b "11/28/07 Copies log files created after the specified start
9:34:00 AM" date*
-e -e '11/28/07 Copies log files created before the specified end
4:38:00 PM' date*
-t -t 12:15:00 Copies logs created in the last x number of hours
-n -n 2 Copies logs created in the last x number of days
*Note: The date format is set from the current locale at run-time and is locale-
specific. In UNIX, use single quotes to enclose the date. In Windows, use double
quotes.

Destination

-f -f /usr/tmp/logs Copies the logs to the specified folder or directory
-f C:\temp The folder or directory must already exist. The -f
parameter is required when using -c to copy log
files.
Use the -s option in place of the –c option in the examples on the slide to list logs
meeting the specified criteria before actually copying them. For example, to view
the raw log entries for originator ID 117 over the last two days, enter:
vxlogmgr –s –o 117 –n 2

Disabling Debug Logging
Disable debug logs unless you are actively trying to log a failure.
• Delete the debug log folders in the

NetBackup log folder.
Disabling legacy
• Do not delete the AltPath or the
logging:
user_ops folders, if present.
• Do not delete the mklogdir script.
• Application level messages cannot be

disabled.
Disabling unified • Use vxlogcfg to reduce or disable
logging: diagnostic and debug level messages.
• Use vxlogmgr to purge existing logs to
reclaim disk space.
26
Disabling Debug Logs
4
Unless you are actively trying to capture a failure scenario, it is very important to
disable, or at least, reduce debug logging. Logs can have a tremendous impact on
NetBackup performance and disk space on the volume where logs exist.
Disable and purge legacy NetBackup logs by deleting the folder for the
corresponding logs in the NetBackup logs directory. The AltPath and
user_ops folders, if present, are part of normal NetBackup operations, and must
not be deleted. Also, do not delete the mklogdir script.
Unified logs cannot be completely disabled, but the amount of detail logged and
the corresponding performance overhead varies a great deal based on the verbose
settings.
The following commands disable debug and diagnostic unified log messages by
setting the verbose message levels to zero (0):
vxlogcfg –a –p 51216 –o ALL –s DiagnosticLevel=0
vxlogcfg –a –p 51216 –o ALL –s DebugLevel=0
Note: Changing the unified logging message level in the Host Properties only
affects debug messaging, and only for the nbpem, nbrb, and nbjm logs.
To reclaim disk space, use the vxlogmgr command. The following command
purges existing unified logs based upon the NumberOfLogFiles configuration
setting.:
vxlogmgr –d -a

When to View the Logs
• Debug logs store information about:

– Activity of the logged process
– Status received from other NetBackup processes
– Messages from the operating system (OS)
• Debug logs should be viewed when:
– Other troubleshooting methods have been exhausted
– Root cause analysis is needed
28
Topic 3: Viewing Debug Logs

• Extract relevant data from legacy logs.
• Use the vxlogview command to format data from raw unified logs.
When to View the Logs

NetBackup debug logs primarily store the operational details of a particular
process. The messages shown in the log reflect the actions the process is taking, in
chronological order. A log may also reveal status information received from other
processes in the NetBackup process flow, or errors and warnings received from the
operating system.
The degree of activity that is reported in the log is determined by the verbose
logging level.

Viewing Legacy Logs
Legacy NetBackup debug logs:

• Are written in plain text
• May be viewed by any text editor
• Are best managed using an editor with advanced search
and mark features
• Are written in the same format by UNIX and Windows
hosts
29
Viewing Legacy Logs
4
By default, legacy logs store a full day of logging in a single file per process, and
can grow quite large. Reading a log can become complicated when multiple
instances of a process are running and logging concurrently to the same log file.
When viewing the logs from a UNIX or Linux host, the inherent tools, such as
grep and vi, are often sufficient to manage the large, complex text files. From a
Windows host, it may be preferable to seek a third-party text editor with advanced
features for searching and marking lines. Isolating a specific subset of log entries
can greatly improve the readability of a log.
Note: The host in question is the system viewing the log files. It does not matter
whether a UNIX or Windows host originally wrote the log file.

Legacy Log Layout
• Debug log content format (UNIX)

Time Process ID Severity Message
Stamp (PID) Level Text
09:14:35.405[9554] <4> db_getSTUNIT:(-)

Translating EMM_ERROR_Success(0) to 0 in the
NetBackup context
• Debug log content format (Windows)

Time PID.Thread Severity Message
Stamp ID (TID) Level Text
09:14:35.405[2768.2008] <4> db_getSTUNIT:(-)

Translating EMM_ERROR_Success(0) to 0 in the
NetBackup context
30
Legacy Log Layout

This slide shows the fields in a legacy log layout on both UNIX and Windows:
• Timestamp
Timestamp indicates the time the entry was posted, based on the local system
clock, in HH:MM:SS format, including the millisecond. When troubleshooting
debug logs from multiple systems, be aware of time zone discrepancies.
• Process ID (PID) and thread ID (TID)
In UNIX, this field is the process ID (PID) of the process posting the log entry.
In Windows, this field is a number that includes both the process ID (PID) and
the thread ID (TID) of the process posting the log entry, in the format PID.TID.
NetBackup processes may run concurrent instances, performing different tasks
or the same task for two different backup jobs. For example, each data stream
in a client backup runs an instance of the bpbkar process on the client side.
• Message severity level
Every entry in a legacy NetBackup log has an assigned severity level. The
following chart identifies the severity code and its associated meaning:
1= Unknown 4= Information 16 = Error

2= Debug 8= Warning 32 = Critical
• Message text:
The message text contains the actual activity or message reported by the
logged process.

Extracting Relevant Data
It is often helpful, or even necessary, to extract only the relevant portion of a log
file before examination is done. This is especially the case for very large log files
with several process instances writing to the log concurrently.
4
Use the following procedure to extract a set of process IDs from legacy debug
logs.
1 Determine a relevant log and process ID (PID).
In this example, the log is bpbrm and the PID is 11340.
Note: The appropriate PID may be found in the detailed status of the Activity
Monitor for the failed job. You may also find the PID by navigating to the time
of the job start or failure within the log, or by searching the log for part of the
error message.

Original Log File
Select log entries
2 Open the original debug log file.

This example uses TextPad by Helios Software Solutions, but any powerful,
general-purpose text editor for plain text files is acceptable, regardless of
which operating system originally created the debug log file.
3 Select the log entries made by the identified PID.

New Text File
4
4 Copy and paste the selected entries to a new text file.
5 Repeat for all relevant PIDs.
This log excerpt can now be examined for clues regarding the job failure.

Viewing Unified Logs
Unified logs are:

• In a proprietary format, and are not meant to be viewed
directly
• Accessed from a system with NetBackup installed
• Read and formatted using the vxlogview command
• Sent to a text file by redirecting the vxlogview output
34
Viewing Unified Logs

Unified logs are stored in a proprietary format (with some text strings visible) and
are not intended to be viewed with a standard word processor such as Windows
Notepad or UNIX vi. Instead, the logs may be read using the vxlogview
command.
The vxlogview command is run with a set of arguments that determine which
unified logs to access, and how to display them.
Because vxlogview is the only way to initially view raw unified logs, raw
unified logs must be viewed on a system with NetBackup installed. In addition, the
NetBackup system must have the same licensed options as the original system for
all log entries to be visible.
By default, the vxlogview output is displayed at the command prompt. For ease
of use, redirect the output to a text file. Traditional methods for searching and
editing the log then become available, as with legacy logs.
NetBackup Technical Support almost always requests the raw unified logs so they
can rerun the vxlogview command as needed with different parameters.

Unified Log Layout
vxlogview -X jobid=147 -d all

Product ID Originator ID
Process ID (PID)
(short name) (short name)
Date Time Message Product ID Originator ID
Stamp Stamp Type (numeric) (numeric)
7/10/2006 12:03:33.765 [Debug] NB 51216 mds 143

PID:11876
[jobid=147] 1 [allocateTwin] EXIT STATUS =
2005000 (EMM_ERROR_MediaUnavailable, No media is
available)
Message Text
35
Unified Log Layout
4
The fields shown in a unified log are determined by the –d switch on the
vxlogview command. The excerpt shown on this slide was taken using the
–d all switch, which displays all available columns. By changing the
parameters used with –d, you may remove any of these columns. The columns
shown with the -d all switch are:
• Date: The date (mm/dd/yyyy) on which the log entry was posted
• Time: The timestamp (hh:mm:ss.millisecond) at which the log entry was
posted
• Message Type: The classification of the message as Application, Diagnostic or
Debug
• Product ID (short name): For example, NB = NetBackup
• Product ID (numeric value): For example, 51216 is NetBackup
• Originator ID (short name): For example, mds is MDS
• Originator ID numeric value: For example, 143 is MDS
• PID: Process ID of the process posting this log entry
• TID: Thread ID of the process posting this log entry
• Message Text: The content of this log entry
At present, unified log outputs do not report their verbose level or the
vxlogview command syntax used. When providing raw logs to NetBackup
Technical Support, specify which verbose level was used, and in the case of a log
output, what command syntax was used.

Unified Log Message Types
• Application message type:

02/17/06 14:05:37.461 [Application] NB 51216 nbpem
116 PID:23688 TID:5 [No context] [Error] V-116-86
[3] backup of client hog, policy mbw-streams-win,
is suspended: exceeded configured number of tries
for a schedule
• Diagnostic message type:
02/17/06 14:08:32.392 [Diagnostic] NB 51216 nbpem
116 PID:23688 TID:11 [No context] 1 V-116-71
[JobScheduler::doCatIncr] no configured session
based incremental catalog schedules
• Debug message type:
02/17/06 14:07:52.439 [Debug] NB 51216 nbpem 116
PID:23688 TID:7 [No context] 1
[JobScheduler::scheduleNextJob] setting timer for
1140206874 (Fri Feb 17 14:07:54 2006 ,fbf052bc),
jobs currently active 1(JobScheduler.cpp:2352)2
36
Unified Log Message Types

All messages posted to a unified log are classified as one of three message types:
Application, Diagnostic or Debug. A column displaying the message type can be
enabled in the log using the vxlogview command. Unified log message types
are:
• Application
Application messages are always displayed and cannot be disabled. These are
the most user-oriented of the three messages types. There are no verbose level
options available for application messages.
• Diagnostic
Diagnostic messages contain more information than Application messages.
The verbose level can only be configured using the vxlogcfg command.
Most processes log diagnostic messages at level 1 by default. Diagnostic
messages can be disabled by setting the verbose level to zero.
• Debug
Debug messages are very low-level operational details intended for use by
NetBackup Engineering to debug problems. The vxlogcfg command can
also be used. Most processes log debug messages at level 1 by default. Debug
messages can be disabled by setting the verbose level to zero. Configuring a
unified log verbose level using the NetBackup Host Properties in the
Administration console manipulates this message level for certain processes.

Using the vxlogview Command
• Format application level errors generated in the last 3

days:
vxlogview -L -n 3 > errors.log
• Format all entries for process ID 12284; display time,
message type, message severity, message text:
vxlogview -P 12284 -d T,m,s,x > pid12284.log
• Format logs for job ID 175; display date, time, message
type, originator, process ID, message text:
vxlogview –X jobid=175 -d D,T,m,o,p,x > job175.log
vxlogview only extracts information from

unified logs. 37
Using the vxlogview Command
4
Use the vxlogview command to selectively pull and format data from the raw
unified logs. The vxlogview command produces human-readable text output
that is concise and relevant to the problem. Unless the output is redirected to a text
file, as shown in the examples on this slide, the data is displayed on the screen.
The following tables describe the switches and parameters for the vxlogview
command.
Message Type
-L Shows all Application type messages

-I Shows all Diagnostic type messages
-D Shows all Debug type messages

Log Source
-p product_id Shows activity for the product ID

NetBackup is product ID 51216 (default) and PBX is
product ID 50936.
-a Shows all activity for all installed products (NBU and
PBX only) and can produce a very large amount of data
-o originator_id Shows activity for the specified originator ID
-P process_id Displays all log entries posted by the specified process ID
-X jobid=job_id Displays all log entries for the specified job ID
This is referred to as a context search. Job ID is the only
context search available. It only displays data from
nbjm, nbpem, nbemm, and nbrb.
Time Frame
-n number_of_days Limits the time frame of the log entries to the last number
of days specified
The vxlogview command starts at the beginning of the
current day, and counts backwards in 24-hour increments.
For example, if the command is run with the option -n 2
at 8:00 a.m., 32 hours of logging are displayed.
-t hh:mm:ss Limits the time frame of the log entries displayed from
the current time, looking backwards the amount of time
specified
-b "mm/dd/yy Limits the time frame of the log entries displayed to the
hh:mm:ss AM/PM" period of time specified with the begin (-b) and end (-e)
-e 'mm/dd/yy switches
hh:mm:ss AM/PM' Note: In UNIX, use single quotes to enclose the date. In
Windows, use double quotes.
Log Layout
-d Determines the fields to display for each log entry

Use -d all to display all fields.

For readability and to save space, display only the necessary fields with the -d
switch. The following shows common uses of the -d option:
D Date T Time
m Message type p Process ID
t Thread ID P Product ID
O Originator ID s Application entry severity
x Text of the message o Originator short name

vxlogview Command Considerations
Consider the following when running the vxlogview

command:
• vxlogview only formats log data that already exists.
• Ensure your rollover settings do not cause relevant log
data truncation.
• Configure verbose levels prior to reproducing the
problem.
• Redirect vxlogview output to a file.
• Customize vxlogview output using the -d flag.
38
vxlogview Command Considerations

Follow the suggestions on this slide before running the vxlogview command.

Lesson Summary
• Key Points
– In this lesson, you reviewed how to establish legacy logging for both
NetBackup and Media Manager. You also learned how to use the vxlogcfg
command to set both unified and robust log settings.
– You learned how to use the vxlogmgr command to prepare unified logs to
be sent to NetBackup Technical Support.
– You learned how to extract relevant data from legacy logs and how to use
the vxlogview command to format data from raw unified logs.
– The support Web site at: http://entsupport.symantec.com
– TechNote 279929
39
4
Lab 4: Using Debug Logs

• Enable debug logging.
• Configure debug log settings.
• Collect debug logs.
• Disable debug logging.
• View legacy debug logs (optional).
• View unified debug logs (optional).

• Appendix A provides step-by-step lab instructions. See “Lab 4 Details: Using
Debug Logs,” page A-32.
Solution: Using Debug Logs,” page B-37.

Lesson 5
Understanding the NetBackup Database
Lesson Introduction
Tools
Database
Issues
Issues
Topic After completing this lesson, you will be able to:

Topic 1: The NBDB • Identify the components of the NBDB and become familiar
with the commands to run, maintain, or troubleshoot the
database.
• Describe the steps NetBackup takes to install the Sybase
server and create the NBDB database.
Topic 2: The EMM Domain Identify the components of the EMM domain and view excerpts
of component logs to develop a familiarity with the EMM server
interactions.
Topic 3: Client Backup Process Trace a successful job as it flows through the EMM server
Flow Through the EMM Server components, in order to detect problems with nbrb, MDS,
nbemm, DA, and the NBDB.
Topic 4: Catalog Backup and Back up, recover, and protect the NBDB.
Recovery
Topic 5: Maintaining the NBDB Perform various tasks to maintain the NBDB.

The NetBackup Database (NBDB)
NetBackup Database (NBDB) running on Sybase

Adaptive Server Anywhere (ASA) 9.0.1
NBDB.db • Libraries, drives, server.conf NBDB.log.n

and drive density
EMM_DATA • Volume pools
and media vxdbms.conf
.db
• Storage units
EMM_INDEX • Allocation databases
.db information .conf
Database Configuration Transaction

Files Files Logs
Topic 1: The NetBackup Relational Database (NBDB)

• Identify the components of the NetBackup Relational Database (NBDB) and
become familiar with the commands to run, maintain, or troubleshoot the
database.
• Describe how the NBDB is created during a master server installation.
5
The NetBackup Database (NBDB)
The NetBackup Relational Database (NBDB) is created during the installation of
the master server. The NBDB stores information used by both the system and the
EMM server. The NBDB runs on Sybase Adaptive Server Anywhere (ASA) 9.0.x.
The NBDB consists of three database files and is supported by configuration files
and a transaction log.
Database Files
The database files contain information used by the Sybase server, NetBackup
daemons or processes and services, and the EMM server. Each of these files is
considered a dbspace. The database files that compose the database are:
• NBDB.db
Used by the Sybase server and some NetBackup daemons, services, or
processes
• EMM_DATA.db
Accessed by the EMM server (nbemm)
• EMM_INDEX.db
Lesson 5 Understanding the NetBackup Database 5–3

Used to access information efficiently in EMM_DATA.db
Note: Do not install two instances of Adaptive Server Anywhere (ASA) on the
same Netbackup server. For example, do not install NOM, which installs its
own Sybase Server, on the EMM Server, which also installs its own Sybase
Server.
Configuration Files
The configuration files are used for startup and during other operational tasks. The
configuration files include:
• server.conf
• vxdbms.conf
• databases.conf
The Transaction Logs

The transaction logs (NBDB.log.n) contain information about changes that have
been made to the NBDB since it was last backed up. The transaction logs are
required to recover the NBDB.
After a catalog backup, the transaction logs are truncated.

NBDB-Related Commands
Scenario Command
Operations are failing because the NBDB nbdb_ping

cannot be accessed. Is it alive?
During database recovery or maintenance, nbdb_admin –start
you are directed to start or stop only the nbdb_admin –stop
NBDB.
Technical Support has directed you to nbdb_admin –auto_start NONE

rebuild your database because the current
database is corrupt. As part of the rebuild
procedure, you must disable auto_start.
You are navigating to one of the database nbdb_admin -list

or configuration files, but you cannot
remember the path.
NBDB-Related Commands
This slide shows common NBDB-related commands.
Commands that start all daemons or processes and services, such as
netbackup start (UNIX) and bpup (Windows), also start the NBDB.
Commands that stop all daemons or processes and services, such as
5
netbackup stop (UNIX) and bpdown (Windows), also stop the NBDB.
By default, when NetBackup starts, the Sybase server automatically starts the
NBDB (NBDB autostart). Use the nbdb_admin -auto_start NONE
command to prevent the database from being started.

Determining if the Sybase Server Is
Running
Daemon or Service Name bpps Output

UNIX NetBackup Database Daemon NB_dbsrv
Windows Adaptive Server Anywhere dbsrv9

VERITAS_NB
Determining if the Sybase Server Is Running

NB_dbsrv (UNIX) and dbsrv9 (Windows) are the daemon or service for the
Sybase server and must be running at all times for proper NetBackup operation. In
UNIX, the daemon is commonly called the NetBackup Database Daemon. In
Windows, the service is named Adaptive Server Anywhere VERITAS_NB.

ASA-Related Commands (UNIX)
Determine if ASA is running, bpps (Look for NB_dbsrv)

nbdbms_start_server –stat
Start ASA, nbdbms_start_server

nbdbms_start_stop start
Stop ASA, nbdbms_start_server -stop

nbdbms_start_stop stop
Stop ASA with force, nbdbms_start_server –stop -f
ASA-Related Commands
This slide shows common ASA-related commands. In addition, you can use the
NetBackup Activity Monitor to determine if ASA is running, or to start or stop
ASA.

Important NBDB Directories, Folders,
and Files
Database-Related
Location
Element
NBDB commands UNIX: /usr/openv/db/bin
Windows: install_path\VERITAS\NetBackup\bin
NBDB files (NBDB.db, UNIX: /usr/openv/db/data
EMM_DATA.db, Windows: install_path\VERITAS\NetBackupDB\data
EMM_INDEX.db, NBDB.log)
and vxdbms.conf
Sybase server log file UNIX: /usr/openv/db/log
(server.log) Windows: install_path\VERITAS\NetBackupDB\log
SQL Scripts to create the UNIX: /usr/openv/db/scripts

database schema Windows:
install_path\VERITAS\NetBackupDB\scripts
Temporary staging UNIX: /usr/openv/db/staging
directory for backing up Windows:
and recovering NBDB files install_path\VERITAS\NetBackupDB\staging
Configuration files UNIX: /usr/openv/var/global
server.conf and Windows: install_path\VERITAS\NetBackupDB\conf
databases.conf
Important NBDB Directories, Folders, and Files

Use this slide as a quick reference for the location of important directories, folders,
and files related to the NBDB.
Note that the nbdb commands are in different directories on UNIX and Windows.

The NBDB Creation Process
1. Install 2. Create the 3. Modify

Sybase ASA server.conf bp.conf or the
9.0.1. file. Registry.
4. Create the 5. Create the

6. Create the
vxdbms.conf databases.conf
NBDB.
file. file.
9. Populate
8. Set up the
7. Create the mapping and
database
NBDB files. attribute
schema.
information.
The NBDB Creation Process

The steps shown on this slide are performed to create the NBDB during
NetBackup master server installation.
The NBDB must be created on the system that is the EMM server, which is the
master server by default. This default can be changed during the installation of the
EMM server, and the NBDB can be moved later.
5
For more detailed information about the steps shown on this slide, see the NBDB
Creation Process appendix at the back of the Lab Guide for this course.

The EMM Domain
EMM
Server
Master Master
Server Server
Media Media
Servers Servers
Topic 2: The Enterprise Media Manager (EMM) Domain

After completing this topic, you will be able to identify the components of the
EMM domain.
The EMM Domain

An EMM domain consists of an EMM server and NBDB and one or many master
servers and their corresponding media servers.
There is one EMM server per EMM domain. Both the EMM server and the NBDB
must be on the same system. By default, the EMM server is on the master server,
but this can be changed, either during installation or later. If the EMM server is not
a master server, it is then considered to be a media server.

EMM Domain-Related Commands
If you want to know … Then enter …
Which master and media servers nbemmcmd -getemmserver nbemmcmd

are members of an EMM domain, –listhosts -verbose
EMM Domain-Related Commands

To view all master and media servers in an EMM Domain, enter:
nbemmcmd -getemmserver
The command output is similar to the following:
5
To list table information from the NBDB about the systems in the EMM domain,
enter:
nbemmcmd –listhosts -verbose

The command output is similar to the following. Notice the machine type.
The MachineNbuType is defined as follows:

• 1 = media server
• 3 = master server
• 6 = EMM server

The EMM Server
NBDB
nbemm
components
nbemm
OID=111
nbrb MDS DA
nbjm emmlib
OID=118 OID=143 OID=144
nbproxy
bpdbm
GUI/ Scan
CLI hosts
Legacy
Master Server processes
Catalogs
The EMM Server

The EMM server communicates with the NBDB for functions, such as data
definition (creating a storage unit), data manipulation (deleting media), and data
access (allocating a drive). This slide shows the components of the EMM server.
As you can see, nbemm, nbrb, Media and Device Selection (MDS), and Device
Allocator (DA) log activity under their own originator IDs (OIDs).
5
If there is a problem with EMM server components, you may see Resources
Cannot be Allocated or Resources Unavailable messages (status
code 800). To troubleshoot these types of errors, you must understand the
functions of the EMM server components, the role they play in backup and restore
operations, and what to look for in their log files.
EMM Server Communication

Normal communication with the EMM server is performed through PBX port
1556.
If NetBackup Access Control is configured:
• Port 2821 is used to communicate with the VxSS Authentication Server.
• Port 4032 is used to communicate with the VxSS Authorization Server.
Communication to bpdbm
The NetBackup Notification Service (nbnos) and the NetBackup Job Manager
(nbjm) are used by the EMM server to connect with bpdbm to obtain resource
information, such as images information and policy details, which are not stored in
the NBDB.

nbemm
NBDB
nbemm
components
nbemm
OID=111
nbrb MDS DA
nbjm emmlib
OID=118 OID=143 OID=144
nbproxy
bpdbm
If you want to … Then enter …
Master Server Start nbemm, nbemm

Catalogs
Stop nbemm, nbemm -terminate
nbemm
nbemm queries and modifies the NBDB by sending SQL statements to the Sybase
server’s database engine for execution. nbemm interacts with the NBDB for the
following operations:
• Configuring resources (devices, storage units, volume pools, media)
• Allocating and deallocating resources
• Displaying resource information through the command line and the GUI
• Changing or deleting resources
nbemm must always be running on the EMM server. To determine if nbemm is
running, use the Activity Monitor or the bpps command.

Example: SQL Statements Sent to the
Database
nbemm
vxlogview –o 111
Example: SQL Statements Sent to the Database

nbemm logs its activity under originator ID 111. To see nbemm activity, enter:
vxlogview –o 111

nbrb
If you want to … Then enter … NBDB
Start nbrb, nbrb

nbemm
Stop nbrb, nbrb –terminate components
Suspend nbrb, nbrbutil –suspend nbemm
OID=111
Resume nbrb, nbrbutil -resume
nbrb MDS DA
nbjm emmlib
OID=118 OID=143 OID=144
nbproxy
bpdbm
Master Server
Catalogs
nbrb
The Resource Broker (nbrb) manages resource requests and allocations and
monitors configuration changes. nbrb receives resource requests from the Job
Manager (nbjm) and passes allocation requests to media and device selection
(MDS). When jobs are complete, nbrb notifies MDS that allocated resources can
be released.
nbrb also monitors configuration changes by performing a full evaluation of the
configuration every 30 minutes and a partial evaluation every 5 minutes.
nbrb must always be running on the EMM server, even though nbrb is a part of
the Intelligent Resource Manager (IRM). To determine if nbrb is running, use the
Activity Monitor or the bpps command.

Example: Allocated Resources
Released
nbrb
vxlogview –o 118
Example: Allocated Resources Released

nbrb logs its activity under originator ID 118. To see nbrb activity, enter:
vxlogview –o 118

Media and Device Selection (MDS)
NBDB
nbemm
components
nbemm
OID=111
nbrb MDS DA
nbjm emmlib
OID=118 OID=143 OID=144
nbproxy
bpdbm
Master Server
Catalogs
Media and Device Selection (MDS)

MDS receives requests from the resource broker (nbrb) to allocate resources for
backup and restore operations. MDS determines which resources to allocate and
passes the information to nbemm, where the appropriate SQL statements are
generated and sent to the NBDB.
The resource broker also notifies MDS when resources can be deallocated. MDS
passes this information to nbemm, where the appropriate SQL statements are
generated and sent to the NBDB.

Example: MDS Deallocating
Resources
MDS
vxlogview –o 143
Example: MDS Deallocating Resources

As a component of nbemm, MDS is not visible as a running process or service.
MDS activity is logged and can be viewed using originator 143.
To see MDS activity, enter:
vxlogview -o 143

emmlib
NBDB
nbemm
components
nbemm
OID=111
nbrb MDS DA
nbjm emmlib
OID=118 OID=143 OID=144
nbproxy
bpdbm
GUI/
CLI
Legacy
Master Server processes
Catalogs
emmlib
emmlib allows legacy processes and services to communicate with the NBDB.
(Legacy processes and services typically start with bp.) emmlib is also the path
for command-line-initiated display and change operations to be sent to nbemm.
As a component of nbemm, emmlib is not visible as a running activity, and
emmlib does not log to an originator ID (OID).

The Device Allocator (DA)
nbemm NBDB
Heartbeat
DA
Media Media Media Media

Server Server Server Server
Scan Host
Shared
Tape
Drives
The Device Allocator (DA)

The DA manages the scan hosts for shared drives. The DA also monitors devices
through the media servers that have been assigned as scan hosts for shared devices.
A shared device may have only one scan host.
Upon startup, a persistent connection is established between the DA and the
NBDB and between the DA and all the media servers. Heartbeats are issued every
5
five minutes to keep connections open. The persistent connection is used to start
and stop scan host functions and dynamically reassign scan host responsibilities to
a different media server.
When a drive has been allocated, the DA notifies the scan host to stop scanning
that drive. When the drive is deallocated, the DA notifies the scan host to start
scanning the drive again. If a scan host has a problem, the EMM server is notified
and can dynamically reassign a new scan host. The new scan host does not need to
register or provide any configuration information.

Example: Displaying the Scan Path for
a Drive
DA
vxlogview –o 144
Example: Displaying the Scan Path for a Drive

As a component of nbemm, DA is not visible as a running process. DA activity is
logged using originator 144. To see DA activity, enter:
vxlogview –o 144

EMM Server Client Backup Operation
Process Flow (1)
Master Server nbpem
1
nbjm
2
EMM Server
nbrb 3 MDS
5
4
nbemm 4 NBDB
1
2
3
4
5
Topic 3: Client Backup Process Flow Through the EMM Server

After completing this topic, you will be able to trace the flow of a client backup job
through the EMM server.
EMM Server Client Backup Operation Process Flow (1)

1 nbpem tells nbjm that a policy needs to run.
5
2 nbjm sends a resource request to nbrb for backup job xx.
nbrb assigns a resource request ID to the resource request.

3 nbrb passes the resource request to MDS.
4 MDS works with nbemm to start a database transaction to allocate the

requested resources.

5 MDS passes status back to nbrb indicating success in allocating resources.
nbrb assigns allocation IDs xxx to the allocated resources.

EMM Server Client Backup Operation
Process Flow (2)
Master Server nbpem
nbjm 7
8
EMM Server 6
nbrb 9 MDS
10
nbemm 10 NBDB
1
2
3
4
EMM Server Backup Operation Process Flow (2)

6 nbrb notifies nbjm that resources for the resource request xxx have been
found.
7 nbjm initiates the job on the media server. The commands are now sent to
bpbrm and job processing continues as usual.
8 nbjm tells nbpem and nbrb that the job was successful.
9 nbrb notifies MDS that allocation IDs xxx, xxx, xxx have been released.
10 MDS works with nbemm to start a database transaction to deallocate the

resources.

Displaying Allocations and Orphaned
Resources
If you … Then use …

Want to determine if the nbrbutil –dump
EMM server components
were able to allocate
resources,
Suspect that resources nbrbutil –listOrphanedMedia

have not been nbrbutil -listOrphanedDrives
successfully nbrbutil –listOrphanedStus
deallocated,
Displaying Allocations and Orphaned Resources

When you are working with Technical Support on request and allocation issues,
you may be directed to use the nbrbutil command. This slide shows the
commands used to display current allocations and orphaned resources.
When large amounts of output are expected, or if you intend to send the output to
Technical Support, redirect the output of the nbrbutil command to a file.
5
The following table defines terminology that is used in parameters in the
nbrbutil command:
Terminology Description
Orphaned media Media that has been reserved in the NBDB, but has not been
allocated
Orphaned drives Drives that have been reserved in the NBDB, but have not
been allocated
Orphaned storage units Storage Units that have been reserved in the NBDB, but
(STUs) have not been allocated

Determining if Resources Were
Allocated
nbrbutil -dump
Determining if Resources Were Allocated

The slide shows the output of the nbrbutil -dump command with system
activity.
With no system activity, the nbrbutil -dump command produces the
following output:

Troubleshooting Resource Allocation
and Deallocation Issues
If you suspect resource allocation or deallocation issues, engage
Technical Support. Only run the following commands under the
supervision of Technical Support.
If Technical Support
Then use …
instructs you to …
Release all resource requests nbrbutil -resetAll

and allocations,
Release a specific resource nbrbutil –release request_ID
request,
Release a specific resource nbrbutil –release allocation_ID
allocation,
Release all resource requests nbrbutil –resetMediaServer media_server
for a specific media server,
Release orphaned resources, nbrbutil –ReleaseOrphanedMedia media_name
nbrbutil –ReleaseOrphanedDrives drive_name
nbrbutil –ReleaseOrphanedStus STU_name
Troubleshooting Resource Allocation and Deallocation Issues

This slide shows common nbrbutil commands used to release resource
requests and allocations.
CAUTION Use the commands shown on this slide only under the direction of
5
Technical Support.

Performing a Hot Online Catalog
Backup
NBDB files Temporary Storage

1 (Staging)
Configura-
1
tion files
2
NBDB.log 1
~ ~ ~
~ ~ ~ 4 Truncated
~ ~ ~
Image DB 3
and Other
Files
1
2
3
4
Topic 4: Catalog Backup and Recovery

After completing this topic, you will be able to back up, recover, and protect the
NBDB.
Performing a Hot, Online Catalog Backup

This graphic describes what happens during a hot, online catalog backup.
1 A parent job copies the NBDB files to /usr/openv/db/staging (UNIX)
or install_path\VERITAS\NetBackupDB\staging (Windows).
The NBDB files that are copied include:
NBDB.db servers.conf NBDB.log.n

EMM_DATA.db vxdbms.conf
EMM_INDEX.db databases.conf
2 A child job starts, which backs up the files from the staging directory or folder
to the storage unit specified in the catalog backup policy. This job backs up the
files in a single stream. The files do not remain in the staging directory or
folder; they are deleted automatically.
3 Another child job starts, which backs up files from the following directories:
– UNIX:
/usr/openv/netbackup/db
/usr/openv/var
/usr/openv/netbackup/vault
/usr/openv/var/global

– Windows:
install_path\VERITAS\NetBackup\var
install_path\VERITAS\NetBackup\db
install_path\VERITAS\NetBackup\vault
Note: If you watch a catalog backup in the Activity Monitor, the job details
only show that EMM_DATA.db is being backed up.
4 After the catalog backup completes, the transaction log (NBDB.log) is
truncated in order to clear database transactions that have been backed up.
Note: At the completion of a hot catalog backup, a disaster recovery (DR) file is
written to a user-defined directory on the master server. This file is required
for successful catalog recovery. For additional protection, configure
NetBackup to e-mail the DR file to one or more valid e-mail addresses.

Catalog Backup Files
After the catalog backup completes, five files exist at
/usr/openv/netbackup/db/images/master_server/c_time
(UNIX) or install_path\VERITAS\NetBackup\db\images\
master_server\c_time (Windows). The files are header files (FULL) and .f
files as follows:
• One header file belongs to the parent job. This header file contains CLIENT_
TYPE = 35 and TIR_EXPIRATION = 0.
• One header file and one .f file belong to the first child job, which is the job that
backed up the NBDB files. This header file contains CLIENT_TYPE = 7 and
TIR_EXPIRATION = 0.
• One header file and one .f file belong to the second child job, which is the job
that backed up the Image database and other files. This header file contains
CLIENT_TYPE = 35 and TIR_EXPIRATION > 0.
The catstore and tmp directories hold the files until the catalog backup
completes successfully.
If the NBDB and EMM servers have been installed on or moved to a media server,
the first child job runs on the media server. In this case, there is additional logging
for the job in the admin log on the media server.
You may view the logs for the catalog backup in the bpdbm logs.

Recovering the NBDB Files After a
Hot, Online Catalog Backup
NBDB files Temporary Storage

2 (Staging)
Configura-
4
tion files
3
1
NBDB.log 2
~ ~ ~
~ ~ ~
~ ~ ~
• NetBackup Catalog Recovery Wizard

• bprecover -wizard
Best Practice: Recover both the NBDB

files and the image files.
1
2
3
4
Recovering the NBDB Files After a Hot, Online Catalog Backup

To recover the NBDB files, you must perform a catalog recovery that recovers all
files. This is the recommended recovery method.
CAUTION You must recover the catalogs to a system running the same version,
5
including the MP levels. For example, a catalog backup taken on a
NetBackup 6.0 system and recovered to a NetBackup 6.0 MP3
system fails with a schema mismatch error. Likewise, a catalog
backup taken on a NetBackup 6.0 MP3 system cannot be recovered
to a NetBackup 6.0 system.
It is a best practice to back up the catalogs prior to and immediately after any
upgrade, including Maintenance Packs.
The graphic on this slide describes what happens when you use either the
NetBackup Catalog Recovery wizard or the bprecover -wizard command to
recover the NBDB files after a hot, online catalog backup.
1 The NBDB files are written to the staging directory at
/usr/openv/db/staging (UNIX) or
install_path\VERITAS\NetBackupDB\staging (Windows).
2 The database files are moved to /usr/openv/db/data (UNIX) or
install_path\VERITAS\NetBackupDB\data (Windows).
3 Transactions recorded in the NBDB.log are applied.
4 The configuration files are created.

Recovering Without the Disaster
Recovery File: The Image DB Is Intact
Step Action
1 Navigate to the master server’s Images directory.
2 Locate and copy the appropriate catalog header file (see

notes).
3 Use the full path of that copy to recover the catalogs using the
Catalog Recovery Wizard or the bprecover -wizard
command.
Recovering Without the Disaster Recovery File: The Image Database

Is Intact
The procedure shown on this slide describes how to recover when your disaster
recovery file is unavailable, but the Image database is intact. In this case, you may
use the header file from the second child job of the catalog backup as the disaster
recovery file.
1 Navigate to the master server’s Images directory as follows:
– UNIX:
/usr/openv/netbackup/db/images/master_server/c_time
– Windows:
install_path\VERITAS\NetBackup\db\images\
master_server\c_time
This directory contains catalog backup header files and .f files. The header
files are identified as cat_policyname_ctime_scheduletype, for
example, Catalog_1147351620_FULL.
2 View the most recent FULL or INCR catalog backup header file. You are
looking for the header file from the second child job of the catalog backup.
Look at the CLIENT_TYPE and the TIR_EXPIRATION. If the
CLIENT_TYPE is 35 and the TIR_EXPIRATION is greater than zero, you
may use this file. If these criteria are not met, use the next most recent header
file that meets these criteria.
3 After you have located a usable catalog backup header file, copy the header file
to a temporary directory. Record the full path to that copy.

4 Recover the catalogs using the Catalog Recovery Wizard or the
bprecover –wizard command.
When prompted, type the full path of the copy of the header file.

Step 2: The Catalog Backup Header File
This slide shows the most recent FULL or INCR catalog backup files. The header
file for the second child job is highlighted. Within that file, you can see that the
CLIENT_TYPE equals 35 and the TIR_EXPIRATION is greater than zero.
Windows only
On Windows only, copy the header file to a different location.

NBU 6.0 – Recovering Without the DR
File: The Image DB Is Not Intact
Step Action
Perform a Phase 1 and a Phase 2 import of the

1
catalog backup media.
2 Navigate to the master server’s Images directory.
Locate copy the appropriate catalog header file

3
(see notes).
4 Recover the catalogs using that copy.
NBU 6.0—Recovering Without the Disaster Recovery File: The Image

Database Is Not Intact
The procedure shown on this slide describes how to recover in NetBackup version
6.0 when both the disaster recovery file and the image database are not available.
To do this, you must know:
• The media ID of the most recent full catalog backup
5
• If the catalog backup spanned tapes
• The media ID of any incremental catalog backups that occurred after the last
full backup

NBU 6.0 – Step 1: Phase 1 and Phase
2 Media Import
bpimport –create_db_info –id media_id

bpimport
NBU 6.0—Step 1: Phase 1 and Phase 2 Media Import

Perform the following steps to recover:
1 Perform a Phase 1 and a Phase 2 import of the catalog backup media as
follows:
a Phase 1: bpimport -create_db_info –id media_id
b Phase 2: bpimport
2 Follow the steps in the “Recovering Without the Disaster Recovery File: The
Image Database Is Intact” section.

NBU 6.5 – Recovering Without the DR
File: The Image DB Is Not Intact
Step Action
Create a temporary disaster recovery (DR)

1
directory.
Perform an import of the DR file into the

2
temporary directory.
3 Locate the header file in the temporary directory.
Recover the catalogs using the full path of the

4
imported header file.
NBU 6.5—Recovering Without the Disaster Recovery File: The Image

Database Is Not Intact
The procedure shown on this slide describes how to recover in NetBackup version
6.5 when both the disaster recovery file and the image database are not available.
To do this, you must know:
• The media ID (tape) or the path (disk) of the most recent full catalog backup
5
• If the catalog backup spanned tapes
• The media ID of any incremental catalog backups that occurred after the last
full backup

NBU 6.5 – Step 2: Importing the DR
File (UNIX)
bpimport –drfile media_id

–drfile_dest temp_dr_directory
NBU 6.5—Step 2: Importing the DR FIle

Perform the following steps to recover:
1 Enter the following command to perform an import of the disaster recovery file
from the catalog backup media:
bpimport -drfile media_id
-drfile_dest temp_dr_directory
The disaster recovery file import automatically includes the Phase 1 and Phase
2 import.
2 Follow the steps in the “Recovering Without the Disaster Recovery File: The
Image Database Is Intact” section specifying the imported DR file.

Performing a Cold, Offline Catalog
Backup
1 The Sybase Server is queried.
2 The NBDB is shut down.
NBDB files
3
Configura-
tion files 3
NBDB.log
~ ~ ~ 3
~ ~ ~
~ ~ ~4 Truncated
Image DB
and other 3
files
5 The NBDB is started.
5
1
2
3
4
Performing a Cold, Offline Catalog Backup

This graphic describes what happens during a cold, offline catalog backup. Keep in
mind that a hot, online catalog backup is the recommended method for protecting
your catalog.
During a cold, offline catalog backup, no activity is able to occur because the
database is shut down.
5
The slide shows what happens when you perform a cold, offline catalog backup.
1 The Sybase Server is queried for the location of the NBDB files.
2 The NBDB is shut down, but the Sybase Server daemon or service continues to
run.
3 The NBDB files, the Image database files, and the other files are backed up.
The NBDB files that are backed up include:
NBDB.db servers.conf NBDB.log

EMM_DATA.db vxdbms.conf
EMM_INDEX.db databases.conf
The images and the other files that are backed up include:
– UNIX:
/usr/openv/db/data
/usr/openv/netbackup/db
/usr/openv/var

/usr/openv/netbackup/vault
/usr/openv/var/global
– Windows:
install_path\VERITAS\NetBackupDB
install_path\VERITAS\NetBackup\var
install_path\VERITAS\NetBackup\db
install_path\VERITAS\NetBackup\vault
4 The transaction log is truncated in order to clear database transactions that
have been backed up.
5 The NBDB is started.

Configuring and Running a Cold, Offline Catalog Backup
To configure a cold, offline catalog backup, on the NetBackup Administration
Console select Catalog—>Actions—>Configure offline NetBackup Catalog
Backup. The NBDB files are included automatically in the list of catalog file
locations under the Files tab.
5
To run a cold, offline catalog backup, select Catalog—>Actions—>Perform
offline backup of the NetBackup catalog.
If you are using the Vault option to back up catalogs, you can only perform hot,
online catalog backups.

Recovering the NBDB Files from a
Cold Offline Catalog Backup
Step Action
1 Re-create the NBDB directories, if necessary.
2 Load the cold catalog backup tape.

Note: Use bprecover -l to identify a cold catalog
backup tape.
3 On the master server, stop nbsvcmon, bprd, bpdbm, ltid,
and vmd.
4 On every media server, stop nbsvcmon, ltid, and vmd.
5 Run the bprecover -r command.
6 Stop and restart NetBackup.
Recovering the NBDB Files from a Cold, Offline Catalog Backup

The procedure on this slide describes how to recover the NBDB from a cold,
offline catalog backup. Keep in mind, however, that a hot, online catalog backup is
the recommended method for protecting your catalogs.
1 If you moved the NBDB files after the catalog backup, re-create the directories
where the files existed at the time of the catalog backup.
2 Load the cold catalog backup tape manually into the recovery device or use
robtest to move the tape from the library to the drive. If you use robtest,
be sure to exit after loading the drive.
If you do not know the media ID of the most recent cold catalog backup tape,
determine which tape holds the most recent cold catalog backup as follows:
a Record the media IDs of all tapes in the NetBackup volume pool. Cold
catalog backup tapes always use the NetBackup volume pool.
b Type the following command for each tape:
bprecover -l -m media_ID -d media_density
c If the tape does not contain a cold catalog backup, the following message is
displayed:
Specified media or path does not contain a valid NB
database backup header

d If the tape does contain a cold catalog backup, output similar to the
following is displayed:
On the master server, stop nbsvcmon, bprd, bpdbm,

ltid, and vmd as follows:
– UNIX: Run the following commands in order on the master server:
nbsvcmon –terminate (stops nbsrvmon)
bprdreq –terminate (stops bprd)
bpdbm –terminate (stops bpdbm)
stopltid (stops ltid)
vmctrldbm –t (stops vmd)
– Windows: Stop the following services:
5
› Service Monitor (nbsrvmon)
› Request Manager (bprd)
› Database Manager (bpdbm)
› Device Manager (ltid)
› Volume Manager (vmd)
3 On every media server, stop nbsvcmon, ltid, and vmd.

4 Issue the bprecover -r -tpath device_path command.

Recovering When the NBDB Is
Corrupt
Step Action
1 Record the media ID and density of the most recent hot or cold
catalog backup.
2 Stop NetBackup.
3 Prevent the Sybase server from starting the NBDB automatically.
4 Start the Sybase server.
5 Re-create the NBDB.
7 Populate the database with device mappings and attributes.
8 Configure a robot and drive.
9 Inventory the robot.
10 Restore the NBDB files.
12 Reconfigure the devices, if necessary, and reinventory the robots.
Recovering When the NBDB Is Corrupt

Use the following procedure to recover when the NBDB becomes corrupt. In
general, first create a new NBDB and then perform a recovery of only the NBDB
files.
5
CAUTION Only perform this procedure under the direction of Technical
Support.
1 Record the media ID and density of the most recent catalog backup; it can be
from a hot or cold catalog backup.
2 Stop NetBackup.
3 Prevent the Sybase server from starting the NBDB automatically as follows:
nbdb_admin -auto_start NONE
4 Start the Sybase server as follows:
– UNIX: nbdbms_start_stop start
– Windows: bpup -e ASANYs_VERITAS_NB

5 Re-create the NBDB as follows:
– UNIX: create_nbdb -drop NBDB
– Windows:
create_nbdb –db_server VERITAS_NB_servername -drop
CAUTION: Only use this command under the direction of Technical Support.

7 Populate the database with device mappings and attributes as follows:
a Download and extract the latest device mappings package from the
Symantec Support Web site.
b NetBackup 6.0: Type tpext on the EMM server.
NetBackup 6.5: Type tpext -loadEMM on the EMM server.
c Type tpext -get_dev_mappings on the EMM server.
d Type tpext -get_dev_mappings_ver on the EMM server to
confirm the registration of the new mappings.
e Restart ltid on each media server.
Note: After the catalogs have been completely recovered, verify that the EMM
Device Mappings are in sync with remote media servers. Type
tpext -get_dev_mappings_ver on each media server and update
media server mappings as necessary.
8 Configure a robot and a drive.

9 Inventory the robot.
10 Restore the database files as follows:
bprecover –r –nbdb -m media_id -d media_density

12 If necessary, reconfigure the devices and perform a robotic inventory.

Backing Up and Recovering the NBDB
Files Manually
Back up the NBDB files manually nbdb_backup –online

while they are online, destination_directory
Back up the NBDB files manually nbdb_backup –offline

when they are offline, destination_directory
Recover the NBDB files manually, nbdb_restore –recover

source_directory
Backing Up and Recovering the NBDB Files Manually

You can manually back up the NBDB files to disk and then restore them. Keep in
mind, this is not a substitute for doing normal catalog backups, but merely a tool
for providing an additional backup of the NBDB database.
Backing Up the NBDB Files Manually

To back up the NBDB files manually while they are online, use:
nbdb_backup –online destination_directory
To back up the NBDB files manually when they are offline, use:
nbdb_backup –offline destination_directory
During an offline backup, the NBDB is stopped.
The destination directory must exist before running the nbdb_backup
command. If this directory does not exist, the command fails. In addition, the
transaction log is not truncated when the nbdb_backup command is used.
Note: A job does not appear in the Activity Monitor when the commands on the
slide are executed.

Recovering the NBDB Files Manually
To recover the NBDB files manually, enter:
• UNIX: nbdb_restore –recover /source_directory
• Windows: nbdb_restore –recover source_directory
This command only recovers the NBDB files that were backed up using the
nbdb_backup command.

Creating a Mirrored Transaction Log
Step Action
1 Perform a full catalog backup.
2 Stop all NetBackup daemons or services and processes.
3 Start the Sybase daemon or service.
4 Create the mirrored log files.
5 Start the NetBackup daemons or services and processes.
Creating a Mirrored Transaction Log

You may want to create a mirrored transaction log to provide extra protection, as
the transaction log is critical to recovering the database. To do this, perform the
following steps:
3 Start the Sybase daemon or service:
4 Create the mirrored log file:
nbdb_move –data -mlog new_mirrored_log_directory
The directory must exist before running the nbdb_move command.

After the move, the mirrored log file is named NBDB.m.log. The name of the
original transaction log, NBDB.log, does not change.

Changing the Location of the NBDB
Files
Step Action
3 Start the Sybase daemon or service.
4 Move the files:

nbdb_move –data new_data_directory
–index new_index_directory
–tlog new_tlog_directory
Topic 5: Maintaining the NBDB

After completing this topic, you will be able to perform various tasks to maintain
the NBDB.
Changing the Location of the NBDB Files

You may need to change the location of the NBDB files in order to improve
performance and prevent corruption if the database files are very large using the
following procedure. Keep in mind that this may take a few minutes, so be patient.
3 Start the Sybase daemon or service:
4 Move the files:
nbdb_move –data new_data_directory
–index new_index_directory –tlog new_tlog_directory
The directories must exist before running the nbdb_move command.

Checking for Catalog Consistency
bpdbm -consistency
Checking for Catalog Consistency

Use the bpdbm –consistency command to compare the contents of the image
catalog against the EMM_DATA.db file. (This command became available starting
in NBU 6.0 MP2.) You can then view the command output for database
consistency errors.
The following graphic shows a catalog inconsistency:

Sending the Database to Technical
Support
Step Action
2 Start the Sybase Server.
3 Unload the database:

• UNIX:
nbdb_unload destination_directory
• Windows:
nbdb_unload.exe destination_directory
4 Send the entire directory to Technical Support.
Sending the Database to Technical Support

When working with Technical Support, you may be directed to send them the
database. To do this, perform the following steps:
2 Start the Sybase Server:
3 Unload the database:
– UNIX: nbdb_unload destination_directory
– Windows: nbdb_unload.exe destination_directory
4 Zip up the entire directory and send it to Technical Support.
Technical support personnel use the reload.sql script that is placed in the
unloaded directory to replicate the contents of your database.
CAUTION If you have changed the database password from the default, you
must provide the new password to Technical Support. Without the
password, Technical Support cannot access your database.

Using the mail_dr_info Script
Step Action
1 Create the script.
Use the following parameters in your script (optional):
%1 = The e-mail address specified on the DR tab

%2 = The subject line of the DR e-mail
%3 = The name of the DR e-mail
%4 = The name of the DR file
2 Run a hot, online catalog backup.
Using the mail_dr_info Script

The presence of the mail_dr_info script provides a hook for you to use to
customize the actions that occur after a hot, online catalog backup. The
mail_dr_info script overrides whatever settings you have specified under the
DR tab for the catalog backup policy. To use the mail_dr_info script, perform
the following steps:
5
1 Create the script.
– UNIX: Create
/usr/openv/netbackup/bin/mail_dr_info.sh. In order to
execute in UNIX, the mail_dr_info script must have a permission of
755.
– Windows: From the master server, copy
install_path\NetBackup\bin\nbmail.cmd to
install_path\NetBackup\bin\mail_dr_info.cmd.
Optionally, you may use the following four parameters in your script. These
parameters are passed by NetBackup after a hot, online catalog backup:
%1 The e-mail addresses specified in the DR tab
%2 The subject line of the DR e-mail
%3 The name of the DR e-mail
%4 The name of the DR file

2 Run a hot, online catalog backup.
After a hot catalog backup, NetBackup checks for the presence of
/usr/openv/netbackup/bin/mail_dr_info.sh (UNIX) or
install_path\NetBackup\bin\mail_dr_info.cmd (Windows).
Note: If the mail_dr_info script exists, NetBackup does not send the DR e-
mail, even if it was configured in the catalog backup policy; NetBackup
passes the parameters to the script and runs the script instead.
NetBackup e-mail notification requires a command-line SMTP client, such as

mailx for UNIX, or a third-party utility, such as the public domain tool
blat.exe for Windows. For more details, see the comments in the
nbmail.cmd script.

Lesson Summary
• Key Points
– In this lesson, you were introduced to the components of the NBDB and the
EMM domain.
– You learned how the NBDB is created during a master server installation.
– You followed the flow of a backup job through the EMM Server.
– You looked at how to back up, recover, protect, and maintain the NBDB.
– NetBackup Media Manager System Administrator’s Guide
– TechNotes 240584, 276098, 281818
Lab 5: Understanding the NetBackup

Database
• Manage the Sybase server and the NBDB database.
5
• Display EMM domain information.
• View resource allocations.
• Protect and recover the NBDB.

Understanding the NetBackup Database,” page A-38.
Solution: Understanding the NetBackup Database,” page B-46.

Lesson 6
Troubleshooting Devices
Lesson Introduction
Tools
Database
Issues
Issues

Topic
able to:
Topic 1: Preliminary Device Verify that all hardware and software
Troubleshooting components are compatible with NetBackup.
Topic 2: Environmental Verify tape and pass-through drivers,

Device Troubleshooting operating system device access, manual
Approach manipulation of the robot, mounts,
unmounts, and device replacement.
Topic 3: NetBackup Device Use NetBackup tools and techniques to

Troubleshooting Approach verify NetBackup device access and
configuration.

When Did the Error Occur?
• How do you know an error occurred?

The way in which you troubleshoot devices depends on the
environment in which the error occurred
• Did the error occur:
– During a new installation?
– When adding devices to an existing installation?
– In a stable environment with no recent changes?
Topic 1: Preliminary Device Troubleshooting

After completing this topic, you will be able to verify that all hardware and
software components are compatible with NetBackup.
When Did the Error Occur?

How you troubleshoot device problems depends upon the situation in which the
error occurred.
• Did you experience errors as you tried to install and configure NetBackup in a
new environment?
• Did you have an existing environment, but encounter errors when you added
new devices?
6
• Did devices suddenly stop working in a stable environment with no recent
changes?
Lesson 6 Troubleshooting Devices 6–3

Are the Devices Supported by NetBackup?
Check the hardware compatibility lists (HCLs) at
http://entsupport.symantec.com for the following items:
• Tape libraries and drives
• Virtual tape libraries (VTLs) and drives
– Does NetBackup support your VTL’s configuration?
– Verify the hardware compatibility list (HCL) for your VTL.
– Read the VTL-specific notes carefully.
• Robot types
• Optical devices
• SSO switches, intelligent switches, bridges, routers, and HBAs
• iSCSI configurations
• Encryption devices

Are the Devices Visible to the
Operating System (1 of 2)?
If not, answer the following questions:
• Do the devices have power?
• Is SAN zoning correct?
• Can non-NetBackup servers see the devices?
• Did you check your cables?
– Fibre
– SCSI
– SCSI termination
– Ethernet (for iSCSI-based devices)
• Is RSM disabled and not set to manual? (Windows only)
• Are there error codes displayed on the device?
– Robotic
– Stand-alone drives
Topic 2: Environmental Device Troubleshooting Approach

After completing this topic, you will be able to verify tape and pass-through
drivers, operating system device access, manual manipulation of the robot,
mounts, unmounts, and device replacement.
Are the Devices Visible to the Operating System (1)?

The operating system must be aware of the devices and there cannot be any errors
detected. If the OS is not aware of the devices or if the OS detects errors, answer
the questions on the slide before trying to get NetBackup to work.

Are the Devices Visible to the
Operating System (2 of 2)?
Answer the following questions as well:
• Is the HBA configured correctly?
– LUN masking
– HBA sees WWNs
Use vendor-supplied tools
– Persistent binding
– HBA drivers and firmware
• Is library hardware tape partitioning configured?
Are the Devices Visible to the Operating System (2)?

This slide shows additional questions that you must answer if your devices are not
visible to the Operating System.

Are the Devices Present?
If the operating Then determine if the devices are present

system is … using …
Solaris, dmesg, sgscan, iostat -En
HP-UX, ioscan, dmesg, getconf
AIX, lsattr, lsdev
Linux, dmesg, lspci, hwinfo
Windows, The Device Manager, the BIOS screen
Reference the operating system documentation for detailed

information
Are the Devices Present?

The table in the slide lists some of the operating system-specific commands or
methods that you can use to determine if the tape devices are visible to the
operating system. For the best results, consult the documentation for your
operating system.
The following is sample output from a Solaris dmesg command:

Windows Device Manager
You can use the Windows Device Manager to determine if devices are present. In
the Administration Console, select Device Manager—>Tape Drives. Right click
the device in question, and select Properties.
Ensure that robots and drives are enabled. Disabled devices appear with a red X in
the Device Manager, but backups and restores fail. If the device is disabled, enable
it by clicking the Enable Device button in the Drive Properties dialog box.
Symantec provides tape drivers for most tape drives; however the manufacturer’s
drivers should work with NetBackup. Check the support Web site at
entsupport.symantec.com to verify that the driver you are using is
supported.
Device Vendor Tools and Utilities

If errors have been detected, remember that the device vendors often provide tools
and utilities to troubleshoot their devices. Reference the device documentation for
information.

Is the SCSI Pass-through Driver Installed?
If the operating Then use the …

system is …
NetBackup-provided sg SCSI pass-through

Solaris,
driver.
NetBackup-provided ovpass SCSI pass-

AIX,
through driver.
Operating system-provided SCSI pass-

Linux or HP-UX,
through driver.
VERITAS driver, unless specified otherwise in

Windows,
the HCL.
Reference the NetBackup Device Configuration Guide

for detailed information
Is the SCSI Pass-through Driver Installed?

If you are performing a new installation or adding devices and you receive the
following type of message, confirm that the SCSI pass-through driver is installed.
Drive's SCSI adapter does not support pass-through
NetBackup discovers and communicates with devices through a series of SCSI
commands. The SCSI pass-through driver is the interface through which
NetBackup sends the commands and receives the command responses for
operations including:
• Drive scanning
• Device discovery
• TapeAlert collection
6
• Locate-block positioning
• Quantum SDLT performance optimization
• SCSI reserve/release operations
The table on the slide identifies the SCSI pass-through driver requirements.
An online text version of the NetBackup Device Configuration Guide is available
in /usr/openv/volmgr/NetBackup_DeviceConfig_Guide.txt
(UNIX) or C:\Program Files\VERITAS\Volmgr\
NetBackup_DeviceConfig_Guide.txt (Windows).

Windows Medium Changer Properties
To display the Windows medium changer properties, expand Medium Changers in
the Windows Device Manager. If the medium changer is identified as Unknown,
the driver is Microsoft. Some hardware vendors recommend using this driver if
you are using a third-party backup application such as NetBackup or Backup Exec.
Symantec does not supply any medium changer or tape library drivers. Right-click
the medium changer in question, and select properties. The Medium Changer
Properties dialog box is displayed.
Windows Tape Drivers

Check the device properties in the Windows Device Manager to ensure that the
tape device drivers are installed on all media servers. Verify that the driver
provider is listed and is not designated as Unknown. If the Windows tape driver is
Unknown, reinstall the tape driver.
The tape driver may display as Unknown if there have been upgrades to the
firmware on the tape drives, unplanned power outages, or SAN configuration
changes.

Can You Manipulate the Devices
Manually?
Operating
Tape device commands
system:
• robtest
NetBackup: • Tape device commands
• Robot and drive diagnostics
Device vendor: Device specific commands
Can You Manipulate the Devices Manually?

When you are troubleshooting device problems, use the following tools and
commands provided by NetBackup and the operating system to determine if you
can manually manipulate the devices. Also, remember to use the tools and
commands provided by the device vendor.
• mt and tctl
• nt_ttu.ext
• robtest
• Robot diagnostics
• Drive diagnostics
• tpreq and tpunmount
6
Using Tape Device Commands
The mt and tctl Commands

Use the mt command (Solaris, HP-UX, Linux) or the tctl command (AIX) to
determine drive status. If there is not a tape in the drive, the command verifies the
ability of operating system to control the robot.

• mt command output without a tape in the drive:
• mt command output with a tape in the drive:
The nt_ttu.exe Command

Windows does not have an operating system command equivalent to mt or tctl.
However, if NetBackup is installed, the nt_ttu.exe -p command acts
similarly to the mt or tctl command. nt_ttu.exe is located at
install_path\VERITAS\Volmgr\bin.

Using robtest
Use robtest to display robot information and verify robot operation. All
robtest commands are directed to the operating system to manipulate the robot,
except for the unload command which uses NetBackup to unload a drive.
Use robtest commands as follows:
• To display the robot configuration, enter mode.
• To read barcodes of tapes in slots, enter s s.
• To read barcodes of tapes in drives, enter s d.
• To move tapes between slots and drives, enter m s# d#, where # is the slot or
drive number.
• To move tapes between drives and slots, enter m d# s#.

• To unload tapes from drives, enter unload d#, where # is the drive number.
robtest can also be used to troubleshoot drive numbering problems as follows:

1 Move a tape into a drive.
2 Open the Device Monitor.
3 Use robtest to compare the physical drive number to what is displayed in
the Device Monitor.

Viewing Detailed Robot Attributes
Use debug mode in robtest to view extended robot attributes. The extended
attributes may help you when you are working with the robot vendor to isolate
errors in the library. To view the extended attributes:
1 Start robtest.
2 Type debug to enter debug mode.
3 Type dm to view extended robot attributes.
4 Type debug to exit debug mode.

Robot and Drive Diagnostics
Run robot diagnostics from the Administration console. Robot diagnostic results
do not appear in the Activity Monitor.
Run drive diagnostics from the Administration console. Drive diagnostic results
appear in the Activity Monitor.
Note: Drive diagnostics fail if there is not an available tape in the NetBackup
volume pool.

Requesting a Tape Mount
tpreq and tpunmount

Use the tpreq and tpunmount commands to test NetBackup’s ability to mount
and unmount drives. tpreq mounts a tape into a drive and creates a file.
tpunmount removes the tape from the drive and removes the file.
Use the tpreq and tpunmount commands as follows:
1 Record the media ID of an available tape.
2 Type tpreq -m media_ID -f file_name
The tpreq command shows up in the Activity Monitor:
3 Use the Device Monitor or use the vmoprcmd command to verify that the tape
was mounted in the drive.

4 Type tpunmount -f file_name
5 Use the Device Monitor or the vmoprcmd command to verify that the tape
was unmounted from the drive.

Replacing Existing Drives
If existing devices are replaced, the database should be updated with the
new robot and drive information automatically.
Determine if
tpautoconf
there are –report_disc
discrepancies:
Update the tpautoconf

robot –replace_robot current_number
information: –path new_path
Update the tpautoconf

drive –replace_drive current_name
information: –path new_path
Replacing Existing Drives

After a device has been replaced, the database should be updated with the new
robot and drive information automatically. If it is not, update the NBDB with new
device information using the tpautoconf command as follows:
• To report discrepancies between the database and the devices, enter:
tpautoconf -report_disc
• To update robot records in the database, enter:
tpautoconf -replace_robot current_number -path new_path
• To updates drive records in the database, enter:
tpautoconf -replace_drive current_name -path new_path

Topic 3: NetBackup Device Troubleshooting Approach
After completing this topic, you will be able to use NetBackup tools and
techniques to verify NetBackup device access and configuration.
Is NetBackup Aware of the Devices?

There are many ways to determine if NetBackup is aware of robots and drives,
with the most familiar method being the NetBackup Administration console,
using:
• Media and Device Management—>Device Monitor
• Media and Device Management—>Devices—>Drives
• Media and Device Management—>Devices—>Robots

Displaying Drive Information Using
Commands
To display this drive
Use these commands …
information,
Name vmoprcmd, tpconfig, vmglob, vmdareq
Physical number tpconfig, robtest, vmglob

Path vmoprcmd, tpconfig, tpautoconf, robtest, sgscan (Solaris)
Serial number tpconfig, tpautoconf, scan, vmglob
Index tpconfig
Vendor tpconfig, tpautoconf, scan, sgscan (Solaris)

Type vmpoprcmd, tpconfig, scan, vmglob
Port, bus, target, LUN scan
Hosts vmoprcmd
Status vmoprcmd, tpconfig, robtest, vmdareq
Ready vmoprcmd
Scan host vmdareq
Shared tpconfig
Displaying Device Attributes Using Commands

Use the commands shown on the slide to view robot and drive information.
Sample commands follow.
• vmoprcmd
If the status column contains RESTART, restart ltid on the appropriate

media server.

• tpconfig -dev_ping
• tpconfig -dl
• To invoke the menu-based Device Management Configuration Utility, enter

tpconfig with no parameters (UNIX only)

• tpautoconf -t, tpautoconf -r
• robtest

• scan -tape
• scan -changer
• sgscan (Solaris only)

• vmglob -listall
• vmglob -listall -b
• vmdareq

Are the Drives Numbered Correctly?
If you encounter an error the first time your robot attempts to mount media, the
drives may be numbered incorrectly. This type of error usually occurs if you
configure your robot and drives manually.
Best Practice: Use the Device Configuration Wizard to configure drives and avoid
drive number problems.
Manual drive configuration allows you to add the drives in any order. For example,
you may configure physical drive three before you configure physical drive one. If
you choose the wrong physical number for the drive, NetBackup does not discover
the error until the robot tries to mount media into the incorrectly numbered drive.
Errors Due to Incorrect Drive Numbers

The type of errors that you may see if the drives are numbered incorrectly include:
• Robot errors
• Down drives
• Frozen media
To troubleshoot incorrect device numbers:
1 Record the physical drive number (starting at 1) and the serial number of each
drive in the library.
Physical drive numbers start at 1.

2 Refer to output from the scan -changer command, which lists the physical
drive number and serial number.
3 Determine if NetBackup has the drive numbers configured correctly.
The following is an example of viewing the Drive Details in the Device
Monitor:
You may also use the vmglob -listall command to confirm the physical
drive number, as follows:

Are the Robotic Controls and Tape
Drives Configured Correctly (UNIX)?
• NetBackup expects the robots and drives to be
configured in a certain way.
• Some of these configuration parameters include:
– Berkeley-style close er
nag
Ma
– Fast-tape positioning edia
M ice
tion
Dev figur a
– Extended file marks Con e
id
Gu
– SCSI reserve/release
– No-rewind device name
Refer to the NetBackup Device Configuration Guide

for detailed information.
Are the Robotic Controls and Tape Drives Configured Correctly?

(UNIX only)
Robotic controls and tape drives must be configured as NetBackup expects them to
be. Some of the robotic control configuration parameters include:
• Berkeley-style close
• Fast-tape positioning
• Extended file marks
• SCSI reserve/release
• No-rewind device name
Use the section in the NetBackup Device Configuration Guide that is applicable to
your operating system to ensure that your robotic controls and tape drives are
properly configured.

Has the SSO Licence Been Installed?
If you are unable to configure shared devices, verify that the SSO license has been
added to all of the media servers that share devices.
The SSO software is automatically installed, but it requires a key to be enabled.
Enter the SSO license on every media server that uses the shared drives.
Determining if the SSO License Is Installed

You can display licenses for a media server from the NetBackup Administration
console. If you want to determine if an SSO key was entered on all media servers
in the EMM domain, use the nbemmcmd -listhosts -verbose, as shown
in the graphic.
An odd number in the MachineFlags field indicates that an SSO license has been
installed because the SSO key sets the 1 bit. Additional flags are set for NDMP and
Remote Client. Machine flags values are set as follows:
6
• 1 = SSO
• 2 = NDMP
• 4 = Remote Client
• 7 = all

Is the EMM Server Aware of the Media Server?
If you add a media server to an existing EMM domain, you must add the media
server to the servers list of the EMM server and issue a command to add the new
media server to the database.
1 Add the name of the media server to the servers list on the EMM server as
follows:
– UNIX:
/usr/openv/netbackup/bp.conf
SERVER = new_media_server
Do not add the new media server as the first SERVER = entry in the
bp.conf file. The first SERVER = entry must specify the master server.
– Windows:
Use the GUI to navigate to Host Properties on the EMM server and add the
media server in the Servers dialog box. This entry is also visible in the
HKLM\SOFTWARE\Veritas\NetBackup\CurrentVersion\
Config\Server Registry subkey.

2 Use the nbemmcmd -listhosts -verbose command to confirm that
the new media server is known to the EMM server as follows:
nbemmcmd -listhosts -verbose
3 If the EMM server is not aware of the media server, simply restart NetBackup
on the the media server. If the EMM server is still unaware of the media server,
run the nbemmcmd -addhost command from the EMM server, as follows:
nbemmcmd -addhost -machinename machine_name
-machinetype machine_type
-masterserver master_server_name
-netbackupversion version_number
-operatingsystem operating_system

Is the Device Mapping File
Up-to-Date?
http://entsupport.symantec.com
tpext
Is the Device Mapping File Up-to-date?

If you encounter any device problems, verify that your device mapping file is up-
to-date. The device mapping file is used:
• By the Device Configuration wizard to discover and configure new devices
• By NetBackup to determine which protocols and settings to use when
communicating with various vendor devices
Updating the Device Mapping File

Use the following procedure to update the device mapping file:
1 Obtain the most recent device mapping file from
http://entsupport.symantec.com.
6
The file names are similar to Mappings_6_nnnnnn.tar (UNIX) and
Mappings_6_nnnnnn.zip (Windows), where nnnnnn is the TechNote
Document ID. For example, Mappings_6_293476.tar relates to
2 Download and extract the mapping file to a temporary location on the EMM
server. This creates two files: Readme.txt and external_types.txt.
3 Copy the external_types.txt file to the following location:
– UNIX: /usr/openv/var/global
– Windows: install_path\NetBackup\var\global
4 Run the tpext command as follows to update the EMM database and related
device mappings from the new external types file.
a NetBackup 6.0: Type tpext on the EMM server.

NetBackup 6.5: Type tpext -loadEMM on the EMM server.
b Type tpext -get_dev_mappings on each media server (including
the EMM and master servers) that has devices attached to it.
5 To verify that the version that is now stored in the Enterprise Media Manager
database is the same as what is in the file stored on the host, type
tpext -get_dev_mappings_ver.
The device mappings for the media server should now match those in the
EMM database.
In the following example, the local device mappings file is up-to date:
tpext -get_dev_mappings_ver
device mappings version in the EMM database is 1.66.8.2
device mappings version from the local file is 1.66.8.2
The local device mappings file is up-to-date.
In the following example, there is a device mappings mismatch:

tpext -get_dev_mappings_ver
device mappings version in the EMM database is 1.66.8.2
device mappings version from the local file is
1.49.4.21.8.1.2.4
The local device mappings file is not up-to-date.
6 Stop and restart ltid on each media server.

Is the Media Server Active for Tape and Disk Jobs?
If a media server has not been configured for tape devices or, if it loses its
configuration for the tape devices, it cannot access the robot and drives.
You can determine if a media server is active for tape and disk jobs from the
Administration Console or from the command line by using the
nbemmcmd -listhosts -verbose command and viewing the
MachineState.
In certain situations, you may need to deactivate the media server. If a media
server needs maintenance, it is possible to deactivate the media server using the
GUI (Media and Device Management—>Devices—>Hosts) or the vmoprcmd
command. Jobs currently in progress on that media server continue to run to
completion. Queued jobs are routed to alternate media servers, if possible. After
the maintenance is complete, the media server can be activated again using the
GUI or the vmoprcmd command.

Status of Shared Drives
LAN
EMM
Server
1 2 3 4

Shared
Tape
Drives
Status of Shared Drives

After a device has been discovered, it is identified by the path to the device from
the media server. In a shared storage environment, devices are identified by
multiple paths. The Device Configuration wizard uses the tpautoconf
command with a special set of options to discover and configure devices.
Each path has an independent status, UP or DOWN. In this slide, all paths are UP.
Device Discovery
NetBackup automatically discovers devices if:
• The SCSI pass-through driver exists
• The robot and tape drives support serialization

Status of Shared Drives: One Path Is
DOWN
LAN
EMM
Server
1 2 3 4

Path 2 is
DOWN. Shared
Tape
Drives
Status of Shared Drives: One Path Is DOWN

If the path between a drive and the media server becomes unavailable, or if the
media server DOWNs a drive, the drive is still available through its other paths. If
a drive is UP on some paths and DOWN on other paths, it is said to be in mixed
mode. In this example, both drives are in mixed mode because one media server
has lost its connection to the SAN.

Status of Shared Drives: TapeAlert
Error
LAN
EMM
Server
1 2 3 4

Tape
Alert
All paths are
DOWNed. Shared
Tape
Drives
Status of Shared Drives: TapeAlert

If a media server DOWNs a drive because of a TapeAlert error, all paths to the
drive are DOWNed by the EMM server.

Drive TapeAlert Codes
TapeAlert Code Error Message NetBackup Action
0x14 CRT* clean now Clean drive
0x15 CRT clean periodic Clean drive
0x1E CRT hardware A Down drive
0x1F CRT hardware B Down drive
*CRT = Critical
Drive TapeAlert Codes

TapeAlert consists of 64 bits used as alert flags for a device. The conditions that set
alert flags are device dependent. The table on the slide lists the tape alerts that
apply to drives and the action that occurs if the codes are set.
NetBackup reads the alert flags from a log sense page before and after every read
and write operation. Reading the log sense page clears all alert flag bits. Tape alerts
are entered into the bptm log, the error log, the job details log, and the system log.
To determine if TapeAlert is enabled on a drive, check the drive details.

If TapeAlert is enabled on a drive, TapeAlert messages appear in the bptm log.

Are the Device-Related Processes or
Services Running?
EMM Server vmd
Media Server vmd, txxd, ltid, avrd
Robot control
vmd, txxcd, ltid
host
Are the Device-Related Processes or Services Running?

Backups and restores cannot operate without the daemons or services and
processes shown on the slide.
ltid, the device manager daemon calls txxd, the robotic drive daemon. txxd
calls txxcd, the robotic control daemon on the robot control host to issue SCSI
commands to mount the media. After the media is mounted, txxd scans the drive
to verify that the correct tape is loaded.
After the tape is loaded and verified, nbjm on the master server is notified and the
backup data flow from the client to the media server begins.
Understanding txxd and txxcd
6
txxcd handles robotic arm requests for TLD and TL8. All other robot controls
are handled by txxd drivers. In addition, robot and drive control are separate, and
can be shared under different servers. The xx in txxd or txxcd is interpreted as
follows ([c] indicates the robot control daemon for some types of robots):
• The first x is the device type
• The second x is the density or media type
t x x [c] d
Library = l 4 = 4mm
Stacker= s d = DLT
8 = 8mm
h = ½”

StorageTek ACS Environments
In a StorageTek ACS environment, on Windows media servers, an application
from Sun/StorageTek must be installed and running on the media server. This
application is called LibAttach and is supplied by Sun/StorageTek. LibAttach is
operating system-specific. Contact Sun/StorageTek to verify which LibAttach
version your environment requires.

Have the Device-Related Logs Been
Viewed?
• UNIX: /usr/openv/volmgr/debug
• Windows: install_path\Volmgr\debug
Create the … folder To enable logging for …
daemon vmd (Volume Manager service)
reqlib The process requesting a vmd service
tpcommand The tpconfig and tpautoconf commands
ltid ltid (Device Manager service)
robots SCSI robotic daemons
Have the Device-Related Logs Been Viewed?

Additional debug and informational messages from the robot and drive processes
(txxcd and txxd) are also logged to the /var/adm/messages file (UNIX)
and the Event Viewer application log (Windows).
• The txxd/txxcd processes write to log files in the robots directory.
• The vmd daemon or service writes to log files in the daemon directory.
• The tpcommand log directory contains debug log files used by the
tpconfig and the tpautoconf commands.

Lesson Summary
• Key Points
In this lesson, you learned how to troubleshoot device errors.
– NetBackup Device Configuration Guide
– OEM Web Sites
– http://entsupport.symantec.com
Lab 6: Troubleshooting Devices
In this lab, you will troubleshoot device-related problems:

• Run scripts that create or simulate device-related
problems.
• Troubleshoot at least two of the simulated problems that
involve NetBackup device-related errors.

Troubleshooting Devices,” page A-44.
Solution: Troubleshooting Devices,” page B-55.

Lesson 7
Troubleshooting Media
Lesson Introduction
Tools
Database
Issues
Issues

Topic
able to:
Topic 1: Media-Related Troubleshoot the most common media-
Status Codes related status codes.
Topic 2: Troubleshooting • Identify the causes of frozen media
Frozen and Suspended and recommend appropriate actions
Media that do not jeopardize legacy data.
• Identify the two reasons for the
automatic suspension of media that
requires retiring or disposing of the
affected media.

Common Media-Related Status Codes
Status Code Description
96 Unable to allocate media
98 Media request error
83 Media open error
84 Media write error
85 Media read error
86 Media position error
129 Disk storage unit full
Topic 1: Media-Related Status Codes

After completing this topic, you will be able to troubleshoot the most common
media-related status codes.
Common Media-Related Status Codes

Media errors commonly generate the following status codes:
• NetBackup status code 96
NetBackup is unable to allocate new media for backup because the policy’s
storage unit has none available.
NetBackup encountered an error requesting media, such as a tape mount
request (tpreq).
NetBackup is unable to open the device or file that the backup or restore must
use.
7
An input/output (I/O) error occurred while NetBackup was writing the backup.
An input/output (I/O) error occurred while NetBackup was reading the backup.
An I/O error occurred while NetBackup was positioning media.
The disk storage unit is full.
Lesson 7 Troubleshooting Media 7–3

Troubleshooting a NetBackup
Status Code 96 Error
• The EMM server is unable to allocate a new volume for
the backup job:
– The media is configured incorrectly.
– The storage unit contains no available volumes in the requested
volume pool.
• Check the following:
– Has the volume:
• Been frozen or suspended?
• Expired?
• Exceeded maximum mounts?
– Is the correct host specified for the storage unit in the NetBackup
configuration?
– Is the media:
• In the correct volume pool?
• Unassigned?
• Available at the required retention level?
Troubleshooting a NetBackup Status Code 96 Error

NetBackup status code 96 is one of the more common media-related errors. This
error occurs when the EMM server is unable to allocate a new volume for the
backup job and indicates that the storage unit contains no available volumes in the
requested volume pool and the scratch volume pool (if implemented).
If your media are defined as a media type other than the media type expected by
the robot and there are no other volumes from which to choose in the expected
pool or in the scratch pool, a volume is not selected, and a status code 96 is
returned.
If you receive a NetBackup status code 96, check the following:
• Is the volume frozen, suspended, or expired, or has it exceeded its maximum
number of mounts?
• Confirm the EMM server host name as follows:
tpautoconf -get_gdbhost
train11
nbemmcmd -listhosts
NBEMMCMD, Version:6.5
The following hosts were found:
server train11
master train11
media train12
Command completed successfully.

nbemmcmd -getemmserver
NBEMMCMD, Version:6.5
These hosts were found in this domain: train11, train12
Checking with the host "train11"...
Checking with the host "train12"...
Server Type Host Version Host Name EMM Server
MASTER 6.5 train11 train11
MEDIA 6.5 train12 train11
Command completed successfully.
– If a media server has the wrong host defined for its EMM server, change
the EMM server host name as follows:
tpautoconf -set_gdbhost emm_server_host
• Is the correct host specified for the storage unit in the NetBackup
configuration?
bpstulist -U refers to the media server as the Host Connection. This
should be the host that has drives attached to it.
• Is the media in the correct volume pool and unassigned, or is the active media
available at the required retention level (also known as retention period)?
There are several ways to display the retention level for each piece of active
media:
– bpmedialist -p pool_name
– The available_media script
– NetBackup Administration Console Media—>Volume Pools section
The reason for an exhausted pool may be because the retention period is too
generous; which means that images on media are retained longer than
necessary. This affects how quickly media can be recycled for reuse. The best
retention period is based upon your business needs and legal data recovery
obligations.
NetBackup has 25 retention levels available for use. There are 10 default levels
(retention levels 0 through 9), which offer typical retention values. Retention
level 9, with a value of infinity, is the only retention level that cannot be
changed.
Define a custom retention level that best suits your needs by using the
NetBackup Administration Console Host Properties—>Master Server—>
Properties—>Retention Periods or by using the bpretlevel command.
7
• Did you rename the scratch pool?

Media that is due to be returned to the scratch pool (due to image expirations)
attempt to use the name of the original scratch pool from which they were
pulled.
• Media in a common library is owned by an alternate media server.
If media sharing is not enabled, one media server owns the media and prevents
other media servers from appending their backup data to that media.

Using the bpretlevel Command
View the current retention levels bpretlevel –L

on the master server,
Change retention level 24 to 7

bpretlevel –r 24 7 weeks
weeks,
Using the bpretlevel Command

You can redefine the retention period (duration) for a retention level using either
the GUI or the bpretlevel command as follows:
• GUI:
Select NetBackup Administration Console—>NetBackup Management
—>Host Properties—>Master Server—>Retention Period.
• bpretlevel
The bpretlevel command is located at:
– UNIX: /usr/openv/netbackup/bin/admincmd
– Windows: install_path\NetBackup\bin\admincmd
The general format of the command is:
-r retention_level retention_period
where:
› retention_level is an integer value from 0 to 24.
› retention_period is an integer value followed by the modifier
days|weeks|months|years.
For example, to view the current retention levels on the master server, enter:
bpretlevel -L
To change retention level 24 to 7 weeks, enter:
bpretlevel -r 24 7 weeks

Status Code 96:
Correcting the Problem
Step Action
1 Identify the selected storage unit using the Activity Monitor or
Problems report.
• Verify that the volumes are in the correct pool.
• Verify that a volume configuration update has been
performed if media has been physically replaced.
2 Check for FROZEN, SUSPENDED, IMPORTED, or FULL volumes using
the available_media.cmd script or the Media Lists report (6.0) or
the Tape Lists report (6.5).
• Add volumes if you are using a robotic storage unit that contains
empty slots.
• Replace FULL volumes if using a robotic storage unit that has no
empty slots.
• Fix or replace FROZEN or SUSPENDED volumes.
4 Establish a scratch pool and keep it populated.
5 Check the Troubleshooting wizard for more recommendations.
Status Code 96: Correcting the Problem

Perform the steps shown on the slide to isolate and correct a status code 96.
The available_media script is located at:
• UNIX: /usr/openv/netbackup/bin/goodies
• Windows: install_path\NetBackup\bin\goodies\
Use the available_media script to identify all volumes in the EMM catalog.
The information that is provided includes the media ID, the robot number, and the
volume status.
To repair FROZEN volumes that are frozen because of format problems:
• Unfreeze the volume using the bpmedia command.
• Write a new media header to the tape using the bplabel command.
CAUTION Running bplabel destroys any data previously contained on the

media. 7

Typical Output from a Status Code 96
Error
Activity monitor job 16 details status text
Output from bperror –jobid 16 -U
Typical Output from a Status Code 96 Error

This slide shows typical output from a status code 96 error. Notice that in both
reports the Status Code 96 error occurs immediately in the backup operation, when
the EMM database is being queried by the Intelligent Resource Manager’s Request
Broker (nbrb) on behalf of the Job Manager (nbjm) for available resources. No
media was available for the backup job.

• Status code 98 is caused by an error requesting media, such as by a tape
mount request.
• Status code 98 may be a ltid problem:
– UNIX: bpps -a
– Windows: bpps
• Status code 98 may be an EMM database problem:
1. Confirm that the EMM database does not include the media:
nbemmcmd -listmedia -mediaid media_id
2. If necessary, add the volume to the database:
vmadd -m media_id -mt media_type -h emm_server -p pool_number
To determine the volume pool number:
vmpool -listall
3. Label the media:
bplabel -m media_id -d media_density –o -p volume_pool_name
-n device_name
To determine the device_name:
tpconfig -d

The NetBackup Troubleshooting Guide and the Troubleshooting wizard both
suggest possible ltid problems for status code 98. Their resolution procedures
have you check to see if ltid is running using the Activity Monitor or bpps –a
on UNIX, or simply bpps on Windows.
TechNote 280309, STATUS CODE: 98 - Unable to label a tape in Veritas
NetBackup 6.0, the bplabel command returns Error reported by
media operation: requested media ID was not found in
the EMM database, outlines another possible reason for status code 98 that
involves the EMM database instead of ltid. Before labeling a tape using either
the NetBackup Administration Console GUI or the bplabel command, the
EMM server needs to be aware of any media IDs.
1 To confirm that the EMM database does not include the specific media ID:
nbemmcmd -listmedia -mediaid media_id
Output similar to the following is returned:
The function returned the following failure status: 7
volume does not exist in database (35)

Command did not complete successfully.
2 If you received an error indicating that the volume does not exist in the
database,
a Add the volume using the NetBackup Administration Console, Media
section of Media and Device Management or the vmadd command as
follows:

vmadd -m media_id -mt media_type -h EMM_server
-p pool_number
Available media types and densities are: 4mm, 8mm, dlt, dlt2, dlt3,
dtf, hcart, hcart2, hcart3, qscsi, odiskwm, and odiskwo.
b To determine the volume pool number, run the vmpool command as
follows:
vmpool –listall
This command lists the pool number and the pool name for each
configured volume pool.
3 Label the media.
CAUTION Do not relabel the media if the media contains valid data that needs
to be restored. Confirm that the media does not contain NetBackup
images by entering the following command:
bpmedialist -mcontents -m media_ID
Monitor the job using the Activity Monitor or bpdbjobs, and verify that the
label operation is successful.
bplabel -m media_id -d media_density -o
-p volume_pool_name -n device_name
The media types and densities listed in step 2 also apply to the bplabel
command.
The vmpool command can also be used to verify the volume pool name. To
obtain the name of the drive, enter:
tpconfig -d

• The root cause of this error is that bptm cannot open the
device or file that the backup or restore must use.
• NetBackup status code 83 can be caused by a:
– Drive configuration problem
– Tape that is write-protected
– Tape drive that needs cleaning
• Use the following to gather specific information
regarding the error:
– The Problems report
– The Troubleshooting wizard
– bptm logs

A NetBackup status code 83 is a media open error. The root cause of the problem
is that bptm cannot open the device or file that the backup or restore must use.
Typically, a status code 83 indicates a drive configuration problem. However, the
following media problems can also generate the error:
• The tape is write-protected.
Check the write protect tab on the tape to ensure that write protection is not
enabled.
• The tape drive needs cleaning.
Use the tpclean command on robotic drives that require cleaning.
If neither of these conditions is the cause of the problem, use the Problems report
and the Troubleshooting wizard to gather additional information about the status
code 83 error.
Establishing and analyzing bptm logs may also be helpful for troubleshooting.
7

NetBackup Status Codes 84, 85, and
86
• Media write error

Status code 84 :
• Usually occurs during backups
• Media read error

Status code 85:
• Usually occurs during restores
• Media positioning error

Status code 86:
• Tape or optical disk
Check the OS system logs on the media server for

hardware errors.
NetBackup Status Codes 84, 85, and 86

NetBackup status codes 84, 85, and 86 are similar in nature.
• Status code 84 is a media write error. The system’s device driver returns an I/O
error while NetBackup is writing to removable media or a disk file.
• Status code 85 is a media read error. The system’s device driver returns an I/O
error while NetBackup is reading from removable media or a disk file. Status
85 most likely occurs during restore operations.
• Status code 86 is a media positioning error. The system’s device driver returns
an I/O error while NetBackup is positioning media (tape or optical disk).
Troubleshooting these three status codes is also very similar. These problems
typically are hardware-related. Refer to the Windows System Event Log (using the
Event Viewer) or the UNIX syslog/messages file for more detail. To confirm
that this is a device-related problem use vendor-specific drive testing tools. If those
tests are successful, then the nature of error codes 84, 85 and 86 may be media-
related. See TechNote 273849, Jobs fail with any of the following status codes: 84
- Media Write, 85 - Media Read, 86 - Media Position. There could be potential
hardware issues that can cause these errors. There are several tools available
from the hardware vendors that can be used to help diagnose these hardware
issues, for more information on hardware testing tools.
Use the Problems report and the Troubleshooting wizard to gather information
about the situation causing the error. Also see the bptm logs. The Media Logs
report and the equivalent command, bperror -media -U, also provide
helpful background information for the problem.

Troubleshooting NetBackup
Status Code 84, 85, and 86 Errors
If … Then …
The tape drive needs cleaning, Use tpclean on the affected devices.
Verify that the media type matches the

The media type is wrong,
drive type.
The media is defective, Expire the volume using vmchange.
Incompatible tapes are being

Use compatible tapes.
used,
The drive is configured Verify the Media Manager and system

incorrectly, configuration for the drive.
Troubleshooting NetBackup Status Code 84, 85, and 86 Errors

The following list describes status codes 84, 85, and 86:
• The tape drive needs cleaning. Use the tpclean command on the robotic
drives.
• The media type is wrong. Verify that the media matches the drive type being
used. On an optical drive, ensure that the platters are formatted correctly.
• The media is defective. Use the vmchange command to set the volume
expiration date so that the tape is not used for further backups.
• Incompatible tapes are being used for the tape drive. TechNote 269177, In a
mixed media environment, media mounted to Super DLT drives are marked as
“frozen” or write-protected by Veritas NetBackup. However when the media is
mounted to the drive and the OS command mt -f /dev/rmt/<path>
status is run, there is no complaint about the media being write-protected,
states that DLT IV media are only backward read (not write) compatible with
the Super Digital Linear Tape (SDLT) drive type, and if they are accidentally
used for a backup, a “Media is write protected” error results.
• The drive is configured incorrectly. On UNIX media servers, the drive must be
7
configured for variable mode. If the drive is configured incorrectly for fixed
mode, a status code 84, 85, or 86 may result. Variable mode is set in the
st.conf file.
Note: Ensure that you do not have duplicate volume labels, which is also known to
cause status code 86 (positioning) errors.

Status Code 86 Positioning Error
Troubleshooting Example
An active volume containing unexpired backups
was intentionally erased.
Step Action
1 Check job status details:
• Activity monitor
• bpdbjobs -jobid job_id
• bperror -jobid job_id –U
2 Check the Problems report or the Media Logs report.
3 Display the content of the media error database.
nbemmcmd –errorsdb
4 Review the per-media server media errors file:
• UNIX: /usr/openv/netbackup/db/media/errors
• Windows: install_path\netbackup\db\media\errors
5 Resolution: expire the deleted images from the Image database.
bpexpdate –m media_id –d 0
Status Code 86 Positioning Error Troubleshooting Example

In this scenario an active volume containing unexpired backups was intentionally
erased using the bplabel command.
Job status details are available from the Activity Monitor in the NetBackup
Administration Console and the bpdbjobs command.
bpdbjobs -jobid 22 -all_columns
Output similar to the following is returned:

Notice the key, asc, and ascq SCSI message codes along with the operation that
failed, which was a Magnetic Tape backward space over file mark (MTBSF)
positioning request.
The bperror command provides another view of this error.
bperror -jobid 22 -U
An export of the Problems Report from the NetBackup Administration Console

provides the following: 7

Note: The Media Logs report provides much of the same information as the
Problems report for this error.
The Sense Key (key), Additional Sense Code (asc) and Additional Sense Code
Qualifier (ascq) are returned in response to a SCSI Request Sense command.
Sense Key = 0x8 Unexpected blank media (tried to read at the

end of a tape)
ASC = 0x0 with ASCQ = 0x5 End of Data detected
Sources:
• ANSI document on SCSI primary commands:
http://www.t10.org/ftp/t10/drafts/spc4/spc4r06.pdf
• ASC/ASCQ additional sense data information:
http://www.t10.org/lists/1spc-lst.htm
The media server catalogs read, write and position errors in an errors file located
at:
• UNIX: /usr/openv/netbackup/db/media/errors
• Windows: install_path\NetBackup\db\media\errors
Displaying the content of the errors file produces:
08/03/06 05:56:58 GAZ715 0 POSITION_ERROR rob0d1
Notice that the same media is giving errors on multiple drives more than once.
This strongly indicates a media- rather than a drive-related problem.
Problem Resolution
Clean the drive heads and attempt to duplicate any salvageable images from the
tape if you are experiencing a positioning problem (and not truly erased, as in this
example). Whether or not the duplication was successful, expire the images on the
known faulty media using the bpexpdate command, as shown on the slide,
delete the volume from the EMM database, and dispose of the tape securely.

Frozen Media
• NetBackup does not write to frozen media.

You can restore data, duplicate images, import images,
and verify files on frozen media.
• Common causes of frozen media include:
– Media that contain non-NetBackup data or a NetBackup cold
catalog backup
– Recurring errors on the media
– Recorded and external media ID mismatch
– Robotic device number and tape device mismatch
– In NetBackup 6.5, media ID on tape header (recorded) and
assigned media ID (requested by job) mismatch
Topic 2: Troubleshooting Frozen and Suspended Media

• Identify the causes of frozen media and recommend appropriate actions that do
not jeopardize legacy data.
• Identify the two reasons for the automatic suspension of media that requires
retiring or disposing of the affected media.
Frozen Media
NetBackup does not write to frozen media. After a tape volume has been frozen, it
can still be used to restore data, to duplicate images, to import images, or to verify
files, but no further backup images can be written to it.
To determine which media in your NetBackup environment are frozen, run the
available_media script, the Media List report, the Media Logs report, or the
All Log Entries report.
NetBackup freezes media for various reasons, including when:
• NetBackup attempts to write a backup to a piece of media that contains non-
7
NetBackup data or a cold catalog backup.

• NetBackup encounters recurring errors with a particular piece of media.
• There is a mismatch between the media’s recorded media ID and its external
media ID (bar-code).
• There is a mismatch between the robotic device number and the tape device.
• In NetBackup 6.5, there is a mismatch between the media ID recorded on the
tape header and the media ID assigned to or requested by the backup job.

Repairing Frozen Media with Non-
NetBackup or Catalog Data
• NetBackup freezes media containing non-NetBackup
tape headers.
• Workarounds:
– Use bplabel to write a NetBackup header.
– Configure ALLOW_MEDIA_OVERWRITE.
Repairing Frozen Media with Non-NetBackup or Catalog Data

By default, NetBackup freezes media containing non-NetBackup tape headers
(ANSI, AOS/VS, CPIO, DBR, MTF1, or TAR format). This occurs frequently
when you upgrade to NetBackup from other backup software and use the media
from the old backup system.
Workarounds for this problem include:
• Write a NetBackup header selectively on each media using the bplabel
command
• Configure ALLOW_MEDIA_OVERWRITE in the bp.conf file (UNIX only)
or use the GUI to set Allow media overwrite in the Media Host Properties for
the media server. This workaround does not apply to media used for cold
catalog backups.
Note: Both of these workarounds destroy any data contained previously on the
media.

Repairing Recurring Errors with the
Media
• Recurring media errors are recorded in:
– UNIX: /usr/openv/netbackup/database/media/errors
– Windows: install_path\NetBckup\db\media\errors
– With consolidated information in the EMM database, visible using:
nbemmcmd -errorsdb
• To set media error thresholds for freezing media:
nbemmcmd –changesetting
–media_error_threshold media_threshold_value
–machinename media_server
nbemmcmd –changesetting
–time_window time_window_value
–machinename media_server
• To set drive error thresholds for freezing media:
nbemmcmd -changesetting
-drive_error_threshold drive_threshold_value
-machinename media_server
• To display the settings:
nbemmcmd -listsettings -machinename host_name
Repairing Recurring Errors with the Media

Each time a read, write, or position error occurs, NetBackup records the time,
media ID, type of error, and drive index in the EMM database. NetBackup then
scans to see whether that piece of media or that drive has had the same type of
error in the specified time window.
There are three tunable parameters that are used together to determine whether to
FREEZE the media or DOWN the drive:
• TIME_WINDOW
TIME_WINDOW represents the amount of time in which errors are tracked.
Use this value in conjunction with the MEDIA_ERROR_THRESHOLD or the
DRIVE_ERROR_THRESHOLD to monitor the number of media errors that
occur within the time window. The default setting is 12 hours. This setting is in
the EMM database and is changed using the following command:
nbemmcmd -changesetting -time_window time_window_value
• MEDIA_ERROR_THRESHOLD 7
MEDIA_ERROR_THRESHOLD represents the number of media errors that can

occur before NetBackup FREEZEs the media.The default value for
MEDIA_ERROR_THRESHOLD is two errors. This setting is in the EMM
database and is changed using the following command:
-media_error_threshold media_threshold_value
• DRIVE_ERROR_THRESHOLD

DRIVE_ERROR_THRESHOLD represents the number of drive errors that can
occur before NetBackup changes the drive state to DOWN and freezes the
current media. The default value for DRIVE_ERROR_THRESHOLD is two
errors. This setting is in the EMM database and is changed using the following
command:
-drive_error_threshold drive_threshold_value
Note: The first time NetBackup encounters a critical error on a piece of media or a
drive, the media is frozen, or the drive is downed.
Display the DRIVE_ERROR_THRESHOLD, the MEDIA_ERROR_THRESHOLD,

and the TIME_WINDOW settings using:
nbemmcmd -listsettings -machinename host_name
Refer to TechNote 278996, Certain bp.conf entries, vm.conf entries, and
touch files are now set using nbemmcmd in Veritas NetBackup 6.0, for more
information on bp.conf, vm.conf, and legacy touch files that are now set
using the nbemmcmd command in NetBackup 6.0.
Refer to TechNote 234412, How Veritas NetBackup determines if a tape should be
frozen or the status of a tape drive should be changed to down, and how to change
this behavior and the NetBackup Commands manual for more information on the
nbemmcmd and its parameters.

Repairing a Mismatch Between
Recorded and External Media IDs
• Causes:
– The media is given a new and different bar-code label.
– The library does not have a bar-code reader and the inventory
function is used to populate the tapes into the EMM database.
• Troubleshooting a recorded and external media ID
mismatch:
– Use robtest to load a known volume.
– Use the Device Monitor or vmphyinv to view and confirm the
tape.
– Change the bar-code label to match the recorded media ID.
Repairing a Mismatch Between Recorded and External Media IDs

If the recorded media ID does not match the external media ID (bar-code), the
media ID read from the volume’s header is different from the media ID recorded in
NetBackup’s database. NetBackup freezes the piece of media to prevent the
possibility of overwriting data. This occurs due to one of the following situations:
• A piece of media that has been used previously with NetBackup (and has a
media ID written to the tape) is given a new and different bar-code label
• When a library does not have a bar-code reader and the inventory function is
used to populate the library’s tapes into the EMM database
When this is done, tapes receive new arbitrary media IDs within the EMM
database. As a result, if previously written NetBackup tapes are inventoried to
add them to the EMM database, they each receive a new media ID that does
not match the media ID written to the tape’s header.
To troubleshoot an external/recorded media ID mismatch:
1 Use robtest to load a volume with a specific, known bar-code into a drive.
2 Use the Device Monitor or vmphyinv to view the recorded media ID of the
7
tape.
3 To correct the problem, use one of the following methods:
– If the robot has a bar-code reader, change the bar-code label to match the
recorded media ID.
– If the robot does not have a bar-code reader, use the NetBackup
Administration Console to move and track volumes in and out of the
library, using the correct (recorded) media ID.

Repairing a Robotic Device Number
and Tape Device Mismatch
1. Use robtest to load a volume into a specific drive.
2. Use the Device Monitor to determine which drive is
actually loaded.
Repairing a Robotic Device Number and Tape Device Mismatch

If your robot and drives do not support serialization, NetBackup has no way to
determine which tape device (presented through the operating system and mapped
to a specific SCSI ID) is mapped to which robotic drive number on the robot. As a
result, it is possible to misconfigure the tape devices with the incorrect robotic
drive number.
When this happens, NetBackup instructs the robot to place a specific media ID into
a particular drive number. NetBackup attempts to read from that drive. Because the
volume is not in the expected drive, NetBackup either finds no media in the drive
(and freezes the volume) or finds a different volume (and freezes the volume).
To troubleshoot this situation:
1 Use robtest to load a volume into a specific drive.
2 Use the Device Monitor to observe the drive where the volume is placed.

Suspended Media
• NetBackup does not write to suspended media.

• You can restore data from suspended volumes.
• You must import expired images from a suspended
volume before you can restore.
• Use bpmedia to suspend or unsuspend volumes
manually.
• NetBackup does not automatically suspend media.
• A suspended volume is unavailable for future backups
until all images on the volume have expired.
• Expired media show as ACTIVE (not SUSPENDED) in the
output of the available_media script.
Suspended Media
NetBackup does not write to suspended media. After a tape volume has been
suspended, backups stored on the suspended media are still available for restores,
but no further backup images can be written to it.
If the backup images on a suspended volume have expired, you must import them
before you can restore.
NetBackup does not automatically suspend media; however NetBackups options
may need to suspend media. For example, NetBackup Vault suspends media when
they go offsite.
You may use the bpmedia command to suspend or unsuspend volumes manually
as a temporary means of software write-protecting volumes, just like freezing
volumes.
A condition similar to suspension occurs when media reaches its volume
expiration. A volume may expire because it has exceeded its:
• Maximum mount count 7
• Media expiration date (not to be confused with NetBackup image expiration
based on retention periods)
By default media has an infinite mount count and no expiration date. A media
expiration count or expiration date must be set manually by the NetBackup
administrator when new media is introduced to NetBackup in order to enforce a
limited use of media. The values are based upon:
• Your previous experience with overused media (referred to as tired or spent
media)

• Recommendations from the manufacturer for that type or brand of media
NetBackup 6.x shows media as ACTIVE in the output of the
available_media script when the volume has met the maximum mount
count or the expiration date. Use the following commands to display the volume
expiration date and remaining number of mounts:
vmquery –m media_id provides the following information for a volume,
allowing you to determine if the reason it cannot be written on is due to media
expiration (not to be confused with image expiration):
• Expiration date
• Number of mounts
• Max mounts allowed
nbemmcmd –listmedia –mediaid media_id provides similar
information as the vmquery command, without the actual mount count:
• Volume Expiration
• Mounts (that is, maximum mounts)
• No indication of a mount count (number of mounts)

Lesson Summary
• Key Points
In this lesson, you learned how to troubleshoot common media
errors and how to correct media that is FROZEN or SUSPENDED.
• Reference materials
– NetBackup Media Manager System Administrator’s Guide
– TechNotes 234412, 269177, 270101, 273849, 278996, 280309
Lab 7: Troubleshooting Media
In this lab, you will troubleshoot media-related problems:

• Run scripts that create or simulate media-related
problems.
involve NetBackup media-related errors.

7
Troubleshooting Media,” page A-46.
Solution: Troubleshooting Media,” page B-57.

Lesson 8
Troubleshooting Network Issues
Lesson Introduction
Tools
Database
Issues
Issues

Topic
will be able to:
Topic 1: NetBackup and • Explain how NetBackup uses TCP/IP.
TCP/IP • List common types of communication
errors and their causes.
Topic 2: Physical Network Isolate physical network errors.

Failures
Topic 3: NetBackup Correct NetBackup configuration errors.

Configuration Errors
Topic 4: Isolating Network Identify and resolve network errors.

Errors

NetBackup and TCP/IP
Communication
Master
Server
Tape
Library
master01
WAN 10.1.5.20
Media
Server Client
media02 client21
10.7.4.5 LAN 10.7.4.17
Topic 1: NetBackup and TCP/IP

• Explain how NetBackup uses TCP/IP.
• List common causes for and types of communication errors.
NetBackup and TCP/IP Communication

NetBackup uses TCP/IP (transmission control protocol/Internet protocol) for all
server-to-server and server-to-client communication. TCP/IP consists of the
following components:
• IP (Internet protocol)
• ICMP (Internet control message protocol)
• ARP (address resolution protocol)
• UDP (user datagram protocol)
• TCP (transmission control protocol)
It is helpful to have a good working knowledge of TCP/IP terminology and
configuration when configuring or troubleshooting NetBackup communications.
Many of the troubleshooting steps discussed in this lesson involve TCP/IP
commands or NetBackup configurations associated with TCP/IP.
Packets
TCP/IP transfers data using packets. Packets have headers that contain information
8
used to deliver the packets. Packets include the MAC (media access control)
address, the IP address, and the port number.
Lesson 8 Troubleshooting Network Issues 8–3

MAC Addresses
The MAC address is a unique number assigned by the hardware vendor. After a
packet reaches the destination network, the MAC address is used to deliver the
packet to the physical network interface. MAC addresses are used at the Open
Systems Interconnection (OSI) Data Link layer.
Address resolution protocol (ARP) is used to identify the MAC address of the
destination or the next hop. Each machine has an ARP cache, which holds recently
determined IP-address-to-MAC-address translations. The system automatically
uses ARP for the MAC address of a machine that is not already in the ARP cache.
IP Addresses
The IP address is a unique number assigned by the system administrator to identify
a logical network interface. IP addresses are hardware-independent and are used at
the OSI Network layer.
Routing
TCP/IP uses routing to direct packets from the source to the destination. Packets
may have to hop through many different gateways (routers) to reach the
destination. When a gateway receives a packet, it either delivers the packet to its
destination or delivers the packet to the next gateway as determined by the routing
table.
Sockets and Ports

NetBackup uses TCP/IP sockets for all interprocess communication. A socket is a
combination of an IP address, protocol type, and port number that identifies the
logical address that a process uses to send and receive communications.
Multiple clients can connect to the same IP:PORT on a server, but each client
process has its own socket for network interprocess communication with that
server.

Types of Communication Errors
• Physical network failures

• NetBackup configuration errors
• Network errors
• Troubleshooting tips:
– Verify connectivity to eliminate physical network failures.
– Verify the NetBackup configuration.
– Verify the network configuration.
Types of Communication Errors

Communication errors can be caused by many conditions, including:
• Physical network failures
– Failed hardware, such as NICs, switches, and routers
– Failed or disconnected cables
– Problems with the Internet service provider (ISP)
• NetBackup configuration errors
– Recent modifications
– Registry and configuration files
– Service entries
• Network errors
– The hosts file, Windows Internet Naming Service (WINS), Network
Information Service (NIS), or Domain Name System (DNS)
– Port shortages
– Multiple network interface cards (NICs)
– Firewalls
– Network timeouts
8

Isolating Physical Network Failures

Display the host network • UNIX
configuration ifconfig -a
• Windows
ipconfig /all
Verify the loopback address to ping 127.0.0.1
ensure that the TCP/IP stack is
working on the system,
ping client_hostname
Verify connectivity between the
server and the client, ping server_hostname
Test the vnetd port on the client telnet client_hostname vnetd

directly,
Topic 2: Physical Network Failures

After completing this topic, you will be able to isolate physical network errors.
Isolating Physical Network Failures

To isolate physical network failures, first check for the obvious signs of problems.
Ensure that no error indicators are lit on the hardware. Verify that all cables are
connected and firmly seated. Ensure that your ISP is functioning.
After you have performed the obvious checks, use utilities and commands to help
you verify network configuration and connectivity. These utilities and commands
are:
• ifconfig -a (UNIX) or ipconfig /all (Windows)
• ping
• telnet
• traceroute (UNIX) or tracert (Windows)
Displaying the Current Network Configuration

Use the ifconfig -a command (UNIX) or the ipconfig /all command
(Windows) to display the current configuration for each NIC on a system.
The type of information that is displayed includes:
• IP address
• Interface name that the operating system uses to communicate with the NIC
• MAC address
• NIC status (UP or DOWN)

• Subnet mask
You can change the interface parameters by accessing the properties of TCP/IP in
the properties of the NIC in Windows.
When troubleshooting problems with multiple NICs, use the route print
command (Windows) or the netstat -r command (UNIX).
On UNIX systems, you can also use the ifconfig command to modify NIC
parameters. Modifications can include:
• Bringing the network interface up or down
• Changing the IP address or network mask
Note: On Solaris, the NIC name is specified in the /etc/hostname.xxxx file,

where xxxx is the interface name. For example: /etc/hostname.eri0
The ping Utility

ping is a utility that is used to verify that the TCP/IP protocol stack is working
and to verify connectivity to a destination machine. ping operates as follows:
• The source machine sends an internet control message protocol (ICMP)
request packet to the destination.
• The destination returns an ICMP reply packet.
Host Names
To isolate physical network failures, you need to know the host names of the
systems. Host names are configured in the NetBackup policy configuration or in
the hosts file in /etc (UNIX) or in
%SystemRoot%\system32\drivers\etc (Windows). Host names are
also configured in NIS and DNS (if applicable).
Using ping and telnet

Each system has an internal loopback address (127.0.0.1). Verify that the system’s
TCP/IP protocol stack is operational by pinging the loopback address.
Verify network connectivity by pinging the client from the server and pinging the
server from the client.
ping client_hostname
ping server_hostname
You can also test the bpcd port on the client directly, as follows:
telnet client_hostname 13724
telnet client_hostname vnetd
If ping fails, the source or destination machines may have broken hardware, IP
network configuration problems, or ICMP packets are blocked by a firewall. If
8
ping passes, but a telnet to one of the NetBackup ports fails to connect, then a
firewall/router/NetBackup configuration problem may exist.

Using traceroute (UNIX) and
tracert (Windows)
Network Layer Functionality/Implementation Coverage
Application NetBackup daemons or services and

processes
TCP Transport Reliable data segment delivery
IP Network Routable datagram service
Data Link LAN/WAN packet frames
Physical Hardware: NIC interfaces, cables,

switches
Test coverage by ping and traceroute/tracert
Test coverage by telnet
Test coverage by a successful backup or restore
Using traceroute (UNIX) and tracert (Windows)

The traceroute (UNIX) and tracert (Windows) utilities function as an
incremental ping by stepping through the intermediate routers and gateways
between a requesting system and the remote destination host. This capability
allows you to quickly determine where there is a break in a complex mesh-
topology network.
By default, traceroute and tracert query three times per hop (router)
instead of sending just one request or reply in order to provide a decent indication
of response delays at each hop. By default, traceroute and tracert also try
to resolve raw IP addresses to hostnames, which can add a significant amount of
time to tracing a multiple-hop route.
Use traceroute –n remote_host (UNIX) or
tracert –d remote_host (Windows) to eliminate the name resolution step
at each hop.

Isolating NetBackup Configuration
Errors
If there have been recent modifications, ensure that the
modifications have not introduced the problem.
• Is the client operating system (OS) supported by this
version of NetBackup?
• Has the client software been installed?
• Are the servers at the same or a higher NetBackup
version than the client?
• Is the client’s binary still supported?
• Have all the latest patches been installed?
Topic 3: NetBackup Configuration Errors

After completing this topic, you will be able to correct NetBackup configuration
errors.
Isolating NetBackup Configuration Errors

After you have eliminated physical network failures as the cause of the problem,
determine if NetBackup configuration errors are causing the problem. NetBackup
configuration errors may occur because of:
• Recent modifications
• Problems with server and client names
• Service entry issues
Recent Modifications
If there have been recent modifications, ensure that the modifications have not
introduced the problem by verifying the following items:
• Is the client operating system (OS) supported by this version of NetBackup?
• Has the client software been installed?
• Are the servers at the same or a higher NetBackup version than the client?
• Is the client’s binary still supported?
• Have all the latest patches been installed?
Note: Mismatched binaries may cause a status code 25 error: cannot
8
connect on socket.

Resources
The NetBackup Release Notes and Symantec Technical Support
(http://entsupport.symantec.com) provide a list of supported
operating system versions for server and client software.
Version and patch information is available in the following files:
• UNIX:
/usr/openv/netbackup/bin/version
/usr/openv/pack/pack.summary
Servers also have additional version information in
/usr/openv/netbackup/version.
• Windows:
install_path\NetBackup\bin\version.txt
install_path\Patch\History.Log
Servers also have additional version information in
install_path\NetBackup\version.txt.
Other methods of determining the version include:
• GUI:
In the NetBackup Administration Console or the Backup, Archive, and Restore
interface, select Help—>About.
• Registry:
Check the VERSION and PackageVersion values in the
HKEY_LOCAL_MACHINE\Software\VERITAS\NetBackup\
CurrentVersion key.
• Windows Explorer:
To check an individual executable in Windows Explorer, right-click the file,
select Properties, and then check the Version tab.

Server and Client Names
• Windows and UNIX clients:

Use the Client Host Properties to verify the:
– Client name setting in the Client Name tab
– Server names in the Servers tab
• UNIX clients only:
Verify the SERVER and CLIENT_NAME entries in the
bp.conf file.
Server and Client Names

On Windows and UNIX servers and clients, use the Client properties dialog box to
check the Client name setting in the Client Name tab and the server entries in the
Servers tab.
For UNIX servers and clients, verify that the bp.conf file in the
/usr/openv/netbackup directory contains server entries for the master
server and media servers. If the CLIENT_NAME parameter is configured, verify
that the entry is correct.
The CLIENT_NAME parameter specifies the name of the client as it is known to
NetBackup. CLIENT_NAME is an optional parameter, but it is sometimes set to a
different host name for special circumstances, such as alternate client restores.
This can cause a problem connecting to the client if the host name is changed for a
special circumstance and not changed back to the original value when the restore is
finished.

Host Name Resolution
The /etc/hosts File (UNIX only)

During packet delivery, the host name is translated to an IP address using the
/etc/hosts file. The /etc/hosts file has a single-line entry for each host
that contains the IP address, the host name, and optional alias names.
When a NetBackup system receives a packet, it resolves the source IP address to a
host name. If the host name is not listed in the bp.conf file, the connection fails.
The master server must be the first SERVER = entry listed in the bp.conf file.
The hosts File (Windows only)

Every machine has a host name (a name that is easier for users to remember than a
number). During packet delivery, the host name is translated into an IP address
using the %SystemRoot%\system32\drivers\etc\hosts file. The
hosts file contains a single-line entry for each host that contains:
IP_address host_name [alias_names ...]
When a NetBackup system receives a packet, it resolves the source IP address to a
host name. If the host name is not listed in the Servers tab of the Host Properties
window, the connection fails.
In the output of the bpgetconfig SERVER command, the master server must
be the first SERVER entry listed.
Typical Host Name Resolution Order (UNIX and Windows)

The IP stack typically resolves host names in the following order:
1 Checks to see if the name queried is itself (localhost)
2 Checks the contents of the:

– UNIX: /etc/hosts file
– Windows: DNS client resolver cache, which initially is loaded with the
contents of the %SystemRoot%\system32\drivers\etc\hosts
file
3 Queries the designated DNS servers
4 Windows allows for a fourth possibility, by performing a NetBIOS over TCP/
IP COMPUTERNAME lookup:
a NetBIOS name cache
b NetBIOS Name Server (NBNS) also known as Windows Internet Name
Server (WINS)
c B-node Broadcast on local subnet
d Contents of the
%SystemRoot%\system32\drivers\etc\lmhosts file
The host name resolution order on Windows is documented in:
http://technet.microsoft.com/en-us/library/
bb727005.aspx.

Using the bpclntcmd Command
Resolve an IP address to a host name, bpclntcmd -ip ip_address
Resolve a host name to an IP address, bpclntcmd -hn host_name
Display what the master server

bpclntcmd -pn
determines is your peer name,
Display the master server’s

bpclntcmd -sv
NetBackup version number,
Using the bpclntcmd Command

The NetBackup bpclntcmd command resolves IP addresses into host names and
host names into IP addresses by using the same system calls as the NetBackup
application software.
bpclntcmd is useful for testing the functionality of the host name and IP address
resolution.
bpclntcmd -ip IP_Address

The -ip option enables you to specify an IP address. bpclntcmd uses
gethostbyaddr() on the NetBackup node, and gethostbyaddr() returns
the host name associated with the IP address as defined in the node’s DNS,
Windows Internet Naming Service (WINS), NIS, or local hosts file entries. No
connection is established with the remote NetBackup host.
bpclntcmd -hn host_name

The -hn option enables you to specify a host name. bpclntcmd uses
gethostbyname()on the NetBackup node to obtain the IP address associated
with the host name defined in the node’s DNS, WINS, NIS, or local hosts file
entries. No connection is established with the remote NetBackup host.
You can use the -ip and -hn options to verify the ability of a NetBackup node to
resolve the IP addresses and host names of other NetBackup nodes.

bpclntcmd -pn
When run on a NetBackup host, the –pn option initiates an inquiry to the
NetBackup master server, and the server returns what it determines to be the
requesting host’s name and IP address. Unlike the –ip and –hn queries, this
command does attempt to communicate with the master server.
This command returns nothing, not even an error message, if the master server is
down, if the requesting host has an incorrect
bp.conf SERVER = master_server_name entry, or if the host cannot
resolve the address of the master server.
bpclntcmd -sv
The -sv option displays the NetBackup version number of the master server.
Similar to the –pn query, this operation waits for a response back from the known
master server. Both bpclntcmd –pn and bpclntcmd –sv are effective, yet
simple commands that can be used to prove high-level connectivity between a
NetBackup host and its master server.

Hosts with Multiple NICs
• Status code 59: Access to client not allowed

• Hung oprd processes
Common
errors • No output from the vmoprcmd command
• No vmd connection
• No route found
Master Server
Client Media Server
REQUIRED_INTERFACE
NetBackup vm.conf
Configuration
Determine the name of the system that initiated the last

communication. Verify that the name matches an entry in the NBU
configuration.
Hosts with Multiple Network Interface Cards (NICs)

Each network interface card (NIC) on a master server, media server, or client has a
NIC name and an IP address. Generally, if there is only one NIC on the system, the
NIC name matches the system’s host name.
Note: In this context, the NIC name is the name associated with the IP address of
the interface, for example, with DNS. This is not the operating system’s
name for the interface, for example, as returned by ipconfig /all
(Windows) or ifconfig -a (UNIX).
When there are multiple NICs on a system, each NIC has a unique name and IP
address. If there are multiple NICs on a system and NetBackup has not been
configured to use a specific NIC, the operating system determines which NIC is
used for backup and restore operations.
If the NIC chosen by the operating system does not have an entry in the
NetBackup configuration, a communication error occurs. Depending on the
direction of the communication (client—>master server, master server—>client,
master server—>media server), error symptoms may include:
• Status code 59: Access to the client not allowed
• Hung oprd processes

Tape device status is sent between rdevmi running on the scan host and
oprd on the media server. oprd writes these messages to the ltid process
by using the LTIR IPC message queue. If the LTIR IPC message queue is not
large enough to hold all messages, oprd is suspended until ltid reads from
the queue.
• No vmoprcmd output
• No vmd connection
• No route found
On a Windows system, WINS associates the NetBIOS name (the host name) with
an IP address. If the host name is not associated with the interface you want to use
for NetBackup, you can run into name resolution problems, even if DNS is
properly configured. A possible problem is a failure when you attempt to start the
NetBackup Administration Console. One solution is to use a different computer
name than the name given to the desired interface. Use the name of the desired
interface as the server name in NetBackup.
Using the REQUIRED_INTERFACE Parameter

To prevent communication errors on a master server, a media server, or a client
with multiple NICs, it may be necessary to force NetBackup or Media and Device
Management to transmit from a specific NIC. A proper routing table configuration
should assist in supporting multiple NICs or subnets. If the routing table is not
sufficient, you may use the REQUIRED_INTERFACE parameter to direct
NetBackup traffic through a specific interface.
Note: If you implement the REQUIRED_INTERFACE option, ensure that you

specify the interface name and not the IP address. The name of the interface
is the name associated with the assigned IP address in the hosts file.
In the NetBackup Administration Console, in the Host Properties window under

the Universal Settings, type the interface name in the Required interface name text
box. Alternatively, use the bpsetconfig command to set this parameter.
On UNIX, set the REQUIRED_INTERFACE parameter in the bp.conf and
vm.conf files.
In the following example from the hosts file, a client named rocky has two
NICs. The first NIC has an IP address of 10.0.10.10 and is named rocky. The
second NIC has an IP address of 10.0.20.15 and is named rocky-bu. The second
NIC is the NIC that is designated for backup and restore operations.
10.0.10.10 rocky rocky.acme.com
10.0.20.15 rocky-bu rocky-bu.acme.com
To force backups and restores over the NIC that is designated for NetBackup
operations, the client should have the following parameters set:
8
CLIENT_NAME = rocky-bu
REQUIRED_INTERFACE = rocky-bu

Note: In this example, the client name in the policy must also be rocky-bu.
If a media server uses the REQUIRED_INTERFACE parameter, ensure that all

references to that media server match the parameter value. Items that must match
include:
• The media server name in the storage unit configuration
• The device host name in Media and Device Management
• The media server listing on all other servers and clients
If the master server uses the REQUIRED_INTERFACE parameter, ensure that all
references to the master server match the parameter value. Specifically, the first
SERVER = entry on all clients and media servers must list the master server by its
REQUIRED_INTERFACE name.
• For Windows media servers and clients, check the Servers tab of the Host
Properties or the Server REG_MULTI_SZ value in the
HKEY_LOCAL_MACHINE\SOFTWARE\VERITAS\NetBackup\
CurrentVersion\Config Registry key.
• For UNIX media servers and clients, the first SERVER = entry in the
bp.conf file must list the master server by its REQUIRED_INTERFACE
name.

The services File
NetBackup Server NetBackup Client
The services File

During the installation process, default services entries for NetBackup are inserted
into the /etc/services file (UNIX) or the
%SystemRoot%\system32\drivers\etc\services file (Windows) on
all NetBackup servers and clients. Verify that each server and client has the
required entries.
• On NetBackup servers, check the services files to ensure that there are
entries for bprd, bpdbm, bpcd, vnetd, bpjobd, NB_dbsrv, vmd, and the
processes for all configured robots.
• On NetBackup clients, check the services files to ensure that there are
entries for bprd, bpcd, and vnetd.
The services file converts service names to port numbers and port numbers to
service names. The services file contains a list of:
• Processes and services
• Process and server port numbers:
– 0–1023 = Well-known services (WKS)
– 1024–49151 = Registered ports
– 49152–65535 = Dynamic/private ports
• The protocol to use for communication
8

Network Errors
• bpcd/vnetd startup
– On Windows servers and clients, verify that the Client service is
running.
– On UNIX servers and clients, verify that the /etc/inetd.conf
file has a bpcd and a vnetd entry.
• Ensure that DNS, WINS, or NIS host name information
matches the NetBackup policy and host name
configuration.
– Check the Servers tab and Client Names tab for Windows
servers and clients.
– Check the bp.conf file on UNIX servers and clients.
• If Network Information Service (NIS) is being used,
ensure that NetBackup services are included.
Topic 4: Isolating Network Errors

After completing this topic, you will be able to identify network errors.
Network Errors
If the NetBackup configuration is not the problem, you may have a network error.
Network errors may be caused by:
• NetBackup Client service startup
• DNS, WINS, or NIS
• Port problems
• Clients with multiple network interface cards (NICs)
• Firewall issues
• Network timeouts
NetBackup Client Service Startup

On Windows servers and clients, the NetBackup Client service listens for service
requests on behalf of bpcd and vnetd. The executable associated with the
NetBackup Client service is named bpinetd.exe.
The Client service should always be running on a Windows machine. Restart the
Client service if you change the service port.

The inetd.conf File (UNIX only)
On UNIX systems, inetd is a process that listens for service requests on behalf
of processes that are listed in the /etc/inetd.conf file. NetBackup uses
inetd on UNIX servers and clients to provide a listener service on behalf of
bpcd. During the NetBackup installation, bpcd and vnetd entries are inserted
into the inetd.conf file, as shown in the graphic.
You can verify that this entry is working properly by examining the inetd.conf
file and using the netstat -a command to see if a listener for bpcd is started.
Use kill –HUP inetd_pid to restart inetd listeners after service port
configuration changes.

Name Resolution
DNS
The domain name system (DNS) handles the mapping between host names and IP
addresses, in addition to other host information in distributed environments.
Before DNS, system administrators had the overwhelming job of maintaining the
hosts file. The hosts file needed an entry for every possible host with which
the user communicated. DNS uses a client-server method of providing network
services.
The resolv.conf File (UNIX only)

The client-server method of providing network services uses resolvers and name
servers. With resolvers and name servers, configuration files are managed in a
single location, eliminating the need to copy configuration files from host to host.
• A resolver identifies the name server to query for host name-to-IP-address
mapping services.
– On Windows, the resolver is configured by the DNS values in the TCP/IP
properties of the network interface.
– On UNIX, the resolver uses a configuration file called
/etc/resolv.conf to identify the name server to query for host name-
to-IP-address mapping services.

• When queried by a resolver, a name server provides information that includes
host name-to-IP-address mapping.
The nsswitch.conf File (UNIX only)

On UNIX systems, the /etc/nsswitch.conf file identifies whether name
resolution is performed by DNS, the /etc/hosts file, or both. If name
resolution is performed by DNS and the /etc/hosts file, it is important that
changes to the DNS are reflected in the /etc/hosts file and that changes to the
/etc/hosts file are reflected in the DNS.
Testing DNS Lookups

Use the nslookup utility to identify your default name server and to query
forward and reverse resolutions for addresses and names for your NetBackup
hosts.
Use nslookup interactively as follows:
1 Type set debug at the prompt.
2 At the next prompt, enter the name or IP address of the host you wish to
resolve.
WINS
The Windows Internet name service (WINS) is a mechanism implemented by
Windows systems to centrally record host name-to-IP translations. WINS is
especially useful when dynamically assigned IP addresses are used. A server or
servers must be configured to run WINS, and all clients that use it must be
configured with the address of the WINS servers to which they register and query.
NIS
The Network Information Service (NIS) is a mechanism implemented by SUN for
keeping major files synchronized between hosts. NIS can be used to manage
/etc/hosts, /etc/services, /etc/password, and other files required
for hosts on a network. On Solaris you can use NIS, DNS, or /etc/hosts. You
can also use DNS as a backup to NIS.

Port Problems
Common causes of port problems include:

• The socket is being used by a non-NetBackup process.
• The operating system has not timed out of its previous
use of a socket.
• Random port assignment has been disabled, and a port
range that is too small has been defined for NetBackup’s
use.
Port Problems
Common causes of port problems include:
• The socket is being used by a non-NetBackup process.
• The operating system has not timed out of its previous use of a socket.
• Random port assignment has been disabled, and a port range that is too small
has been defined for NetBackup’s use.
To verify that a port can be contacted, enter the telnet command as follows:
telnet host_name port_number

Isolating a Port Shortage Problem
Status code Cannot connect to the backup restore

205: manager
Status code
Storage unit is not available
213/219:
Status code 54: Timed out connecting to the client
• NetBackup: vnetd, bpcd, bprd

Helpful logs:
• Media Manager: reqlib
Isolating a Port Shortage Problem

Port shortages produce a variety of status codes, including:
• 205: Cannot connect to backup restore manager
• 213: No storage unit is available for use
• 219: The required storage unit is not available
• 54: Timed out connecting to client
The vnetd, bpcd, bprd, and reqlib debug logs are helpful when
troubleshooting port shortage problems.
Note: When using debug logs to verify a port problem, set the verbose level to 5
before retrying the backup.

Troubleshooting Port Problems
Step Action
Stop the NetBackup processes (if possible) on the server and
1
the client.
2 Use netstat –a to display the sockets being used.
3 Start the NetBackup processes and force the client backup.
Check the debug logs (vnetd, reqlib, and bpbrm) to

4 determine if NetBackup is trying to use sockets that are
already in use by other processes.
If a conflict exists, modify ports that are used by the other
5
processes or modify ports that are used by NetBackup.
Troubleshooting Port Problems

To determine if a socket is being used by a non-NetBackup process:
1 Stop the NetBackup processes (if possible) on the server and the client.
This eliminates the possibility that NetBackup is holding the ports open.
2 Use netstat –a to display the sockets being used.
Note the sockets that are used when NetBackup is not running.
3 Start the NetBackup processes and force the client backup.
4 Check the debug logs (vnetd, reqlib, and bpbrm) to determine if
NetBackup is trying to use sockets that are already in use by other processes.
The sockets that NetBackup is trying to use are identified by messages similar
to the following:
getsockconnected: Connect to server_name on port
port_number
bind_on_port_addr: bound to port port_number
Compare the ports that are listed and the ports that are displayed by using the
netstat -a command to identify if NetBackup is conflicting with another
process.
5 If there is a conflict, modify ports that are used by other processes (if possible)
or modify ports that are used by NetBackup.

Using the bptestbpcd Utility
Check NetBackup bptestbpcd

connectivity to a client, –client client_name
Check NetBackup
bptestbpcd
connectivity to a
–host server_name
server,
bptestbpcd –verbose
Display connectivity –debug
check details, –host server_name|client_name
Using the bptestbpcd Utility

Client-server connections at a NetBackup communication level can be tested with
bptestbpcd. Parameter switches and client connect options allow you to test a
NetBackup client or a host server for accessibility. You may also use -verbose
and -debug switches to gain additional detailed troubleshooting information.
bptestbpcd Usage Examples

The following are examples of how to use bptestbpcd to test various
conditions:
• Test the master server (train1) itself for vnetd/bpcd connectivity:
• Test from the master server to a NetBackup client (train12):

8

• Test from the master server to a known media server (train2):
• Test where a media server is also a client:
• Problem: The media server is unknown to the client (there is no

server = train2 in the client’s bp.conf file):
• Problem: Test from a master server to a system that does not have NetBackup
installed (train5):

About CONNECT_OPTIONS

Try different connect bptestbpcd
options, –connect_options 1 2 3
–host server_name
–client client_name
bpcd connect-
Port type to use
back method
DEFAULT_CONNECT_OPTIONS = [ 0 | 1 ] [ 0 | 1 ] [ 0 | 1 | 2 ]
Daemon
connection port
About CONNECT_OPTIONS
Use CONNECT_OPTIONS to specify one of three options designed to enhance
firewall efficiency with NetBackup:
• Connect to the host using a reserved or non-reserved port number.
• Connect to the host using the traditional call-back method or using the Veritas
Network Daemon (vnetd).
• Connect to the host using one of the following methods:
– vnetd or, if vnedt fails, the daemon's port number
– vnetd only
– The daemon's port number only
The bptestbpcd utility can help identify which NetBackup
CONNECT_OPTIONS are required for your network environment. By default,
NetBackup is “firewall friendly,” using the fewest possible ports required, but the
default may not work for a legacy site.
After you find an operable and acceptable connectivity method, you may force
NetBackup to use a new DEFAULT_CONNECT_OPTIONS or even allow per-
client/per-server CONNECT_OPTIONS overrides.
8

These options can be set in the NetBackup Administration Console Firewall host
properties. On UNIX systems these options can also be directly entered in the
/usr/openv/netbackup/bp.conf file, in the following format:
DEFAULT_CONNECT_OPTIONS = [ 0 | 1 ] [ 0 | 1 ] [ 0 | 1 | 2 ]
CONNECT_OPTIONS = host_name [ 0 | 1 | 2 ] [ 0 | 1 | 2 ]
[ 0 | 1 | 2 | 3 ]
Where
• host_name is the host name of the server or client to which to connect. The
host must run NetBackup version 4.5 or greater.
• Port type to use:
0 = Use a reserved port number (default).
1 = Use a non-reserved port number.
You must also enable Accept Connection on Non-reserved Ports for the
selected host.
2 = Use the DEFAULT_CONNECT_OPTIONS value.
• BPCD connect-back method:
0 = Use the traditional connect-back method.
1 = Use the VNETD no connect-back method (default).
• Daemon connection port:
0 = Connect to a daemon on the host using VNETD if possible; otherwise,
connect using the traditional port number of the daemon (default).
1 = Connect to a daemon on the host using VNETD only. This setting enables
unidirectional BPCD.
2 = Connect to a daemon on the host using only the traditional port number of
the daemon.
Note: If VNETD only (1) is selected as the daemon connection port, the “BPCD
connect-back method” setting is not applicable and the non-reserved ports setting
(1) is always used regardless of the value of the “port type to use” setting.
NetBackup Performance Note: (source: NetBackup Release Impact Bulletin)
When running many simultaneous jobs, the CPU usage of the master server may
become very high. To reduce the usage and improve your system performance, you
can adjust the network connect options for the local machine on the Firewall Host
Properties page in the NetBackup Administration Console, or you can add the
following bp.conf entry to the UNIX master server:
CONNECT_OPTIONS = localhost 0 0 2

CONNECT_OPTIONS Example
The following example shows how to test connectivity with specific
CONNECT_OPTIONS (using a reserved port and the traditional connect-back
method, connecting to the daemon using the traditional port number of the daemon
only):
train1 !# bptestbpcd -connect_options 0 0 2 -host
train2 -verbose 0 0 2
192.168.27.101:841 -> 192.168.27.102:13782
192.168.27.101:935 -> 192.168.27.102:541
PEERNAME = train1.veritas.com
HOSTNAME = train2
CLIENT_NAME = train2
VERSION = 0x06000000
PLATFORM = solaris8

Debug Logs Identify a Problem
• If a port problem is identified in the logs but no conflicts

are detected, the operating system may not have timed
out of a socket now being requested by NetBackup.
• Try the following solutions:
– Reduce the setting for the TcpTimedWaitDelay
(tcp_time_wait_interval) parameter.
– Expand the reserved port range.
– Configure usage of the non-reserved port range.
– Enable use of random port assignments.
Logs Identify a Problem—No Conflicts Detected

If a port problem is identified in the logs (bprd, reqlib, bpcd) but no port
conflicts are detected, the operating system may not have timed out of a socket
now being requested by NetBackup. This problem may occur on very busy media
servers with many concurrent client backups. Try the following solutions:
• Reduce the setting that determines how long to reserve the port after a process
has released it. Because database extension backups start multiple processes, it
is possible that a busy server can run out of ports that it is allowed to allocate
for these backups, because it is waiting four minutes (by default) for each
process. Most modern networks that do not deal with high latency can safely
reduce this value to a more logical length. Reducing the length to 30 seconds,
for example, allows more ports to reopen in a shorter amount of time, enabling
additional processes to reuse these ports. The reduction of this parameter has
two effects:
– Reduce the amount of time an established connection waits before giving
up, increases the number of ports available for use at any particular
moment.
– Although there are more available ports on the host, the impact is increased
network traffic due to excessive reestablishment of these connections.
The exact parameter and the method of changing it is operating-system
dependant:
– On Windows, this parameter is a Registry value named
TcpTimedWaitDelay in the key
HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\

Services\Tcpip\Parameters; it is a REG_DWORD value, measured
in seconds, with a default of 240.
– On Solaris 8 and later, this parameter is the
tcp_time_wait_interval parameter. The default setting on a
Solaris installation is 240000 milliseconds (four minutes).
To display the current setting (in milliseconds), run the following
command:
ndd -get /dev/tcp tcp_time_wait_interval
To change the current setting, run the following command:
ndd -set /dev/tcp tcp_time_wait_interval value
The ndd command makes the change immediately, without requiring a
reboot. This setting returns to the default after a reboot.
• Expand the reserved port range.
This is a quick way to make more ports available for use. However, because
the decision to use a reserved port range is often made due to security
considerations, security policies may prohibit having an expansive range of
ports available.
• Configure usage of the non-reserved port range.
Using non-reserved ports requires the regular inspection of the state of many
ports to maintain security. The failure to inspect the state of ports regularly can
represent a vulnerability in network security where traffic originating outside
the network can masquerade as established TCP/IP traffic.
– When Allow Non-reserved Ports is enabled, bpcd can accept remote
connections from nonprivileged ports (port numbers 1024 or greater).
– When Allow Non-reserved ports is disabled, bpcd requires remote
connections to come from privileged ports (port numbers less than 1024).
• Determine if Random Port Assignments has been disabled.
Enabling Random Port Assignments reduces the probability of a specific
socket or socket range being in use. However, before enabling this parameter,
ensure that it was not disabled because of a firewall requirement.

Configuring Port Parameters
Use the Port Ranges dialog box on the master server, media server or clients to:
• Enable or disable Use random port assignments.
• Enable or disable Use OS selected non-reserved ports.
• Expand port ranges.
Refer to TechNote 234618, Unable to allocate reserved port yields 219 or 205 in
bptm log, for more information on ports.

Configuring vnetd for Server-to-Server Communication
The Veritas Network Daemon (vnetd) was designed to enhance firewall
efficiency with NetBackup during server-to-server and server-to-client
communications. vnetd reduces the number of ports required by eliminating the
call-back protocol used by bpcd and bpjava.
The graphic on this page shows that NetBackup 6.x uses vnetd by default, so no
further configuration is required, unless you want to disable this feature for a
particular host. To do this, add the media server to the Hosts list, select Random
port for the BPCD connect back attribute, and select Automatic or Daemon port
only for the Daemon connection port attribute
Use the bpgetconfig command to display Port Ranges and Firewall
information. Output is similar to the following:
CLIENT_RESERVED_PORT_WINDOW = 512 1023
SERVER_RESERVED_PORT_WINDOW = 512 1023
CLIENT_PORT_WINDOW = 0 0
SERVER_PORT_WINDOW = 1025 1025
ALLOW_TPC = NO
CONNECT_OPTIONS = pc3train05 0 1 1
8

DEFAULT_CONNECT_OPTIONS = 0 1 0
ACCEPT_PORT = AUTOMATIC
On UNIX, this information is also available in the bp.conf file.

PBX Logging
Example: Review PBX logging for the past 4 hours:
vxlogview -p 50936 -o 103 -t 4:00:00
08/20/06 17:34:08.223 [Info] PBX_Manager:: handle_input with fd = 4

08/20/06 17:34:08.226 [Info] PBX_Client_Proxy::parse_line, line = ack=1
08/20/06 17:34:08.227 [Info] PBX_Client_Proxy::parse_line, line =
extension=EMM
08/20/06 17:34:08.227 [Info] hand_off looking for proxy for = EMM
08/20/06 17:34:08.228 [Info] is_accepting about to return true
08/20/06 17:34:08.228 [Info] Client Expects an ACK, sent one.
08/20/06 17:34:08.229 [Info] Proxy found.
08/20/06 17:34:08.229 [Info] PBX_Client_Proxy::handle_close
08/20/06 17:34:08.244 [Info] PBX_Manager:: handle_input with fd = 4
08/20/06 17:34:08.246 [Info] PBX_Client_Proxy::parse_line, line = ack=1
08/20/06 17:34:08.246 [Info] PBX_Client_Proxy::parse_line, line =
extension=EMM
08/20/06 17:34:08.247 [Info] hand_off looking for proxy for = EMM
08/20/06 17:34:08.247 [Info] is_accepting about to return true
08/20/06 17:34:08.248 [Info] Client Expects an ACK, sent one.
08/20/06 17:34:08.248 [Info] Proxy found.
08/20/06 17:34:08.250 [Info] PBX_Client_Proxy::handle_close

NetBackup 4.5 introduced vnetd to reduce the number of ports that needed to be
open through a restricted router or firewall. NetBackup 6.x continues that trend,
with the goal to direct interprocess communications through a common
infrastructure core services (ICS) component called Private Branch Exchange
(known by the software package VRTSpbx, or the daemon pbx_exchange, or
just simply PBX).
The new services in NetBackup 6.x, including EMM, require a PBX port (1556) to
be open bidirectionally between the master, media, and EMM servers.
UNIX
On UNIX, determine if the service is running with the bpps –x command.

Windows
Issuing a NET START command on Windows shows Veritas Private Branch
Exchange if it is a currently running service.
UNIX and Windows
On both UNIX and Windows, use the pbxcfg utility to identify current
configuration settings. Run pbxcfg -p from the following location:
• UNIX: /opt/VRTSpbx/bin
• Windows: c:\Program Files\VERITAS\VxPBX
The output is similar to the following:
Auth User:0 : root (UNIX) or localsystem (Windows)
Secure Mode: false
Debug Level: 10
Port Number: 1556
PBX service is not cluster configured
PBX uses unified logging with an originator ID of 103. Because PBX is a core
service, error messages regarding it may appear not only in the PBX log, but also
in logs for nbemm, nbpem, nbrb or nbjm. Review PBX logs using the
vxlogview command.
Configure verbose PBX logging by using pbxcfg –s –l 10.
Raw unified logs for PBX are located at the following location:
• UNIX: /opt/VRTSpbx/log
• Windows: C:\Program Files\VERITAS\VxPBX\bin

Open Port Requirements for Media
Servers Behind a Firewall
• Master server Æ port 13724 (vnetd) on media server
• Master server Æ port 1556 (veritas_pbx) on media
server
• Media server Æ port 13724 (vnetd) on master server
• Media server Æ port 1556 (veritas_pbx) on master
server
• Media server Æ port 1556 (veritas_pbx) on EMM
server
• EMM server Æ port 1556 (veritas_pbx) on media
server
• Media server Æ port 13724 (vnetd) on media server
See the NetBackup Port Usage Guide
Open Port Requirements for Media Servers Behind a Firewall

Open the following ports on the firewall if filtering by destination port:
• Master server—>port 13724 (vnetd) on media server
– Determines the NetBackup version of the media server
– Starts bpbrm for backups and restores
– Starts bptm to manage tape storage units
– Starts bpstsinfo to manage disk storage units
– Accesses or updates host properties for the media server
• Master server—>port 1556 (veritas_pbx) on media server
– Is the connect-back for job information
– Is the connect-back for resource information
• Media server—>port 13724 (vnetd) on master server
– Accesses legacy policy information from bpdbm
– Accesses legacy job information from bpjobd
– Updates image catalog information to bpdbm
– Makes miscellaneous requests to bprd
• Media server—>port 1556 (veritas_pbx) on master server
– Accesses job information
– Accesses resource information
8
• Media server—>port 1556 (veritas_pbx) on EMM server

Accesses information about device, media, and storage databases

• EMM server—>port 1556 (veritas_pbx) on media server
Is the connect-back for information about device, media and storage databases
• Media server—>port 13724 (vnetd) on media server
Establishes sockets to other media servers for duplication, disk staging, and
synthetics
The source port number is taken from the client port window or the client reserved
port window.
For additional information, refer to the NetBackup Port Usage Guide, which is
downloadable from the support Web site at

Configuring vnetd for Client-to-Server Communication
In NetBackup 6.x, communication with the client is routed through vnetd by
default. To change the default setting, use Host Properties—>Master Servers
—>Client Attributes, as shown above.
To ensure vnetd is enabled, use bpclient -client client_name -L.
The output produced is similar to the following:
bpclient -client train2 -L
Client Name: train2
Current Host:
Hostname: train2
IP Address: 192.168.27.102
Dynamic Address: no
Free Browse: Allow
List Restore: Not Specified
Max Jobs This Client: Not Specified
WOFB Enabled: yes
8
WOFB FIM: VSP

WOFB Usage: Individual Drive Snapshot

WOFB Error Control: Abort on Error
Connect options: 2 1 3
For vnetd to be used, the second option (the 1 in the 2 1 3 sequence in this
example) must be either 1 (vnetd port) or 2 (use the
DEFAULT_CONNECT_OPTION value).

Open Port Requirements for Clients
Behind a Firewall
• Client Æ port 13724 (vnetd) on master server
• Media server Æ port 13724 (vnetd) on client
• Master server Æ port 13724 (vnetd) on client
Open Port Requirements for Clients Behind a Firewall

The following are the open port requirements for clients behind a firewall:
• Client—>port 13724 (vnetd) on master server
Makes backup, restore, and miscellaneous requests to bprd
• Media server—>port 13724 (vnetd) on client
– Determines the NetBackup version of the client
– Establishes sockets to back up or restore a client
• Master server—>port 13724 (vnetd) on client
– Determines the NetBackup version of the client
– Obtains a list of mount points for multistreamed backups
– Accesses or updates the host properties for the client
The source port number is taken from the client port window or the client reserved
port window.
8

Common Firewall Issues
Status code 54: Timed out connecting to the client
Status code 57: Client connection refused
Status code 59: Access to client not allowed
Common Firewall Issues

The following status codes are often seen in a firewall configuration:
• Status code 54: Timed out connecting to client
The server could not complete the connection to the client. The accept system
call timed out after 60 seconds.
• Status code 57: Client connection refused
The client refused a connection on the port number for bpcd/vnetd.
• Status code 59: Access to the client was not allowed.
The server is trying to access the client, but the server is not recognized by the
client as a valid server.
Normal network troubleshooting procedures may not work in a firewall
configuration, depending on the firewall restrictions.

Troubleshooting Firewall Issues
• Use the bpcd/vnetd log to ensure that the client and

master/media server are able to establish a connection
through the firewall.
• Ensure that the firewall has a path in both directions;
master/media serverÆclient and clientÆmaster/media
server.
• For network connection timed out errors, ensure that the
firewall has the required ports open for the TCP
transport.
• Ensure that the firewall TCP idle timeout setting is set
high enough for NetBackup operations.
• Ensure that the firewall is not performing dynamic
network address translation (NAT) or port address
translation (PAT).
Troubleshooting Firewall Issues

1 Add a bpcd log directory to the /usr/openv/netbackup/logs
directory (UNIX) or the install_path\NetBackup\logs directory
(Windows) on the client.
2 Ensure that VERBOSE mode is enabled.
3 Retry the backup. If the firewall is configured correctly, create an entry in the
bpcd/vnetd log.
– If an entry is not created in the bpcd/vnetd log, there is no path from the
master or media server to the client. This can be a network issue or an
improper configuration of the firewall.
Try to telnet to the client to verify that bpcd is configured and
listening.
– If an entry is created in the bpcd/vnetd log, there is a path through the
firewall to the client. However, there may be a problem because there is no
path back from the client to the master or media server.
Refer to TechNote 267977 on How to enable Microsoft XP and Windows 2003
Internet Connection Firewall for NetBackup Clients. Other services may also need
open ports, as vnetd does.
Troubleshooting a Network Connection Timed Out Error

When a firewall is installed between the master or media server and the client,
ports are not the only firewall configuration issue to consider. Consider the
8
following scenario:
• A firewall is installed between the master/media server and the client.

• The required ports are open on the firewall. Name resolution is working in both
directions.
• ping is working in both directions with both IP addresses and host names.
• The REQUIRED_INTERFACE parameter is set in the bp.conf file.
• The firewall has been configured to allow the ICMP ping messages. ping
works between the server and the client.
However, even though the ports required by NetBackup are enabled at the firewall,
they may not be configured for the TCP used by NetBackup, which requires a
physical and logical connection to the corresponding peer.
Work with the firewall administrator to ensure that the firewall ports required by
NetBackup operations are opened for TCP.
If you receive a network connection timed out message in the middle
of a backup or restore operation, you may need to change the TCP idle timeout
setting on the firewall. Work with the firewall administrator to ensure that the
timeout value is set high enough for NetBackup operations.
NAT or PAT Through a Firewall Configuration

Symantec does not support a configuration that involves a NetBackup server or
client separated by a firewall performing dynamic network address translation
(NAT) or port address translation (PAT). There is support for static NAT as of
NBU 6.0 MP4. Refer to TechNote 286035, Explanation of the new FIREWALL_IN
option for backups, restores and the Windows Administration Console under
NetBackup 6.0 MP4, for implementation details.

Network Timeouts
• Network timeouts are caused by problems connecting to

a client or maintaining a connection to a client.
– 41: Network connection timed out
– 54: Timed out connecting to a client
• Causes for network timeouts include:
– An installed database extension (Oracle, Lotus Notes, Exchange)
– A client backup that consists of thousands of files (This is more
common with incremental or user backups.)
– A very busy network or client
– A client that is not online
– A client that reboots during the backup
– Network cards that are obsolete or not performing correctly
Network Timeouts
Status codes 41 and 54 are similar errors (network timeouts) caused by problems
connecting to a client or maintaining a connection to a client.
• Status code 41: Network connection timed out
• Status code 54: Timed out connecting to a client
Causes and Solutions for Network Timeouts

Causes and solutions for network timeouts may include:
• An installed database extension
– The NetBackup for Oracle extension relies on the Oracle RMAN utility to
perform database consistency checks and the actual read. Delays while
these tasks are performed can cause a backup to have an error with a status
code of 41 or 54.
– Increase the Client Read Timeout value to compensate for the additional
overhead required to interface with the database processes.
Locate the Client Read Timeout parameter by selecting Client
Properties—>Universal Settings tab.
Backups using database extensions may benefit from reducing the
TcpTimedWaitDelay value or the tcp_time_wait_interval
parameter.
• Client backups consisting of thousands of files
8

Increase the Client Read Timeout value or the Client Connect Timeout value.
Locate the Client Connect Timeout parameter by selecting Master Server
Properties—>Timeouts tab.
• A very busy network or client
– Increase the Client Read Timeout value or the Client Connect Timeout
value.
– Retry the backup during a less busy period.
– For Windows clients, disable the Client Job Tracker. The Job Tracker is
designed to notify users of pending backups, but it also creates overhead
that can worsen a timeout problem on a busy client.
› To disable the Job Tracker:
Press Ctrl+Alt+Delete and select the Task Manager. Click the
Processes tab. Right-click tracker.exe and select End Process.
› To prevent the Job Tracker from restarting on the next login:
Delete NetBackup Client Job Tracker from the following folder:
%ALLUSERSPROFILE%\Start Menu\Programs\Startup.
• A client that is not online
Power on the client.
• A client that reboots during the backup
Ensure that there are no hardware or software errors.
• Network cards that are obsolete or not performing correctly
Ensure that the latest drivers are installed on the network cards.

Other Built-In Utilities
• arp -a
• ifconfig –a
• ipconfig /all
• netstat –an
• netstat –nr
• nslookup
• netsh
• tasklist /svc
• tasklist /m
Other Built-In Utilities

Many other configuration and testing tools available with the base operating
system can assist in network troubleshooting, such as:
arp –a Address resolution protocol IP—>MAC address

query tool
ifconfig –a Host network configuration information (UNIX)
ipconfig /all Network configuration information (Windows)
netstat –an Active/listening port connections
netstat –nr Routing table information
nslookup Name/IP resolution query tool
netsh Network configuration and documentation tool
(Windows)
tasklist /svc Running services (Windows)
tasklist /m Module information for each process (Windows)
8

Lesson Summary
• Key Points
In this lesson, you learned how to detect and correct physical
network errors, NetBackup and network configuration errors, and
other errors, such as port shortages, multiple network interface
issues, and firewall issues.
– NetBackup Release Notes
– NetBackup Port Usage Guide (TechNote 281623)
– TechNote 278569, Flowchart for Error Code 25 Troubleshooting
– TechNote 278427, Flowchart for Error Code 54 Troubleshooting
– Additional TechNotes: 234618, 267977, 286035
Lab 8: Troubleshooting Network

Issues
In this lab, you will troubleshoot network-related problems:
• Run scripts that create or simulate network-related
problems.
involve NetBackup network-related errors.

Troubleshooting Network Issues,” page A-48.
Solution: Troubleshooting Network Issues,” page B-59.

Lesson 9
Troubleshooting Performance Issues
Lesson Introduction
Tools
Database
Issues
Issues

Topic
will be able to:
Topic 1: NetBackup Determine the maximum possible
Performance Overview throughput in an environment.
Topic 2: Isolating Bottlenecks • Use various tools to identify performance

bottlenecks.
• Describe how shared memory data buffers
are used.
Topic 3: Addressing • Correct client performance issues.

Bottlenecks • Configure and tune data buffers.
• Configure optimal network settings.

NetBackup Performance Goals and
Strategies
9
• Achieve all backups within the allotted
window
Performance
• Minimize backup impact on client
goals:
systems
• Reduce time to recover data
• Troubleshoot bottlenecks in the data

Performance stream
strategies: • Tailor the backup method to the data
type and the available resources
Topic 1: NetBackup Performance Overview

The context of this lesson is to examine an existing NetBackup configuration in
order to isolate bottlenecks, and to discuss methods of addressing those
bottlenecks. Ideally, a well-planned infrastructure is already in place, allowing the
best chance for improvement through configuration adjustments, rather than
adding physical resources.
After completing this topic, you will be able to determine maximum possible
throughput in an environment.
NetBackup Performance Goals and Strategies

The incentive to improve NetBackup performance is typically to allow backups to
complete within the allotted time (the backup window), and to minimize the length
of time a backup impacts the operation of individual clients. Additionally, there
may be a concern to restore mission critical client data as fast as possible.
These goals can be pursued on two fronts:
• Improve the speed of individual data streams.
Update drivers, defragment client disks, and tune shared memory buffers.
• Back up only the necessary data, in the most efficient method available.
Use NetBackup features, such as Synthetic backups, disk staging, and
Flashbackup.
By first identifying the cause of unacceptable performance, and then choosing the
most appropriate backup method, you may reduce the required backup window,
use tape and disk space more efficiently, and improve data recovery time.
Lesson 9 Troubleshooting Performance Issues 9–3

Throughput Bottlenecks
Tape
Device
SAN
Media
Server
Network
Client
Processing
Legend:
Data Stream 6
Throughput Bottlenecks
For a given flow of data, the slowest single point in that flow is considered a
bottleneck. The bottleneck may exist because it is simply the slowest device in the
chain, or because it is where multiple data streams converge.
Every data stream in NetBackup has a bottleneck, and addressing one bottleneck
always exposes another. In an ideal configuration, each point in the data flow has
throughput potential roughly equal to the next. A situation to avoid is where a
single low performance operation throttles the throughput of high performance
operations in the chain.
When maximizing performance in an environment, it is important to identify not
only where a bottleneck is, but how close it is to the maximum potential
performance for the environment (the subsequent bottlenecks). This helps
determine whether addressing a particular bottleneck is worthwhile.
The slide shows a sample data path, and how addressing one bottleneck leads to
the next.

Determining Maximum Theoretical
Throughput
Disk Storage
Disk Array Unit
9
200 MB/s
Total Network
Concurrent NetBackup
Data (all 1000BaseT Media Server
clients): Network Links
~400 MB/s 90 MB/s each 66 MHZ Bus: Quad
1000BaseT NIC
HBA 100 MB/s
SAN 1 Gb Fiber link

100 MB/s
NetBackup
Clients
Tape Drives
75 MB/s each
(compressed)
1000BaseT
Network Links
Total 90 MB/s each
Concurrent Network Tape Robot
Data (all
clients):
~300 MB/s
1
2
3
4
5
6
7
Determining Maximum Theoretical Throughput

Before performance can be considered poor, weigh it against the throughput
potential of the environment. This means thinking not only in terms of individual
data stream performance, but the aggregate throughput of all concurrent data
streams, and whether the expected performance of the hardware has been
achieved.
Follow the data path from end to end, and note the maximum throughput (based on
the hardware rating) at each potential bottleneck. Also note points that must share
bandwidth with other data streams, and factor in the maximum number of
concurrent streams that may occur.
If performance is significantly lower than what should be possible (assuming the
maximum number of concurrent streams) with the hardware in place, begin
investigation to find the bottleneck.
In the example in the slide, the maximum theoretical throughput through this
media server is 100 MB/s. The bottleneck is the HBA on the media server. If this
bottleneck were to be addressed, for example, by adding an additional HBA, the
new throughput limit would be 200 MB/s, due to the SAN links to the storage
units.

Throughput Testing Guidelines
Guidelines for testing throughput:

• Determine your maximum practical throughput.
• Establish baseline throughput under favorable
conditions.
• Duplicate an existing policy, but delete the backup
window.
• Ensure that the systems and network are “quiet.”
• Test a large and identical chunk of data.
• Run multiple tests and establish an average.
• Expire media after each test.
• Document everything.
8
Throughput Testing Guidelines

Throughput tests may be performed for several reasons. Generally, the first testing
should be under favorable conditions in order to establish a baseline.
Favorable conditions include a media server running no jobs, and a client that will
be inactive for the duration of the test. Additional factors, such as large numbers of
small files and open file backups, are discussed later in this lesson. The results of
this backup can be considered a best case scenario throughput. Ensure that this
value is consistent with the hardware capabilities.
Also test throughput before and after a significant change is made to the
environment, or when you are actively tweaking performance factors, such as the
data buffer size.
When comparing results, it is essential to establish a set of conditions and adhere
to them throughout testing. A good way to emulate production backups is to copy
an existing policy, but delete the backup window in each of the schedules. This
prevents scheduled backups, but still allows for manual jobs.
The larger the amount of data backed up, the more accurate the results will be.
Back up data that represents most of the production backups, such as whether most
files are large or small, and whether they are highly compressible.
When backing up to tape, expire the images using bpexpdate so that each
subsequent test may position to the beginning of the media.

Measuring Performance:
The All Log Entries Report
Nonmultiplexed backup
9
1155138742 1 4 4 pc1train07 438 438 0 pc1train07 bptm
successfully wrote backup id pc1train07_1155138161, copy
1, fragment 2, 2796542 Kbytes at 6926.923 Kbytes/sec
Multiplexed backup
1155142721 1 2 4 pc1train07 0 0 0 *NULL* bptm successfully

wrote 2 of 2 multiplexed backups, total Kbytes 2692412 at
20541.562 Kbytes/sec
Restore
1155144079 1 2 4 pc1train07 0 0 0 *NULL* bptm successfully

restored 3 of 3 requests, read total of 3645244 Kbytes at
10131.261 Kbytes/sec 9
Measuring Performance: The All Log Entries Report

Performance statistics are commonly collected by NetBackup and displayed either
in the Activity Monitor job details, or in the throughput summaries displayed in an
All Log Entries report. In nearly all cases, the data reported through the All Log
Entries report is most relevant.
This slide shows snippets of the All Log Entries report.
The All Log Entries report displays the throughput based on a calculation of the
time spent reading or writing from tape only. The report excludes the time it took
to mount and position the tape.
In addition, the All Log Entries report displays the total aggregate throughput for a
multiplexed backup operation. This is crucial information when measuring
multiplexed backup performance.
The All Log Entries report is a filtered view of the NetBackup Error log. This log
can be accessed directly through the bperror command, allowing the output to
be sent to a text file. Information in the error log is kept for 28 days by default.

Identifying and Addressing
Performance Problems
Establish a baseline of
expected performance.
Does the current

performance meet the Done
business needs? Yes
No
Run tests to identify

the bottleneck.
Make an adjustment
and measure
performance. 10
Identifying and Addressing Performance Problems

At least two things should be known before seeking a bottleneck for a poorly
performing backup operation: expected throughput and observed throughput.
Expected throughput is approximate and can be established after weighing the
maximum theoretical throughput against baseline performance testing. At that
point, investigate any large discrepancies between the two, or accept the
performance testing as “real world” expected throughput.
This is often an iterative process.
Determine observed throughput by reviewing the job statistics provided by
NetBackup.
Locating a bottleneck is often performed by measuring the throughput of specific
components in the data stream, and by comparing the speed to the original
observed throughput. After the approximate location of the bottleneck is identified
through testing, specific configuration changes can be made to further isolate or
address it.
If any changes are required, repeat the process until the expected performance is
met.
Remember that a discrepancy between a poorly performing operation and the
observed throughput is not a problem inherently. This discrepancy becomes a
problem when a business need is impacted, such as the ability to meet a backup
window or complete a restore operation to meet a time-sensitive Service Level
Agreement (SLA).

Initial Data Collection
To troubleshoot a poorly performing operation, start by
9
collecting the information already available:
• How many clients are affected?
• Do they have anything in common?
– Network segment
– Operating system
– Data type (database, many small files, and so on)
• Which media servers are affected?
• Has performance always been poor?
12
Topic 2: Isolating Bottlenecks

• Use various tools to identify performance bottlenecks.
• Describe how shared memory data buffers are used.
Initial Data Collection

After a poorly performing operation has been identified, collect some preliminary
information. If answering some basic questions about the circumstances of the job
does not immediately resolve the problem, it may at least narrow the focus of the
required testing.
Use the Activity Monitor and related reports to determine basic information that
may shed light on the problem. For example, if a client is seeing poor performance
during backup, check to see if other volumes on that client are also seeing the same
poor performance. If another volume is consistently performing at the expected
speed for the environment then everything beyond the client (from the network
down to the storage unit) is unlikely to be the bottleneck.

Shared Memory Data Buffers: Backup
Client Data Backup Operation to Tape
Shared Memory
Buffers
64K
64K
64K
Producer: 64K
Consumer:
bpbkar (local backup) 64K
bptm (local backup)
bptm child (remote backup) 64K
bptm parent (remote backup)
64K
64K
Tape
Example Values Storage
NUMBER_DATA_BUFFERS = 8 Unit
SIZE_DATA_BUFFERS = 65,536 (64K) 13
Shared Memory Data Buffers: Backup

NetBackup media servers use a shared memory buffer to stage data for reading
from or writing to a storage destination. This basic process occurs on all backup
and restore operations, for both disk and media manager storage units.
The operation involves two acting processes with access to the buffer: a data
producer that writes data to the buffer, and a data consumer that extracts data from
the buffer.
Processes acting as the producer or consumer are determined by two factors:
• Is the operation local or remote?
• Is the operation a backup or a restore?
If the operation is a local backup, meaning that the media server is also the client,
the producer is bpbkar for UNIX and bpbkar32 for Windows and the
consumer is an individual bptm process. Otherwise, if the client is not the media
server, the producer is a bptm child spawned by the bptm parent that is acting as
the consumer.
Certain aspects of this process are configurable at the media server level, including
the size of the individual data buffers and the number of buffers used. The size of
the data buffer is particularly significant because it also determines the block size
NetBackup attempts to use when it is writing to tape.

Shared Memory Data Buffers: Restore
Client Data Restore Operation from Tape
9
Shared Memory
Buffers
64K
64K
64K
Consumer: 64K
Producer:
tar (local restore) 64K
bptm (local restore)
bptm child (remote restore) 64K
bptm parent (remote restore)
64K
64K
Tape
Example Values Storage
NUMBER_DATA_BUFFERS = 8 Unit
SIZE_DATA_BUFFERS = 65,536 (64K) 14
Shared Memory Data Buffers: Restore

During a restore operation the producer / consumer relationship is reversed: the
producer writes data from the storage to the shared data buffer, and the consumer
extracts this data for delivery to the client system.
If the operation is a local restore, meaning that the media server is also the client,
the producer is an individual bptm process, and the consumer is tar for UNIX
and tar32 for Windows. Otherwise, if the client is not the media server, the
producer is a parent bptm process that spawns a child bptm process to act as the
consumer. Data is then transferred over the network to the remote tar or tar32
process.
Restore operations attempt to use the same buffer size value that was used for the
backup.

Data Buffer Waits and Delays:
Consumer
Client Data
Shared Memory
Buffers
64K
64K
64K
Producer: 64K
Consumer:
bptm (local backup)
64K
64K
Consumer
WAIT DELAY
c e
d
c
f Tape
Storage
Unit
1
2
15
Data Buffer Waits and Delays: Consumer

The producing and consuming processes each have a condition that they require
before they can proceed. Data producers require an empty buffer before
proceeding, while a consumer needs a full data buffer. For example, in the case of
eight data buffers, the producer must have filled one or more of the buffers before
the consumer is able to remove any data and write it to the storage unit.
When a consuming process discovers that there are no full data buffers available, it
increments the Wait and Delay counters. After a brief period of time, the consumer
checks the buffer availability again. If a full buffer is still not available, only the
delay is incremented. The delay counter continues to increment until a buffer
becomes available.
The period of time that a process waits before rechecking for a full or empty buffer
condition is specified using the PARENT_DELAY and CHILD_DELAY files in
/usr/openv/netbackup/db/config (UNIX) or
install_path\NetBackup\db\config (Windows). The default value for
the PARENT_DELAY is 15 ms and the default value for the CHILD_DELAY is 10
ms.
You can apply different settings to parent and child processes. Note that the parent
always exists between the data buffer and the storage unit, while the child always
exists between the data buffer and the client. Also note that the default values are
appropriate in most cases, and are rarely tweaked.

Data Buffer Waits and Delays:
Producer
Client Data
9
Shared Memory
Buffers
64K
64K
64K
Producer: 64K
Consumer:
bptm (local backup)
64K
64K
Producer
WAIT DELAY
c e
d
c
f Tape
Storage
Unit
1
2
16
Data Buffer Waits and Delays: Producer

When a producing process discovers that there are no empty data buffers available,
it increments the Wait and Delay counters. After a brief period of time, the
producer checks the buffer availability again. If an empty buffer is still not
available, only the delay is incremented. The delay counter continues to increment
until an empty buffer becomes available.

Determining Waits and Delays
• Producer (bpbkar):
<4> tar_backup::OVPC_EOFSharedMemory: INF - bpbkar
waited 3959 times for empty buffer, delayed 3991
times
• Consumer (bptm):
<2> write_data: waited for full buffer 13530 times,
delayed 30376 times
• For both producer and consumer calculate:
(data transferred)/(block size)=(# of blocks)
(# of waits)*100/(# of blocks)= wait percentage
• Guideline: Values greater than 5% are “high.”
17
Determining Waits and Delays

When you are trying to determine where a bottleneck is for a given operation, start
by reviewing the wait count on each side of the shared buffer. For a backup
operation, the producer wait count is found in the bpbkar log (even if a bptm
child is the actual producer), and the consumer wait count is found in the bptm /
bpdm log.
Next, classify the results as low or high. The classification is relative to the amount
of data transferred. Express the number as a percentage to obtain a measurable
result using the formulas in the slide. For example, if data transferred is 6GB and
the block size is 64K then the total number of data buffers is 98304.
6GB / 64K = 98304
Calculate the wait percentage as follows:
• Producer: 3959 / 98304 = 4% waits
• Consumer: 13530 / 98304 = 14% waits
As a general guideline, values greater than 5% for large operations are considered
high.

Interpreting Backup Waits
Is the bottleneck the storage unit or the data
9
stream?
Storage Unit: Data Stream:

Producer High/ Producer Low/
Consumer Low Consumer High
Both:
Producer Low/Consumer Low
Producer High/Consumer High
18
Interpreting Backup Waits

Classify the data buffer waits and delays for a backup operation as follows:
• Producer low, consumer low
The throughput of the stream from the client and the speed of the tape device
are roughly equal. The tape device should be writing at or near its “real world”
capacity.
• Producer low, consumer high
The data stream is failing to provide data to the tape device fast enough. First
optimize performance at the individual stream level, ensuring that the client
can provide data to the media server as fast as possible. If the condition still
exists, increase the multiplex level until it is resolved.
• Producer high, consumer low
The tape device is not writing as fast as the data is being provided. This is a
normal or even ideal condition in many cases, because the tape device is the
primary bottleneck. In this situation, decrease the multiplexing value and
verify if the tape device is still the bottleneck. If throughput is not as high as
expected, increase the data buffer size (within OEM recommendations), or
update the tape device driver and firmware.
• Producer high, consumer high
The bottleneck shifts between the tape device and the speed of the data stream.
This may indicate a fast but unreliable data stream, resulting from a congested
network or a very active client. Determine why the data stream is inconsistent
and try increasing the number of data buffers. The primary goal is to reduce
waits on the consumer side.

Client-Side Bottlenecks
• To test the client’s ability to process data for backup:

1. Enable bpbkar logging on the client system.
– UNIX: Set VERBOSE=5 in the bp.conf file
– Windows: Set the NetBackup Client Properties Troubleshooting
General Debug Level to 1 and set the Verbose level to 5.
2. Use bpbkar to process data on the client.
3. Determine performance from the bpbkar log.
• Syntax for running bpbkar output to null:
– UNIX:
/usr/openv/netbackup/bin/bpbkar –nocont –dt 0
–nofileinfo –nokeepalives file_system > /dev/null
– Windows:
install_path\NetBackup\bin\bpbkar32
–nocont data_path > NUL
19
Client-Side Bottlenecks
Using bpbkar to process client data is an effective way of isolating a bottleneck
to the client system. The bpbkar process is used to process and package data to
be sent from the client to the media server for backup. When used as documented
in the slide, the client goes through the normal motions for processing the data, but
stops short of sending it to the media server.
The process is complete when the bpbkar (UNIX) or bpbkar32 (Windows)
process exits. Performance statistics can then be collected from the bpbkar log
similar to the following:
TAR - backup: 11124 files
TAR - backup: file data: 1073230716 bytes
TAR - backup: image data: 27099136 bytes 1 gigabytes
TAR - backup: elapsed time: 82 secs 13424889 bps
The most accurate measurement can be taken from the bytes per second (Bps)
report in the bpbkar log.
13424889 Bps = 13110 KBps = 12.8 MBps
If this performance is similar to that of the poorly performing backup, it is likely
that the client is the bottleneck. Increasing performance for this backup stream is
dependent upon improving the client’s ability to process data for backup.
OS utilities, such as perfmon in Windows and vmstat in Solaris, may reveal
the specific client resource that is “choking” on the data.

Additional Bottleneck Testing
9
If you want to test the … Then, …
Connection from the media server Back up to an alternate SSO

to the storage unit, media server.
Network throughput between the FTP data from the client to the
client and media server, media server.
Media manager storage unit, Back up to a disk storage unit.
20
Additional Bottleneck Testing

A poorly performing client or group of clients can be tested in a number of ways to
help isolate the problem. Some common methods are:
• Back up to an alternate SSO media server.
If a backup takes place on a shared tape drive, try performing the backup
through another media server with a data path to that drive. This will isolate the
network path and the media server, while the actual storage destination and
source data remain constant.
• FTP data from client to the media server.
Data may be transferred from the client to the media server in order to test the
network throughput between the two. Test in a period of low network activity,
and again when under the sort of load expected during the backup window.
FTP is a preferred method for this type of test because of the low overhead.
• Back up to a disk storage unit.
A backup to a disk storage unit isolates a tape device as the bottleneck. In cases
where this performance test result is still poor but significantly improved from
the tape device, this is a symptom of a start-stop condition, where the tape
device is not the root bottleneck, but is exaggerating the problem by managing
a slow data stream inefficiently.

Tape Device Shoe-Shining
Client Data
Shared Memory
Buffers
64K
64K
64K
64K
bpbkar 64K
bptm
64K
64K
64K
64K
64K
Tape
64K
Tape Drive 64K
Status 64K
64K
Stop
Stop 64K
Rewind
Rewind 64K
Reposition
Reposition
3
1
2
21
Tape Device Shoe-Shining

Many tape technologies are susceptible to a condition known as shoe-shining.
Shoe-shining is a condition that occurs during a write operation due to an
insufficient data flow. In this case, as a tape device takes time to come to a full
stop, it needs to rewind and reposition itself to the end of the data. If data is not
received at a rate fast enough to keep up with the tape drive, the device must halt
progress and reposition. This can dramatically impact the performance of an
already slow backup job.
Additionally, this condition can obscure the problem when trying to isolate the
actual bottleneck. A high wait and delay count on the part of the data buffer
consumer is a strong indicator that the shoe-shining condition is occurring.
Another indication is if a slow data stream achieves significantly higher
throughput to a disk storage unit on the same media server.
A common method of avoiding this condition is to ensure sufficient data flow to
the tape device by multiplexing data streams. Increasing the number of data
buffers on the media server may also reduce its occurrence. Check with the device
manufacturer for information on minimum throughput requirements to avoid shoe-
shining or stop-start conditions.

Client Performance Factors
• Debug logging
9
• Notify scripts
• Open file handling
• Many small files
• Compression
• Encryption
23
Topic 3: Addressing Bottlenecks

• Correct client performance issues.
• Configure and tune data buffers.
• Configure optimal network settings.
Client Performance Factors

This slide shows factors that may affect client performance:
• Debug logging
NetBackup debug logging can have a significant impact on performance at any
tier: master server, media server, or client. Disable debug logs unless you are
actively troubleshooting an issue. When logs must be enabled, particularly on
the master server, enable only the logs relevant to the problem.
• Notify scripts
Only use NetBackup notify scripts where necessary. NetBackup waits for a
start script to execute fully before beginning an operation, and it waits for an
exit script before completing an operation. This can create a large time delay
when used across many systems. On systems that must run scripts, explore
ways to speed up their execution.
• Open file handling:
– UNIX: Configure busy file processing only for the clients, folders, and
files that require it.

– Windows: The Windows Open File Backup (WOFB) option can potentially
add several minutes of overhead to the start of a Windows client backup.
This time is used to create a Copy On Write (COW) snapshot on the client,
allowing the backup of open files. WOFB is enabled by default, and it
should be disabled on clients where this is unnecessary.
• Many small files
Backing up small files is inherently less efficient during a backup due to the
increased overhead. Exclude temporary and other nonessential files, or change
the backup method to a raw partition or Snapshot Client backup type.
• Compression
Be aware that compression, when enabled at the policy level, refers to a
software-based compression that occurs on the client side, while nearly all
modern tape technologies use hardware compression by default. Software
generally consumes a large amount of processing power, and it will not reduce
the size of the data on tape when hardware compression is already in place.
Only enable software compression if there is a need to compress client data
prior to encrypting it or prior to sending the data over the network.
• Encryption
As with compression, encryption takes place on the client side, and it can
consume a substantial amount of processing power. Enable encryption only for
clients that require it. It may be beneficial to multiplex the encrypted data
streams to tape.

Windows Client Performance Factors
9
• Taxes client resources
Virus scanning: • Consider disabling “outbound” or “on
backup” virus scanning.
• Slows data access

Fragmented data:
• Defragment client data regularly.
• Creates overhead to generate job

NetBackup Job statistics
Tracker:
• Prevent it from starting automatically.
24
Windows Client Performance Factors

There are performance factors that affect Windows clients only:
• Virus scanning
Virus scanning at the time of the backup job can tax client resources and create
a bottleneck. Check the program’s configuration to determine if scanning files
during a backup can be disabled. Often this can be performed by disabling “on
backup” or “outbound file” scanning.
Caution: Only disable these features where the scan on backup is redundant to
any other scanning that takes place.
• Fragmented data
Fragmented data on Windows file systems slows data access just as it does
outside of backup operations. However, fragmented data exacerbates the
problem during backups because of the volume of data that is accessed.
This is also an issue with UNIX, but unless you are using an advanced file
system, such as VxFS (part of Storage Foundation for UNIX), online
defragmentation is not possible.
• NetBackup Job Tracker
The NetBackup Job Tracker is used by users working with the NetBackup
client; it alerts a user when jobs are taking place and reports job progress. On a
file server, a database server, or any other NetBackup client where this feature
is unimportant, disable it and prevent it from starting up automatically.

Maintaining Device Drivers and
Firmware
NetBackup OEM Tape
Clients NetBackup Device Drivers
Media 4.17 - 9/2004
Switch Firmware
Servers 2a - 8/2001
RAID HBA
SAN Switch
Controller 7.94 – 7/2005
Client NIC Client NIC 6.4h – 12/2003
1.73 – 7/1999 5.12 – 7/2005
Media NIC Fiber / SCSI
5.12 – 7/2005 Bridge
HBA
7.94 – 7/2005 Bridge Firmware
3.0.9 - 3/2004
Tape Device
Network Switch Firmware
4.10 – 6/2005
Switch
Firmware Media NIC Tape Device
4.5a - 8/2004 5.12 – 7/2005 Firmware
4.17 – 8/2005
OS Tape Device
Drivers
2.3.7 - 3/2000 Tape Robot
1
2
25
Maintaining Device Drivers and Firmware

Bottlenecks can result from outdated firmware and device drivers in the
environment. Older drivers may contain severe inefficiencies, particularly if they
were not designed for the specific hardware in use.
Take inventory of driver and firmware versions periodically for devices relevant to
backup and restore operations. Read OEM documentation on issues resolved
between your version and the latest. As a best practice, ensure that drivers and
firmware are consistent across common devices.
The slide highlights driver and firmware versions that are old enough to warrant
review, or versions that mismatch their counterparts in the environment; for
example, the two identical tape devices are running different firmware levels.
Note that unless you are troubleshooting a problem, it is generally a best practice
to maintain a stable environment. Whenever it is determined that a device driver or
firmware should be upgraded, always have a plan to back out of the update. If
possible, install a new driver or firmware version on test equipment before
deploying it into production.

Buffer Tuning Checklist
; Check waits and delays to ensure that a sufficient data
9
stream is delivered.
; Check with the tape device and the HBA manufacturer
for the recommended size.
; Carefully document the configuration and the results.
; Make changes in small increments.
; Thoroughly test restores on all required media servers.
26
Buffer Tuning Checklist

Tape device and HBA manufacturers typically have recommendations or
requirements regarding the block size that should be written to tape. Factor in this
information before proceeding with any testing.
Before considering results, consider the wait and delay values in the debug logs to
ensure that the tape device is the bottleneck. Performance tests using an
insufficient data stream give misleading results.
Make changes to the size or number of data buffers in small increments, in order to
identify if a bell curve exists. This is a scenario where increased values begin to
cause a decline in performance.
It is very important to test restores on all media servers that may be needed to
recover data. A block size written by one server may not be readable on another.
With a new block size in place, verify restores against preexisting backups that
used the original block size.

Configuring Data Buffers
SIZE_DATA_BUFFERS:
• The size of an individual shared memory segment

• Determines the I/O transfer size, which becomes the block size on the
tape
• Solicit OEM recommendations
NUMBER_DATA_BUFFERS:
• The number of concurrent shared memory segments used by

NetBackup per data stream
• Configure based upon performance testing and available shared
memory resources
Formula:
NUMBER_DATA_BUFFERS * SIZE_DATA_BUFFERS_BYTES * drives *
MPX_level = total shared memory
Example: 16 * 262144 * 2 * 4 = 32768K (32 MB) 27
Configuring Data Buffers

The SIZE_DATA_BUFFERS value must be defined in bytes, and as a multiple of
32768 (32K). Common settings for this value are 65536 (64K), 131072 (128K),
and 524288 (512K). This parameter also determines the I/O transfer size
NetBackup uses, and subsequently the block size on tape. Larger settings offer
improved performance for some drive types. It is recommended to engage the
hardware OEM for the recommended block size.
The NUMBER_DATA_BUFFERS value defines how many segments or “chunks”
of shared memory (of the size defined by SIZE_DATA_BUFFERS) are used
concurrently by a single data stream. Performance testing and available shared
memory on a system determine the optimal value. Common settings are 16 and 32.
To define values for NUMBER_DATA_BUFFERS and SIZE_DATA_BUFFERS,
create a specific file as described below, and save the desired value as the only text
within the file. Remove any file extensions, such as .txt.
Create the files at /usr/openv/netbackup/db/config (UNIX) or
install_path\NetBackup\db\config (Windows).
The files to be created are:
• SIZE_DATA_BUFFERS
• SIZE_DATA_BUFFERS_DISK (override for disk storage units)
• NUMBER_DATA_BUFFERS
• NUMBER_DATA_BUFFERS_DISK (override for disk storage units)
• NUMBER_DATA_BUFFERS_RESTORE (override for restore operations)

UNIX Block Size Limitations
In a UNIX environment, the maximum data buffer size (and
subsequent block size on tape) is typically limited only by the
9
shared memory kernel parameters.
Recommended minimum shared memory values for the Solaris
/etc/system file:
Kernel Parameters Description Value

shmsys: Maximum SHM size per segment
16777216
shminfo_shmmax (bytes)
shmsys: Minimum SHM size per segment
1
shminfo_shmmin (bytes)
shmsys: Maximum SHM segments per
100
shminfo_shmseg process
shmsys: Maximum number of SHM
230
shminfo_shmmni segments system-wide
28
UNIX Block Size Limitations

The table in the slide shows values sufficient for most NetBackup master and
media server configurations. Although these settings have a direct impact on the
NetBackup shared memory data buffer, they also affect the system as a whole. If
these values are insufficient, NetBackup operations may slow or fail. Where these
settings already exist, the higher of the two values should be used.
These settings apply to Solaris installations, using the /etc/system file. A
system reboot may be required for the new settings to take affect.
The following table contains recommended minimum HP-UX values:
Kernel Description Value

Parameter
shmem Enable or disable shared memory support 1
shmmni Maximum number of SHM segments system-wide 320
shmseg Maximum SHM segments per process 230
shmmax Maximum SHM segment size system-wide *
AIX systems use dynamic shared memory allocation, and they do not require
tuning.

Windows Block Size Limitations
Limiting factors for the max block size in Windows:
• One megabyte ceiling imposed by Windows

• Tape device driver
• SCSI controller or HBA driver
Block size limited tape device driver:
18:43:01.609 [23116.23184] <16> io_open: The tape device

at index -1 has a maximum block size of 65536 bytes, a
buffer size of 131072 cannot be used
29
Windows Block Size Limitations

Windows environments are typically more restrictive on the maximum block size
than UNIX environments. The maximum buffer size is limited to 1 MB (1048576
bytes) by Windows, and it can be limited further by the SCSI controller, HBA, or
tape device driver.
In cases where the desired block size cannot be written, updating to the latest OEM
or Veritas tape drivers may remove the limitation.
If the SCSI controller or HBA is suspected, contact the OEM to determine the
maximum block size (or DMA transfer size) supported by the controller. The OEM
may also have special instructions for modifying the maximum block size.
When the attempted block size is larger than what the environment allows, the
backup job fails, and the bptm log may report the following:
• Block size limited tape device driver:
18:43:01.609 [23116.23184] <16> io_open: The tape
device at index -1 has a maximum block size of 65536
bytes, a buffer size of 131072 cannot be used
• Block size limited to a 64K controller configuration (MaximumSGList):
20:13:10.640 [3916.1916] <16> write_data: cannot write
image to media id QZA898, drive index 1, The parameter
is incorrect.

Additional Windows Block Size
Limitations
• The maximum scatter-gather list setting limits the
9
maximum block size.
HKEY_LOCAL_MACHINE\SYSTEM\
CurrentControlSet\Services\
controller_id\Parameters\Device\
MaximumSGList:REG_DWORD:0x21
• To determine the value based on the desired block size:
(block_size_in_KB / 4) + 1 = MaximumSGList decimal
value
30
Additional Windows Block Size Limitations

Windows hosts use a scatter-gather list to determine the largest possible size for a
single I/O transmission through a controller. Ultimately, the size is defined by the
MaximumSGList setting for the controller (the number of pages allowed),
multiplied by 4K (the page size in Windows). This single instance I/O transfer is
referred to as a Direct Memory Access (DMA) transmission.
Acceptable decimal values are 0 – 255, with 255 representing 1MB. Values above
255 are treated as 17 (64K). The value can be determined from the desired block
size by using the formula in the slide. For example, to allow for a 256K block size:
(256 / 4) + 1 = 64 + 1 = 65
Therefore, the MaximumSGList value = 65 (decimal).
The type of driver used by the device (such as Windows STORport or
miniport), or the device itself, may cap the block size regardless of the
MaximumSGList setting.
controller_ID represents the name of the controller driver, for example,
lpxnds.
Note: Always consult the device manufacturer before modifying this value.
Configuring a value unsupported by the device may result in data loss or
hardware failure.

Multiplexing Considerations
• Efficient multiplexing manages slow data streams from

client systems.
• Implement multiplexing only after data streams have
been tuned.
• Do not configure an unnecessarily high MPX value.
• Thoroughly test backup and restore performance before
deciding on a multiplexing level.
31
Multiplexing Considerations
There are many conditions where individual client data stream performance cannot
be improved to the point of “maxing out” the speed of the tape device. In such
cases, multiplexing is a highly effective way of maintaining peak performance of
the tape devices.
Only raise the multiplexing level to the point of making the tape device the
bottleneck for backup operations. Setting the multiplex value too high can have a
negative impact on restore performance. Measure performance for both backups
and restores before deciding on a multiplexing level.

Configuring Optimal Network Settings
Master Server
Verify the data rate and duplex HOSTS File:
setting at each port. MediaServA
9
Production Network Example: 1000 megabit Full Duplex MediaServB
MediaServC
Client HOSTS
files: Populate HOSTS files for
MediaServA faster name lookup.
MediaServB
MediaServC
Media Server
HOSTS files:
Master
ClientA-bu
ClientB-bu
ClientC-bu
ClientD-bu
Keep backup traffic off

the production network.
Backup Network
32
Configuring Optimal Network Settings

Network hardware vendors highly recommend that their equipment be allowed to
“autonegotiate” the best speed and duplex settings between NICs and switches. In
fact, autonegotiation must be enabled in order to use 1Gbps Ethernet (10000Base-
T) networks. NetBackup performance can suffer greatly in environments where
network link partners autonegotiate to a lowest common denominator (such as
10Mbps, or 100Mbps half-duplex) due to transient link problems, cross-vendor
incompatibilities, or out-of-date equipment. When link partners fail to
autonegotiate, the hardware attempts to autosense the network settings, typically
resulting in sub-optimal performance settings. When troubleshooting 100Mbps
network performance issues, Technical Support may ask you to configure speed
and duplex settings manually, but only do so as a short-term workaround until your
hardware problems can be resolved.
Ensure that the most efficient transfer mechanism supported by the hardware (such
as 100 megabit or 1000 megabit full duplex) is specifically configured for each
host involved with moving NetBackup data. Configure network hardware between
the hosts (switches and routers) for full duplex.
Also consider maintaining HOSTS file entries for NetBackup systems. NetBackup
performs forward and reverse name lookups very frequently during normal
operation, and configuring HOSTS file entries increases performance in many
cases.

At a minimum, configure the NetBackup master and media servers with HOSTS
file entries for each other. It may also be desirable to configure a media server with
entries for the clients it protects, though this involves a potentially large
administrative burden.
Ideally, configure a separate network between media servers and client systems to
segregate backup traffic from production traffic. If this is the case, distinguish a
separate host name for each IP in DNS and in the HOSTS file.

Configuring UNIX Network Buffers
Performance
9
How to Configure It
Guideline
Configure for the same /usr/openv/netbackup/NET_BUFFER_SZ

size as the media server
network buffer (Client). Default = 32K
/usr/openv/netbackup/NET_BUFFER_SZ
Configure for a size
greater than Default = 32K
SIZE_DATA_BUFFERS NET_BUFFER_SZ_REST
(Media server).
Default = 32K
33
Configuring UNIX Network Buffers

For remote backup and restore operations, network buffers exist between the client
and server. For any operation, there are two network buffers for the data flow: one
on the client and one on the media server.
Network buffers in UNIX are configured similarly to the way that the shared
memory data buffer size (SIZE_DATA_BUFFERS) is configured. Specify the size
in bytes, within the configuration file /usr/openv/netbackup/
NET_BUFFER_SZ. This value can be changed for both clients and media servers.
Often, the best performance is achieved when configuring the media server
network buffer to a size greater than the shared memory data buffer. In general,
configure the client for the same value as the media server. These are general
guidelines. Test both backups and restores to reach the ideal configuration.
If a NET_BUFFER_SZ_REST value is not specified for a media server, the value
set in NET_BUFFER_SZ is used for restore operations.

Configuring Windows Network
Buffers
Performance
How to Configure It
Guideline
Configure for the Client PropertiesÆWindows ClientÆClient

same size as the SettingsÆ
media server network Communications Buffer
buffer (Client) Default = 16K
install_path\VERITAS\
Configure for a size NetBackup\NET_BUFFER_SZ
greater than Default = (data_buffer_size * 4) + 1024
SIZE_DATA_BUFFERS
(Media server) NET_BUFFER_SZ_REST
Default = (data_buffer_size * 2) + 1024
34
Configuring Windows Network Buffers

The network buffer size is also configurable on Windows media server and client
systems. Note the following differences specific to Windows:
• The network buffer size for a Windows client is configured through client Host
Properties.
• The default network buffer size on a Windows client is 16 KB.
• The media server network buffer configuration file is created in a different
directory than the shared memory data buffer.

Managing Slow Data Streams
9
• Reduce the need for frequent full
Synthetic backups
backups:
• Reduce client resource usage
• Stages data to disk before moving to

tape
Disk staging:
• Removes the tape device as a
bottleneck during the backup window
Various snapshot solutions aimed at

Snapshot
removing client side bottlenecks, such as
Client:
off-host backups and FlashBackup
35
Managing Slow Data Streams

It is not always possible to improve a data stream sufficiently through traditional
backup methods. This is often the case with very large single sources of data,
which limit the use of multiple data streams. It can also be difficult to improve
backups with many small files, short of upgrading the client hardware.
In such cases, there are a number of options to manage these slow data streams:
• Synthetic backups
NetBackup combines backups already performed for a client into a single
image, reducing the need for frequent full backups.
• Disk staging
Disk staging removes the tape device as a bottleneck, and it removes the
chance of shoe-shining due to slow data streams during the backup window.
Tape is quickly moved from the clients to disk during the backup window, and
then migrated from disk storage to tape during production hours.
• Snapshot Client
The Snapshot Client offers a variety of snapshot-based solutions with the
potential to reduce backup or restore time drastically. For example,
FlashBackup performs a raw partition backup and collects a dump of the file
system metadata. This provides the benefit of raw partition backup speed,
without sacrificing individual file restore ability.

Lesson Summary
• Key Points
In this lesson, you learned how to examine an existing NetBackup
configuration in order to isolate and address bottlenecks. You also
learned how to improve performance through configuration
adjustments, such as by tuning data and network buffers, rather than
through adding physical resources.
– NetBackup Administration (Fundamentals II)
– NetBackup Snapshot Client Quick Start Guide
– NetBackup Snapshot Client Administrator’s Guide
– NetBackup Backup Planning and Performance Tuning Guide (TechNote
2818420)
– Operating System Vendor Web sites
– TechNote 273532, 288300
36

Glossary
A administrator A user that is granted
special privileges to install, configure, and
A type of Enterprise Media Manager
manage the operation of a system,
(EMM) robotic control. This robot type is
network, or application.
supported only by NetBackup Enterprise
Server. AIT Sony Advanced Intelligent Tape, a
type of tape drive or media type.
The StorageTek (STK) system for robotic
control. alternate client backup A backup
performed by one client on behalf of
The highest-level component under STK’s
another client. Requires a snapshot method
ACS library software, which refers to a
(included in Advanced Client).
specific stand-alone robotic library or to
multiple libraries connected with a media alternate-client restore See redirected
pass-through mechanism. restore (different client).
access control list (ACL) Security alternate-path restore See redirected
information associated with files on some restore (different path).
file systems.
alternate-read server In Vault, an
ACS Automated Cartridge System. ACS alternate-read server is a server used to
can refer to any of the following: read a backup image that was originally
written by a different media server. The
active job A job for which NetBackup is
media server specified as analternate-read
currently processing backup or restore
server must have access to the media that
data.
contains the backup image or images.
active node The designated node in a
alternate-target restore See redirected
clustered environment where NetBackup is
restore (different target).
installed and from which NetBackup can
push to other nodes in the cluster. The API robots A set of Enterprise Media
NetBackup daemons are running on an Manager (EMM) robot types where the
active node. robot-vendor software or the operating
system (in the case of RSM robots)
activity logs See debug logs.
manage their own media.
activity monitor A NetBackup
archive A special kind of backup where
administration utility that displays
NetBackup backs up the selected files, and
information about NetBackup jobs and
if the backup is successful, deletes the files
provides limited control over them.
from the local disk. In this manual,
ADAMM database The Backup Exec references to backups also apply to the
database that maintains device and media backup portion of archive operations
information (Advanced Device and Media except where otherwise noted.
Management).
archive bit A file-status bit that the
administration client See remote Microsoft-based operating system sets
administration console.
Glossary-1

when it writes a file, thereby indicating bar code A label attached to the media
that the file has changed. that associates the media to its slot location
in a robot. The alphanumeric bar code is
attributes for a policy Configuration also usually included on the magnetic label
parameters that control the behavior of that is written on the media.
NetBackup during operations involving
this policy. bar code rule A rule that specifies
criteria for assigning attributes to new
autochanger See robotic library. robotic volumes.
autoloader See robotic library. BCV The mirror disk in an EMC
automatic backup A scheduled backup primary-mirror array configuration (see
by the master server. mirror). BCV stands for “Business
Continuance Volume.”
avrd The automatic volume recognition
daemon on UNIX and process on block size The number of bytes in each
Windows. block of data written on the media during a
backup.
B
boot server In Bare Metal Restore, a
back up The act of copying and saving
server that contains the resources to boot a
files and folders to storage media.
client, rebuild the client system, and begin
backup The process of copying and a restore or a discovery.
saving files and directories to storage bp A backup, archive, and restore utility
media. For example, the backup is for users on NetBackup UNIX clients. It
complete. This term can also refer to the has a character-based menu interface that
collection of data that NetBackup saves for can be run from terminals that do not have
a client during a backup or archive. For X Windows capabilities.
example, duplicate the backup.
bpadm An administrator utility that runs
Backup is two words when used as a verb. on NetBackup UNIX servers. It has a
For example, back up the file. character-based menu interface that can be
backup agent A general term for the run from terminals that do not have X
host that manages the backup on behalf of Windows capabilities.
the NetBackup client. This is either bp.conf file A NetBackup configuration
another client, the NetBackup media file on UNIX servers and also on UNIX,
server, or a third-party copy device. Macintosh, and OS/2 clients.
backup, archive, and restore bp.ini file The NetBackup initialization
interface The name of the NetBackup
file for Novell NetWare target clients.
Microsoft Windows- and Java-based user
interfaces for clients. On servers, these bpcd The NetBackup Client service on
interfaces can be started through the Windows and the NetBackup Client
NetBackup Administration Console. daemon on UNIX.
backup window The period of time bprd The NetBackup Request Manager
during which backups can begin. service on Windows and the NetBackup
Request daemon on UNIX.
Glossary-2 Veritas NetBackup 6.x Troubleshooting Techniques

bridge In a SAN network, a bridge compression The process of
connects SCSI devices to Fibre Channel. A compacting data to enable more efficient
third-party copy device can be transmission and storage.
implemented as part of a bridge or as part
of other devices. Note that not all bridges configuration The parameters that
function as third-party copy devices. govern the behavior of an application. This
term can also refer to the manner in which
BusinessCopy One of several snapshot a network or system is laid out or
methods included in Advanced Client. connected (for example, a network
BusinessCopy is for making snapshots of configuration).
client data on HP disk arrays.
consolidated eject In Vault,
consolidated eject is the process of ejecting
C media for more than one Vault session at a
cancel a job To terminate a job and time. A consolidated eject can be
remove it from the job queue. performed for one or more logical vaults at
one time.
carousel See robotic library.
consolidated report In Vault,
catalogs Internal NetBackup and consolidated reporting is the process of
Enterprise Media Manager (EMM) generating reports for more than one Vault
databases. These catalogs contain session at a time. A consolidated report can
information about configuration, media, be performed for one or more logical
devices, status, errors, and the files and vaults at one time. Consolidated reports are
directories in the stored backup images. organized by report title, not by vault.
changer See robotic library. copy-on-write In NetBackup Advanced
Client, one of two types of supported
class See policy.
snapshots (see also mirror). Unlike a
client The system with the files to back mirror, a copy-on-write does not create a
up, archive, or restore. separate copy of the client’s data. It creates
a block-by-block “account” that describes
client configurations In Bare Metal which blocks in the client data have
Restore, snapshots of system-specific changed and which have not, from the
configuration information, including the instant the copy-on-write was activated.
number of disk drives, volume and file This account is used by the backup
system information, number and type of application to create the backup copy.
network adapters, network properties,
drivers, and other pertinent information. A cpio A UNIX command that can be used
client configuration may change over time for copying files to or from a cpio archive
as you update system hardware and on disk or tape.
software.
ctime The time that a UNIX inode was
cluster See Master and media server changed.
cluster.
cumulative incremental backup A
command lines Commands that users backup that is scheduled by the
can execute either from the system prompt administrator on the master server and
or in scripts. backs up files that have changed since the
last successful full backup. All files are
Glossary-3

backed up if no prior backup has been cumulative- and differential-incremental
done. Also see differential incremental backups between full backups). With this
backup. backup method, only transaction logs are
backed up and they are not truncated upon
See also cumulative incremental backup completion of the backup. To perform a
(NetBackup for Lotus Notes Agents) and full restore of a Microsoft Exchange
cumulative incremental backup database, the data needed is contained in,
(NetBackup for Microsoft Exchange at most, two NetBackup images. When this
Agent). backup method is used, the transaction
cumulative incremental backup logs remain intact since the last full
(NetBackup for Lotus Notes Agents) backup.
A cumulative incremental backup will In a Microsoft Exchange data recovery
perform differently depending on the type scenario where it has been determined that
of Lotus database. the transaction logs are all intact, you may
For unlogged databases or local databases: need only to restore the database from the
A cumulative incremental backup will last full backup. During recovery,
automatically back up all unlogged or local Microsoft Exchange will replay all the load
databases identified in the file list that have in the log folder. This will bring the
been modified since the last full backup. Microsoft Exchange database back to the
The last modification date is determined by current date instead of to the time of the
the time the database itself was modified, last full or incremental backup.
not the time/date stamp of the database Differential-incremental and cumulative-
file. incremental backup types will fail if
For logged databases: A cumulative Database Circular Logging is configured
incremental backup will automatically for the Microsoft Exchange. When
back up only those logged databases Circular Logging is disabled, incremental
identified in the file list that have been backups may then be performed. See your
assigned a new DBIID since the last full Exchange Server Administration Guide for
backup. more information on configuring Circular
Logging.
For transaction logs: When the
BACKUP_TRANSACTION_LOGS file
list directive is encountered in the file list, D
a cumulative-incremental backup will daemon A program on a UNIX system
automatically back up all transaction logs that runs in the background and performs
identified as available for backup by the some task (for example, starting other
Domino server. The transaction logs will programs when they are needed). Daemons
not be marked as ready to be recycled upon are generally referred to as services or
successful completion of the backup. processes on Windows server systems.
cumulative incremental backup DAR Direct Access Recovery, in which
(NetBackup for Microsoft Exchange the NDMP host positions the tape to the
Agent) A cumulative incremental exact location of the requested files,
backup backs up all changes to the reading only the data needed for those
database since the last full backup or files. Restore times can be reduced from
differential incremental backup (However, hours to minutes.
it is not standard practice to mix

data mover A Fastrax component that device monitor An Enterprise Media
moves data between an EMC Symmetrix Manager (EMM) administration utility that
system and Fastrax-attached tape drives. provides monitoring and manual control of
Each mover has connectors for one or two EMM storage devices. For example, an
Symmetrix systems, and also has SCSI administrator or computer room operator
channels to which multiple tapes drives can use this utility to manually reset
can be attached. devices or set them to the UP or DOWN
state.
database-agent clients Clients with
additional NetBackup software that is device serialization A device firmware
designed to back up relational databases. feature used by Enterprise Media Manager
(EMM), that allows for easier device
database-extension clients See identification and configuration.
database-agent clients.
DHCP Dynamic host configuration
debug logs Logs that can be optionally protocol. This TCP/IP protocol
enabled for specific NetBackup and automatically assigns temporary IP
Enterprise Media Manager (EMM) addresses to hosts when they connect to the
programs and processes and then used to network.
investigate problems.
differential incremental backup A
destination storage unit In Vault, a backup scheduled by the administrator on
storage unit that contains the resources to the master server that backs up files which
which the copies of the backup images will have changed since the last successful
be written. There may be up to four storage incremental or full backup. All files are
units. If the duplicated backup images are backed up if no prior backup has been
to be vaulted, the destination storage unit done. Also see cumulative incremental
must correspond to the robotic volume backup.
group.
See also differential incremental backup
device delays Delays caused by the (NetBackup for Lotus Notes Agents) and
device that are beyond the control of the differential incremental backup
storage application. An example is the time (NetBackup for Microsoft Exchange
required to position tape under the read and Agent).
write heads.
differential incremental backup
device discovery An exploratory (NetBackup for Lotus Notes Agents)
method used by Enterprise Media Manager A differential incremental backup
(EMM) to determine which peripheral performs differently depending on the type
devices are visible from a given of Lotus database it uses.
host.Visibility depends on physical
attachment, device state, and the host- For unlogged databases or local databases:
based system device-layer configuration. Backs up all unlogged or local databases
identified in the file list that have been
device host A host where a drive or modified since the last full or incremental
robotic control is attached or is defined that backup. The last modification date is
also has Enterprise Media Manager determined by the time the database was
(EMM) installed. last modified, not the time/date stamp of
the database file.
Glossary-5

For logged databases: Backs up only those disk group A configuration of disks to
logged databases identified in the file list create a primary-mirror association, using
that have been assigned a new DBIID since commands unique to the disks’ vendor. See
the last full or incremental backup. mirror and volume group.
For transaction logs: When the disk-image backup A bit-by-bit rather
BACKUP_TRANSACTION_LOGS file than a file system backup of a disk drive on
list directive is encountered in the file list, a Windows platform.
a differential-incremental backup backs up
dissimilar disk restore (DDR) In Bare
all transaction logs identified as available
for backup by the Domino server. The Metal Restore, the capability to restore to a
transaction logs are marked as ready to be machine that has a different disk
recycled upon successful completion of the configuration than the original machine.
backup. dissimilar system restore (DSR) In
differential incremental backup
Bare Metal Restore, the capability to
(NetBackup for Microsoft Exchange restore a machine into completely different
Agent) A differential incremental hardware (Windows only).
backup only backs up changes to the DLT Digital-linear tape or tape drive
database since the last full or differential type.
incremental backup. Only transaction logs
are backed up. After the successful backup Domain Name Service (DNS) A
of the transaction logs, all committed logs program that handles name translation for
are truncated (deleted). See the NetBackup network communications.
for Microsoft Exchange Server Guide for
Domino Server database (NetBackup
more information on performing this type
for Lotus Notes Agents) Domino
of backup.
Server databases are managed by the
directory depth The number of levels Domino Server. Domino Server databases
below the current directory level that the are located at or in another directory in the
NetBackup interfaces show in their Domino data directory, typically
directory and file list displays. \Lotus\Domino\Data (Windows) or
/db/notesdata/ (UNIX). Domino
directory tree The hierarchical structure Server databases may also be linked to this
in which files are organized on a disk. Each Domino data directory using Lotus Linked
directory lists the files and directories that Databases. Domino Server databases can
are directly below it in the tree. On UNIX, be Logged or Unlogged.
the topmost directory is called the root
directory. drive cleaning The use of a special
cleaning tape to clean the heads on a drive.
disaster recovery Recovering data
from backups after a disk crash or other drive status A status indicating if a
catastrophe. drive is available. Drive status can be UP
or DOWN.
discovery boot In Bare Metal Restore,
a non-intrusive boot that collects device duplicate backup A copy of a backup
and disk layout information. image created by a Vault duplication
operation.
disk Magnetic or optical disk storage
media.

duplicate image A copy of a backup mapping methods in Advanced Client
image. determine the list of extents and send the
list to the backup agent.
E external media ID An identifier written
eject To move media out of a robotic on a media cartridge or canister to help the
library. operator identify the volume before
inserting it into a drive or robot. For
encryption Provides additional security labeled media, the external media ID
by encoding backup data on the client. should be the same as the media ID
This capability is available only with the recorded on the media.
NetBackup Encryption option.
Enterprise Media Manager (EMM) F
Domain A domain in which all of the
FastResync Formerly known as Fast
media, device, and storage unit
Mirror Resynchronization or FMR,
information for a master server and all of
FastResync performs quick and efficient
its media servers is stored in a single
resynchronization of disk mirrors.
database. Formerly, the Media and Device
NetBackup Persistent Frozen Image uses
Management (MDM) domain.
this feature to create and maintain a point-
The Enterprise Media Manager domain in-time copy of a production disk volume.
server is the host where the EMM database Fibre Channel A type of high-speed
for a domain is located. network composed of either optical or
Enterprise Media Manager (EMM) copper cable and employing the Fibre
Host Host where EMM is installed (may Channel protocol. NetBackup supports
have devices attached). both arbitrated loop and switched fabric
(switched Fibre Channel) environments.
entry and exit ports See media access
port. file system The term file system has two
different meanings:
EVSN See external media ID.
When referring to a product, such as the
exclude list A list that designates files ufs (Sun Solaris) or VxFS (Veritas) file
or directories to exclude from automatic system, it refers to the management and
backups. allocation schemes on which the entire file
expiration (image) The date and time tree is structured.
when NetBackup stops tracking a backup When referring to a particular component
image. in a file tree, file system means a directory
expiration (volume) The date and time (with any subdirectories and files) that is
when the physical media (tape) is attached to the UNIX file tree by means of
considered to be no longer usable. the mount command. When a file system
is selected as an entry in the NetBackup
extent A contiguous set of disk blocks file list, this definition applies.
allocated for a file and represented by three
values: device identifier, starting block FlashBackup A policy type that
address (offset in the device) and length combines the speed of raw-partition
(number of contiguous blocks). The backups with the ability to restore
individual files.
Glossary-7

flush level Controls how often truncated (deleted) after they are
NetBackup clears its log files on a Novell successfully backed up.
NetWare or Microsoft Windows client
platform. FULL media state If this appears in a
report or listing, it indicates the volume is
fragment A part of a backup or archive FULL and cannot hold more data or be
image. NetBackup can be configured to used for further backups.
divide images into fragments when they
exceed a certain size or span tapes.
G
frequency (backup) How often global attributes NetBackup
NetBackup performs scheduled backups. configuration attributes that affect all
For example, if the frequency is seven days policies.
then backups occur once a week.
global device database A database
frozen image See snapshot. that is the repository for global device
FROZEN media state If a volume is configuration information. This
FROZEN, NetBackup keeps it indefinitely information is used by Enterprise Media
and can restore from it but not use it for Manager (EMM) to automate device
further backups or archives. configuration and is the basis for the
device configuration presented in the
full backup A backup that copies, to a GUIs.
storage unit, all files and directories that
are beneath a specified directory. GNU tar A public domain version of the
UNIX tar program.
See also full backup (NetBackup for
goodies directory A directory
Microsoft Exchange Agent) and full backup
(NetBackup for Lotus Notes Agents). containing programs, scripts, and other
files that are not formally supported.
full backup (NetBackup for Lotus
Notes Agents) A backup type used to GUI Graphical user interface.
automatically back up all the Lotus
databases identified in the file list and/or H
the transaction logs if the
hard link On UNIX, a hard link is a
BACKUP_TRANSACTION_LOGS
pointer to the inode for the data. On a
directive is encountered in the file list. All
Windows server, a hard link is a directory
transaction logs, identified as available for
entry for a file. Every file can be
backup by the Domino server, are marked
considered to have at least one hard link.
as ready to be recycled after they are
On NTFS volumes, each file can have
successfully backed up. The Domino
multiple hard links, and a single file can
server handles the actual recycling of
appear in many directories (or even in the
transactions logs.
same directory with different names).
full backup (NetBackup for Microsoft
heap level A parameter for memory-
Exchange Agent) A schedule type used
heap debugging on a Novell NetWare or
to back up the Microsoft Exchange
Windows NetBackup client.
database and associated transaction logs.
All committed transaction logs are

hierarchical storage management insert To physically place a volume in a
The process of automatically migrating robot without using an add or move option
selected files from a managed file system to update the volume database.
to specified migration levels on secondary
storage, while maintaining transparent install path The directory where
access to those files. NetBackup and Enterprise Media Manager
(EMM) software is installed. The default
host A computer that executes on Windows servers is C:\Program
application programs. Files\VERITAS and on UNIX it is /
usr/openv.
host name The name by which a host
computer is identified by programs and instant recovery A snapshot of a client
other computers in the network. file system or volume that is retained on
the client disk. Client data can be restored
HSM See storage migrator.
directly from this snapshot, even after a
system reboot.
I
image The collection of data that J
NetBackup saves for an individual client jbpSA The Java-based NetBackup
during each backup or archive. The image interface for performing user backups,
contains all the files, directories, and archives, and restores.
catalog information associated with the
backup or archive. jnbSA The Java-based NetBackup
interface for administrators.
import The process of recreating
NetBackup records of images so the job A parcel of work submitted to a
images can be restored. computer. NetBackup jobs are backups,
archives, or restores.
inactive node An inactive node is a
node in which the NetBackup cluster group
does not reside. The NetBackup daemons K
are not running on an inactive node. kernel The nucleus of an operating
include list A list that designates files or system.
directories to add back in from the exclude keyword phrase A textual description
list. of a backup.
incremental backup See cumulative kill a job See cancel a job.
incremental backup and differential
incremental backup.
L
inject To move media into a robotic
label The identifier of a tape or optical
library.
disk volume. A recorded label includes a
inode A UNIX data structure that defines media ID.
the existence of a single file.
A bar code label allows a bar code scanner
inport See media access port. to be used for media tracking.
Glossary-9

labeled volume A volume with a databases. Transaction logging must be
recorded media ID (that is, the volume was enabled in order to initiate the recovery of
labeled by NetBackup or Backup Exec). logged databases using NetBackup for
Lotus Notes.
library See robotic library.
logs Files where a computer or
library sharing Allows different drives application records information about its
in a robot to be connected to different activities.
hosts. Library sharing is supported on
NetBackup Enterprise Server only. ltid The Enterprise Media Manager
(EMM) device daemon on UNIX and the
Also known as robot sharing or remote NetBackup Device Manager service on
robot control. Windows.
link See hard link or symbolic link.
load (noun) The amount of work that is M
being performed by a system or the level of mailslot See media access port.
traffic on a network. For example, network
man pages Online documentation
load affects performance.
provided with UNIX computer systems
(verb) To copy data to internal memory. and applications.
For example, load the installation program.
mapping The process of converting a
(verb) To indicate tape drive initialization file or raw device (in the file system or
performed when new media are being Volume Manager) to absolute physical disk
added. addresses or extents for use by backup
agents on the network. NetBackup
local database (NetBackup for Lotus ServerFree Agent uses the VxMS library to
Notes Agents) Local databases are perform file mapping.
Lotus databases that are not found in the
Domino data directory, cannot be shared, mapping methods A set of routines for
and cannot be logged. For this type of converting logical file addresses to
database, it is necessary to back up the absolute physical disk addresses or extents.
database itself for all backup types (full, NetBackup ServerFree Agent includes
differential-incremental, and cumulative- support for file-mapping and volume-
incremental). The database can be restored mapping methods.
only to the point of the latest database
Master and media server cluster A
backup.
NetBackup master server and the remote
logged database (NetBackup for media servers that it is using for additional
Lotus Notes Agents) A feature of storage. It is possible to configure clusters
Domino R5 Server is the ability to log with NetBackup Enterprise Server only.
transactions against one or more Lotus NetBackup Server supports only a single
databases. If transaction logging is enabled server, the master.
on the server, all logged database
transactions go into a single transaction master server The NetBackup server
log, consisting of one or more files or that provides administration and control
extents. Where archive-style transaction for backups and restores for all clients and
logging is used, the archived log files serve servers in a master and media server
as the incremental backup for the logged

cluster. NetBackup Server supports only a optical media with differing physical
single server, and it is the master. characteristics.
MDS See Media and Device Selection. menu interface A character-based
interface for use on terminals that do not
media Physical magnetic tapes, optical have graphical capabilities.
disks, or magnetic disks where data is
stored. Microsoft Windows (noun) A line of
operating systems developed by Microsoft,
media access port A slot or other Inc.
opening in a robot where you can insert or
remove a tape without having to access the For more information on the Windows
interior of the robot. After inserting a tape, operating systems that NetBackup
you move it to a slot by using an inject supports, refer to the Symantec Support
command. Prior to removing a tape, you Web site at http://
move it to the port by using an eject www.support.veritas.com.
command. The inject and eject commands
are supported through the add and move migration Migration, commonly
screens in the Enterprise Media Manager referred to as failover, occurs when a node
(EMM) administration interface. becomes unavailable, and the cluster
resources are migrated to an available
Media and Device Management (MDM) node.
domain See Enterprise Media Manager
(EMM) Domain. mirror A disk that maintains an exact
copy or duplicate of another disk. A mirror
Media and Device Selection (MDS) disk is often called a secondary, and the
MDS determines the available media and disk that it copies is called the primary. All
device resources for any tape-based writes to the primary disk are also made to
operation in NetBackup. the mirror (or secondary) disk.
media host The NetBackup server to A type of snapshot captured on a mirror
which the job (client) sends the data. disk. At an appropriate time, all further
writes to the primary disk are held back
media ID An identifier that is written on
from the mirror, thus causing the mirror to
a volume as part of the recorded label.
be “split” from the primary. As a result of
Media Manager (Enterprise Media the split, the mirror becomes a snapshot of
Manager, or EMM) Software that is part the primary. The snapshot can then be
of NetBackup and manages the storage backed up.
devices and removable media.
mount To make a volume available for
media server A NetBackup server that reading or writing.
provides storage within a master and media
mount point The point where a file
server cluster. The master can also be a
system on a disk logically connects to a
media server. A media server that is not the
system’s directory structure so the file
master is called a remote media server.
system is available to users and
Remote media servers are supported on
applications.
NetBackup Enterprise Server only.
MPX See multiplexing.
media type An Enterprise Media
Manager (EMM) classification of tape or
Glossary-11

mtime The point in time when a UNIX directly to the NDMP host or to a
or NTFS file is modified. NetBackup media server acting as an
NDMP host (for remote NDMP). These
multiplexing The process of sending drives cannot be used to store data for non-
concurrent-multiple backups from one or NDMP hosts.
more clients to a media manager device
and interleaving those images onto the NetBackup authentication A
media. NetBackup security level. Authentication
verifies NetBackup client-to-server or
multiplexed group A set of backups server-to-server access and also controls
that were multiplexed together in a single access to the services available on that
multiplexing session. host.
multistream (NetBackup for Teradata NetBackup authorization A
Agent) A NetBackup backup or restore
NetBackup security level. Authorization
operation that is split into multiple
verifies if a NetBackup administrative user
component streams based upon Teradata
has permission to use the services available
cluster configuration.
on that host. Authorization provides
additional security over the security
N provided by authentication.
NDMP A network data management NetBackup Client service A
protocol. NDMP is a widely used protocol NetBackup Windows service that runs on
through which an NDMP-compliant clients and servers and listens for
backup application can control the backups connections from NetBackup servers and
and restores for an NDMP host. clients in the network. When a connection
NetBackup requires the NetBackup for is made, this service starts the necessary
NDMP separately priced option to support programs.
NDMP.
NetBackup configuration options On
NDMP client An NDMP-compliant UNIX servers and on UNIX and
backup application that is a client of an Macintosh clients, these settings are made
NDMP server. An NDMP client sends in the bp.conf file. On NetWare target
commands to the NDMP server to control and OS/2 clients, they are in the bp.ini
the backups and restores on an NDMP file. On Windows servers and Windows
host. NetBackup for NDMP is an clients, these settings are called properties
application that allows NetBackup to be an and are made through the Backup,
NDMP client. Archive, and Restore interface or the Host
NDMP host A NAS (Network Attached Properties dialog box in the NetBackup
Storage) system that runs an NDMP server Administration Console.
application to allow backup and restore. In NetBackup Database Manager
a NetBackup configuration, the NDMP service The NetBackup Windows
host is considered a client of NetBackup. service that runs on the master server and
However, NetBackup client software is not manages the NetBackup internal databases
installed on an NDMP host. (called catalogs). This service must be
NDMP storage unit A storage unit that
running on the master server during all
stores the backup data for an NDMP host. NetBackup administrative operations.
The tape drives in this storage unit attach NetBackup databases See catalogs.

NetBackup Device Manager service no rewind on close A device name that
The NetBackup Windows service that runs applies to tape drives attached to (or
on a NetBackup server and starts the controlled by) UNIX devices. This type of
robotic control processes and controls the device remains at its current position on a
reservation and assignment of volumes. close operation.
This service runs only if the server has
devices under Enterprise Media Manager Nodes In a server cluster, two or more
(EMM) control. The process is ltid. servers (called nodes) are linked in a
network, and run cluster software that
NetBackup for NDMP server A allows each node access to the shared SCSI
NetBackup master or media server that has bus to which any number of disks can
NetBackup for NDMP installed on it. connect.
NetBackup Media Server method An nonrobotic See stand-alone.
off-host backup method provided by
NetBackup Advanced Client, in which
data movement is performed by a
O
NetBackup media server. ODL Optical disk library.
NetBackup Operations Manager off-host backup The offloading of

(NOM) An advanced, high-performance backup processing to a separate backup
managing and monitoring application that agent executing on another host.
works in conjunction with an installed base NetBackup Advanced Client provides the
of NetBackup Enterprise Server or following off-host backup options:
NetBackup Server master and media alternate client backup, NetBackup Media
servers, and associated clients. Server, Third-Party Copy Device, and
Network Attached Storage.
NetBackup properties Same as
NetBackup configuration options but are offsite volume group In Vault, an off-
called NetBackup properties on Microsoft site volume group is a volume group that
Windows platforms. indicates media are in offsite storage. After
Vault ejects media, they are moved
NetBackup Request Manager logically from the robotic volume group to
service The NetBackup Windows the offsite volume group.
service that runs on the master server and
starts the scheduler and receives requests offsite volume pool In Vault, an off-
from clients. site volume pool is a volume pool that
Vault searches for backup images to eject
NetBackup Volume Manager service and transfer offsite. Backup images can be
A NetBackup Windows service that runs written to an offsite volume pool by a
on a NetBackup server, allows remote NetBackup backup policy or by Vault.
administration of Enterprise Media
Manager (EMM), and manages volume original backup An original backup is a
information. The process is vmd. backup image created by a NetBackup
backup job. NetBackup can create up to
NFS Network file system. four backup copies during a policy backup;
NIS Network information service. all are considered original backups. (A
backup image created by the Vault
NLM NetWare loadable module. duplication process is a duplicate backup.)
Glossary-13

outport See media access port. NetBackup duplicates an image, the
original is designated as the primary copy.
P primary disk In a primary-mirror array
partitions The logical partitions into configuration, the primary is the disk on
which a magnetic disk is divided. which client data is stored, and which is
directly accessed by client applications. An
patch A program that corrects a problem exact duplicate of the primary disk is the
or adds a feature to an existing release of mirror.
software.
primary node or preferred node The
path length The number of characters in node in a clustered environment that is
a path name. intended to be active the majority of the
time.
pathname The list of directories in the
path to a destination directory or file. privileges The tasks or functions that a
peername The name by which a
user, system, or application is authorized to
computer identifies itself when perform.
establishing connections to other systems. profile A Vault profile is a template for a
vault job; it is a logical construct that
pending actions Special requests for
contains the rules for selecting,
operator assistance to complete a tape
duplicating, and ejecting media. A profile
mount request, when the request causes an
is associated with a specific vault, and at
error.
least one profile must exist for every vault.
persistent frozen image (PFI) See
progress report A log where
instant recovery.
NetBackup records events that occur
policy Defines the backup characteristics during user operations.
for a group of one or more clients that have
protection domain A logical grouping
similar backup requirements.
of Bare Metal Restore servers and clients
port A location used for transferring data that includes one BMR master server, one
in or out of a computer. or more BMR boot servers, and one or
more protected clients.
Also see media access port.
proxy copy An extension to the
prepare to discover In Bare Metal Oracle8i Media Management API that
Restore, an operation that runs on the enables media management software such
master server and prepares all the as NetBackup to perform data transfer
resources necessary to do a discovery boot directly.
of a client.
proxy restore A method to restore files
prepare to restore In Bare Metal to which you have write access on a
Restore, an operation that runs on the machine other than your desktop. The files
master server, which prepares all the must be in a backup image of the machine
resources necessary to restore the client. to which they are being restored.
primary copy The copy of an image that
NetBackup uses to satisfy restores. When

Q redirected restore (different target)
On a Novell NetWare server platform
QIC Quarter-inch-cartridge tape.
running the NetBackup target version of
queued job A job that has been added to client software, this operation restores files
the list of jobs to be performed. to a different target than the one from
which they were backed up.
R registry A Microsoft Windows database
raw partition A single section of a raw that has configuration information about
physical disk device occupying a range of hardware and user accounts.
disk sectors, without a file system or other remote administration console A
hierarchical organization scheme (thus, a Windows NetBackup client that has the
“raw” stream of disk sectors). This is administration interface software installed
different from a block device, over which and can be used to administer NetBackup
the file system is mounted. servers.
raw-partition backup A bit-by-bit remote media server A media server
backup of a partition of a disk drive on that is not the master. Only NetBackup
UNIX. On Windows, this is called a disk- Enterprise Server supports remote media
image backup. servers; NetBackup Server supports only a
rbak The program that Apollo clients use single server, the master.
to read data from tape during a restore. remote NDMP backup/restore A form
recorded media ID This is an identifier of three-way backup/restore, in which data
written as part of the label on a volume and travels from an NDMP host on the network
used by Enterprise Media Manager (EMM) to a storage device attached to a
to ensure that the correct volume is NetBackup media server. The storage
mounted. The recorded media ID should device on the NetBackup media server is
match the external media ID. used for NDMP data only. Data movement
is controlled by an NDMP “mover agent”
Recovery Manager (RMAN) Oracle's (a daemon/service) running on a supported
backup and recovery program. RMAN NetBackup media server.
performs backup and restore by making
requests to a NetBackup shared library. remove The act of taking a volume out
of a robot without using a move or eject
redirected restore (different client) command.
Restores files to your client when they
were originally backed up from a different residence Attributes in the Enterprise
client. The administrator using the Media Manager (EMM) volume database
interface on the master server can direct a including information to show the robotic
restore to any client (this variation is called location and including the robot host, robot
a server-directed restore). type, robot number, and slot location.
redirected restore (different path) resource A Novell NetWare term that

Restores files to a different directory than refers to a data set on the target. For
the one from which they were backed up. example, in DOS, resources are drives,
directories, and files. Also see target
service.
Glossary-15

restore (verb) The act of restoring mounting and dismounting of media in
selected files and directories from a tape or optical disk drives. A robot may
previous backup or archive and returning also be called a robotic library, media
them to their original directory locations changer, automated library, jukebox, or
(or to a different directory). tape stacker.
(noun) The process of restoring selected robotic volume group In Vault, a
files and directories from a previous volume group that indicates volumes
backup and returning them to their original reside in a robot. After Vault ejects a
directory locations (or to a different volume, it is moved logically from the
directory). robotic volume group to the offsite volume
group.
retention level An index number that
corresponds to a user-defined retention root The highest level directory in a
period. There are 10 levels from which to hierarchical directory structure. In MS-
choose (0 though 9) and the retention DOS, the root directory on a drive is
period associated with each is designated by a backslash (for example,
configurable. Also see retention period. the root on drive C is C:\). On UNIX, the
root directory is designated by a slash (/).
retention period The length of time that
NetBackup keeps backup and archive Also, a UNIX user name having
images. The retention period is specified administration capability.
on the schedule.
RS-232 An industry-standard interface
robot control host The host that is for serial communications and sometimes
providing the robotic control for a robot. A used for communicating with storage
robot control host that is separate from the peripherals.
host on which NetBackup is installed
applies only to NetBackup Enterprise RVSN See recorded media ID.
Server.
robot number The unique, logical
S
identification number of a robot. SAN (Storage Area Network) A Fibre
Channel-based network connecting servers
robot type The Enterprise Media and storage devices. The storage devices
Manager (EMM) classification of robots are not attached to servers but to the
according to one of the following: the network itself, and are visible to all servers
physical characteristics of the robot, the on the network.
media type commonly used by that class of
robots, or the communication methods SAN media server A server that can
used by the underlying robotics. only back up its own data to devices. The
backing up of data residing on other clients
robotic arm The component of a robotic on a network is not allowed.
library that physically selects the media
(tape or optical disk). SCSI Small computer system interface.
This is a type of parallel interface that is
robotic control path The control path frequently used for communicating with
to a robot through a SCSI connection. storage peripherals.
robotic library A peripheral device that
contains a mechanism for the automated

SCSI reserve/release A device-level session usually consists of a single backup
protection that is used by Enterprise Media or archive.
Manager (EMM) to avoid unintended
sharing of tape devices and possible data Session (Vault) A vault session that
loss problems. consists of executing a profile or profiles.
schedules These control when backups ShadowImage One of many snapshot

can occur in addition to other aspects of the methods included in Advanced Client.
backup, such as the type of backup (full, ShadowImage is used for making
incremental) and how long NetBackup snapshots of client data on Hitachi disk
retains the image. arrays.
secondary disk See mirror. shared drive A tape drive that is shared
among hosts when the Shared Storage
server independent restore Restoring Option (SSO) is installed. SSO applies
files by using a NetBackup server other only to NetBackup Enterprise Server;
than the one that was used to write the therefore, a shared drive applies only to
backup. Because NetBackup Server NetBackup Enterprise Server. See Shared
installs on one system only, this feature is Storage Option (SSO).
available only with NetBackup Enterprise
Server. shared resource tree (SRT) In Bare
Metal Restore, a compilation source of
server list The list of servers that a baseline system resources, including the
NetBackup client or server refers to when means to rebuild the client system and
establishing or verifying connections to restore all system files.
NetBackup servers. On a Windows server
and Microsoft Windows clients, you Shared Storage Option (SSO) A
update the list through a dialog box in the separately priced Veritas software option
interface. On a UNIX server and UNIX that allows tape drives (stand-alone or in a
and Macintosh clients, the list is in the robotic library) to be shared dynamically
bp.conf file. On NetWare target and OS/ among multiple NetBackup and Storage
2 clients, the list is in the bp.ini file. Migrator servers.
server-directed restore Using the user This option is supported only on

interface on the master server to restore NetBackup Enterprise Server.
files to any client. Only the administrator single-stream (NetBackup for
can perform this operation. Teradata Agent) A NetBackup backup
service A program on a Windows server or restore that is performed over one
system that runs in the background and stream.
performs some task (for example, starting SMDR Storage management data
other programs when they are needed). requestor, a Novell NetWare program that
Services are generally referred to as provides its services transparently to all
daemons on UNIX systems. SMS modules and allows remote and local
session An instance of NetBackup modules to communicate with one another.
checking its schedules for backups that are SMS Novell NetWare storage
due, adding them to its worklist, and management services.
attempting to complete all jobs in the
worklist. For user backups and archives, a
Glossary-17

snapshot A stable disk copy of the data stand-alone A qualifier used with drives
prior to backup. A snapshot is created very and media to indicate they are not
rapidly, causing minimal impact on other associated with a robot. For example, a
applications. There are two basic types: stand-alone tape drive is one for which you
copy-on-write and mirror. must manually find and insert tapes before
using them. A stand-alone volume is one
snapshot method A set of routines for that is located in a stand-alone drive or is
creating a snapshot. stored outside of a drive and designated as
snapshot mirror An exact copy of a stand-alone in the volume configuration.
primary volume at a particular moment, standard device Refers to the primary
reproduced on a physically separate disk in an EMC primary-mirror disk array
device. Snapshot mirrors are created by the (see primary disk).
Volume Manager (VxVM).
status code A numerical code, usually
snapshot source (Formerly known as accompanied by a troubleshooting
frozen image source.) This designates the message, that indicates the outcome of an
entity (file system, raw partition, or logical operation.
volume) to which a snapshot method is
applied. NetBackup automatically selects storage migrator The Veritas Storage
the snapshot source based on the entries in Migrator line of hierarchical storage
the policy’s Backup Selections list. management products for UNIX and
Windows. These products make extra
source volume group In Vault, a room on a disk by transparently moving
volume group from which Vault selects data to other storage and then transparently
backup images to duplicate. Selecting a retrieving the data when it is needed by a
source volume group restricts the search user or application.
for images to those in that volume group
rather than images in all volume groups. storage unit A storage device where
Usually, a source volume group is NetBackup or Storage Migrator stores
specified if your master server has access files. It can be a set of drives in a robot or
to multiple robots and you want to consist of one or more single tape drives
duplicate images that reside on media in that connect to the same host.
one robot to media in another robot. The
images that are read are in the source SUSPENDED media state If a volume
volume group in one robot; the images is SUSPENDED, NetBackup can restore
written to media are in the robotic volume from it but cannot use it for backups.
group in another robot. NetBackup retains a record of the media
ID until the last backup image on the
If you do not perform duplication in Vault, volume expires.
you do not have to specify a source volume
group. SYMAPI The Symmetrix Application
Programming Interface from EMC that
SSO See Shared Storage Option (SSO). allows NetBackup to communicate with
Symmetrix disk arrays, Fastrax, and the
stacker Usually, a small robotic library Fastrax-attached tape drives.
that contains one drive only. See robotic
library. symbolic link On a UNIX system, a
pointer to the name of the file that has the
source data.

SYMCLI The Symmetrix Command Line need to perform cleaning operations.
Interface from EMC that allows TapeAlert is a function of the tape drive.
NetBackup to communicate with
Symmetrix disk arrays, Fastrax, and the tar The tape archive program that
Fastrax-attached tape drives. NetBackup uses to extract backup images
during a restore.
Symmetrix A line of EMC disk storage
systems (disk arrays). In a NetBackup for target See target service.
EMC Fastrax configuration, the data to be target service A Novell NetWare
backed up resides on a Symmetrix storage service that needs storage management.
system. For a backup, the data is moved The SMS views all services (for example,
from the Symmetrix storage through a print services, communication services,
Fastrax unit to a Fastrax-attached tape workstations) as targets.
drive.
Target Service Agent A Novell
synthetic backup A synthetic full NetWare agent that prepares the target's
backup is a backup assembled from a data for SMS during a backup and for the
previous, traditional (non-synthesized) full target during a restore.
backup, and subsequent differential
backups and/or a cumulative incremental Third-Party Copy Device A backup
backup. A client can then use the agent on the SAN that operates on behalf
synthesized backup to restore files and of backup applications. The third-party
directories in the same way that a client copy device receives backup data from a
restores from a traditional backup. disk attached to Fibre Channel and sends it
to a storage device, using the SCSI
system administrator A person with Extended Copy command. The third-party
typical UNIX or Windows administrator copy device is sometimes called a copy
privileges and responsibilities. manager, third-party copy engine, or data
mover. In SAN hardware configurations, a
T third-party copy device can be
implemented as part of a bridge, router, or
tape format The format that an
storage device. The third-party copy device
application uses to write data on a tape.
may or may not be the device to which the
tape mark A mark that is recorded storage units are connected.
between backup images on a tape.
An off-host backup method in NetBackup
tape overhead The space required for ServerFree Agent that allows backups to
data that is not part of the backup images. be made by means of a backup agent on the
For example, tape marks and catalogs of SAN.
what are on the tape are considered
three-way backup/restore Data travels
overhead.
between an NDMP host and a storage
tape spanning Using more than one device that is attached to another NDMP
tape to store a single backup image. host (or to a NetBackup media server) on
the network. This contrasts with local
TapeAlert A tape drive status monitoring NDMP backup/restore, where the data
and messaging utility. The TapeAlert travels between an NDMP host’s disk and
utility can detect tape quality problems, a storage device directly attached to the
defects in tape drive hardware, and the same NDMP host.
Glossary-19

TimeFinder One of many snapshot transfer rate The rate at which
methods included in Advanced Client. computer information is transferred
TimeFinder is for making snapshots of between a source and a destination.
client data on EMC disk arrays.
transport See robotic arm.
TIR See true image restore.
true image restore Restores the
TLD - Tape Library DLT An Enterprise contents of a directory to what it was at the
Media Manager (EMM) designation for a time of any scheduled full or incremental
category of robot. For the specific vendor backup. Previously deleted files are
types and models in this category, see the ignored.
Symantec Support Web site.
TSA See Target Service Agent.
TLH - Tape Library Half-inch An
Enterprise Media Manager (EMM) TSH - Tape Stacker Half-inch An
designation for a category of robot. For the Enterprise Media Manager (EMM)
specific vendor types and models in this designation for a category of robot. For the
category, see the Symantec Support Web specific vendor types and models in this
site. category, see the Symantec Support Web
site.
This robot type is supported only by
NetBackup Enterprise Server.
U
TLM - Tape Library Multimedia A ufs file system The UNIX File System
Enterprise Media Manager (EMM) (ufs), which is the default file system type
designation for a category of robot. For the on Sun Solaris. The ufs file system was
specific vendor types and models in this formerly the Berkeley Fast File System.
category, see the Veritas support web site.
unassigned media Media that contain
This robot type is supported only by no valid images. A piece of unassigned
NetBackup Enterprise Server. media has an entry in the volumes database
but no entries in the images database.
TL4 - Tape Library 4MM An Enterprise
Unassigned Media do not have a “time
Media Manager (EMM) designation for a
assigned” in the Media section of the GUI.
category of robot. For the specific vendor
types and models in this category, see the unlabeled volume A volume that does
Symantec Support Web site. not have recorded media IDs.
timeout period The period of time that unlogged database (NetBackup for
an application has allotted for an event to Lotus Notes Agents) A database in
occur. which transaction logging is not enabled,
or has been disabled for specific server
tpconfig An Enterprise Media Manager
databases.
(EMM) administration utility for
configuring devices that is started from the Unlogged Domino Server databases are
command line. On UNIX, it has a backed up when a full backup, a
character-based menu interface that can be differential-incremental backup, or a
run from terminals that do not have X cumulative-incremental backup is
Windows capabilities. tpconfig also has performed. The database can be restored
a command-line interface.

only to the point of the latest database virtual host name The name of the
backup. cluster as known by the NetBackup
environment. This is the name that the
clients have in their server list. It is also the
V name that the server uses when
Vault A separately priced NetBackup communicating to the client nodes.
option that provides offsite backup
management. Vault automates the process virtual IP address The IP address of the
of offsite media rotation (a critical cluster, and the address to which the virtual
component of any backup or disaster server name should resolve.
recovery strategy). Vault manages offsite vmadm An Enterprise Media Manager
storage and retrieval of media for original (EMM) administrator utility for managing
backups, duplicate backups, and catalog volumes. It runs on UNIX and has a
backups. Additionally, NetBackup Vault character-based menu interface that can be
generates reports to track the location and run from terminals.
content of each piece of media.
vm.conf An Enterprise Media Manager
vault In the context of the NetBackup (EMM) configuration file with entries that
Vault, a logical entity associated with a include the servers that can manage local
particular robot that acts as a designated devices and default media ID prefixes for
holding place for backups that will media that do not contain bar codes.
eventually be sent to a physical offsite
vault. The term “vault” is used to refer vmd The Enterprise Media Manager
both to the process, and to the physical (EMM) volume daemon on UNIX and the
storage location of an offsite set of tapes. NetBackup Volume Manager service on
Windows.
vault process Vaulting is the process of
choosing backup images to duplicate or volume A virtual device configured over
eject, optionally duplicating backups, raw physical disk devices. Consists of a
ejecting duplicate or original media, block and character device.
storing it at an offsite location, and later
returning expired media to your robot. Enterprise Media Manager (EMM)
Vaulting is an integral part of the disaster volumes are logical units of data storage or
recovery process. cleaning capability on media that have
been assigned media IDs and other
verbose flag A configuration file entry attributes, which are recorded in the
that causes a higher level of detail to be Enterprise Media Manager (EMM) volume
written in the logs. database.
verify An operation that compares the list volume configuration Configuration
of files that are actually on a volume with information that is stored in the Media
what NetBackup has recorded as being on Manager EMM database.
it. The data that is on the media is not
verified. volume database The repository for all
Enterprise Media Manager (EMM) volume
virtual circuit (VC) The connection, configuration information about the media
between an EMC Symmetrix array and a in storage devices.
tape drive, through which the backup
image flows during a Fastrax backup.
Glossary-21

volume database host An Enterprise X
Media Manager (EMM) host where this
xbp The X Windows-based backup,
database is stored.
archive, and restore program for users on
volume group A logical grouping that NetBackup UNIX clients. (xbp was retired
identifies a set of volumes that reside at the beginning with NetBackup 6.0.)
same physical location.
volume pool A logical grouping that
identifies a set of volumes by usage.
VxMS (Veritas Federated Mapping
Services) A library of routines
(methods) used by NetBackup ServerFree
Agent to obtain the physical addresses of
logical disk objects such as files and
volumes.
W
wakeup interval The time interval at
which NetBackup checks for backups that
are due.
wildcard characters Characters that
can be used to represent other characters in
searches.
Windows (adjective) Used to describe a
specific product or clarify a term, for
example Windows 2000, Windows .NET,
Windows servers, Windows clients, or
Windows GUI.
Windows (noun) See Microsoft
Windows.
Windows Display Console A
NetBackup-Java interface program that
runs on Windows platforms that are
supported by Symantec. Users can start
this interface on their local system, connect
to a UNIX system that has the NetBackup-
Java software installed, and then perform
any user operations that their permissions
allow.
WORM media Write-once, read-many
media. Can be tape or optical disks.

Index
A bprecover 5-35
bpretlevel 7-6
ALLOW_MEDIA_OVERWRITE 7-18
bpsetconfig 8-17
avrd 3-15
bpstulist 7-5
bptestbpcd 8-27, 8-29
B bptm 3-15, 3-16, 3-23, 3-27, 3-28, 3-29
backup and archive program. See bpbkar. bptm log 6-39
backup and restore manager. See bpbrm. bpup 5-5
backup operation process flow 5-23–5-26
backup process flow 3-22–3-25 C
bottlenecks
client side 9-16 catalog backup
throughput 9-4 cold, offline 5-41–5-43
hot, online 5-30
bpbkar 3-16, 3-20, 3-23, 3-24 manual 5-50
bpbrm 3-15, 3-16, 3-23, 3-24, 3-25, 3-26, 3-27 catalog consistency 5-55
bpcd 3-20 catalog recovery
bpclient 1-5, 8-41 from cold, offline catalog backup 5-44
bpclntcmd 8-14 manual 5-51
bpconfig 1-5 NBDB is corrupt 5-47
without DR file
bpdbjobs 1-6
imageDB intact 5-34
bpdbm 1-6, 3-16, 3-21, 3-22 imageDB not intact in NBU 6.0 5-37
bpdbm -consistency 5-55 imageDB not intact in NBU 6.5 5-39
bpdm 3-23, 3-27 commands
bpdown 5-5 bpclient 1-5, 8-41
bpclntcmd 8-14
bperror 2-12, 7-12, 7-15, 9-7
bpconfig 1-5
bpgetconfig 8-35 bpdbjobs 1-6
bpimage 1-6 bpdbm 1-6
bpimport 5-38 bpdbm -consistency 5-55
bpinetd 3-20 bpdown 5-5
bperror 2-12, 7-12, 7-15, 9-7
bpjobd 3-22, 3-23, 3-25, 3-26, 3-29 bpgetconfig 8-35
bpjobs 7-14 bpimage 1-6
bplabel 7-7, 7-10, 7-18 bpimport 5-38
bpmedialist 7-5 bpjobs 7-14
bplabel 7-7, 7-10, 7-18
bpmedialist 7-10
bpmedialist 7-5, 7-10
bpplinfo 1-18 bpplinfo 1-18
bpps 2-11, 5-16, 8-37 bpps 2-11, 5-16, 8-37
bprd 3-26, 3-29 bprecover 5-35
bpretlevel 7-6
bprd utility 3-13
bpsetconfig 8-17
Index-1
bpstulist 7-5 Device Allocator 5-21
bptestbpcd 8-27, 8-29 Device Allocator. See DA.
bpup 5-5
device manager daemon. See ltid.
dmesg 6-7
ifconfig 8-6 dmesg 6-7
ipconfig 8-6 DNS 1-9, 8-14, 8-16, 8-17, 8-22
kill 8-21 DRIVE_ERROR_THRESHOLD 7-19
mt 6-12
nbdb_admin -auto_start 5-5
nbdb_move 5-54 E
nbemmcmd 1-20, 1-25, 5-11, 6-31, 6-35, 7-
4, 7-9, 7-19, 7-20, 7-24 e-mail notification
nbpushdata 1-9, 1-11, 1-12, 1-19 detecting problems using 2-17
nbrbutil 5-29 global 2-17
ndd 8-33 EMM database
netbackup start 5-5 populating 1-19
netbackup stop 5-5 upgrade issues 1-26
netstat 8-7, 8-21, 8-26 EMM domain 5-10
nslookup 8-23
EMM server 5-13
nt_ttu.exe 6-12
pbxcfg 8-38 emmlib 5-20
ping 8-7 external volume serial number 7-21
robtest 6-13, 6-23, 7-21, 7-22
route print 8-7
scan 6-24 F
sgscan 6-24
files
tctl 6-11
bp.conf 1-8, 1-26, 4-6, 6-30, 7-18, 7-20,
telnet 8-7, 8-24
8-11, 8-12, 8-17, 8-18
tpautoconf 1-8, 1-25, 6-19, 6-23, 7-4
CHILD_DELAY 9-12
tpconfig 1-8, 6-22, 7-10
device mapping 6-33
tpext 6-33
errors 7-16
tpreq 6-17
external_types.txt 6-33
tpunmount 6-18
history.log 8-10
traceroute 8-8
hosts 8-5, 8-7, 8-12
tracert 8-8
inetd.conf 8-21
vmadd 7-10
nbsvcmon.conf 3-17
vmdareq 6-25
NET_BUFFER_SZ 9-31
vmglob 6-25, 6-27
nsswitch.conf 8-23
vmoprcmd 6-21
PARENT_DELAY 9-12
vmphyinv 7-21
password 8-23
vmpool 7-10
Readme.txt 6-33
vmquery 7-24
resolv.conf 8-22
vxlogcfg 4-12, 4-14, 4-23, 4-32
services 8-19
vxlogmgr 4-21, 4-23
st.conf 7-13
vxlogview 4-30, 4-31, 4-33, 5-15, 5-17,
system 9-25
5-19, 5-22
version 8-10
version.txt 8-10
D vm.conf 7-20
firewall ports 8-39
DA 3-14
Index-2 Veritas NetBackup 6.x Troubleshooting Techniques

FQDN. See fully qualified domain name. M
fully qualified domain name 1-9
maintenance packs 1-10
MDS 3-14, 5-18
G media
frozen 7-7, 7-17
Global Device Database 1-8
suspended 7-23
global e-mail notification 2-17
Media and Device Selection. See MDS.
MEDIA_ERROR_THRESHOLD 7-19
H mirrored transaction log 5-52
mt 6-12
hardware compatibility list 6-4
HCL. See hardware compatibility list.
N
I naming convention
unified logs 4-10
ifconfig 8-6
NBCC. See NetBackup Consistency Checker.
inetd 3-20
NBDB. See NetBackup relational database.
ipconfig 8-6
nbdb_admin -auto_start 5-5
IRM 3-10
nbdb_move 5-54
work list 3-11, 3-12
nbemm 5-14
nbemmcmd 1-20, 1-25, 5-11, 6-31, 6-35, 7-4,
J 7-9, 7-19, 7-20, 7-24
Job Manager. See nbjm. nbjm 3-10, 3-15, 3-21, 3-22, 3-23, 3-25, 3-27,
3-28, 3-29
nbpem 3-10, 3-21, 3-25
K nbpemreq utility 3-13
kill 8-21 nbproxy 3-21
nbpushdata 1-9, 1-11, 1-12, 1-19
nbrb 3-10, 3-21, 3-22, 3-25, 3-27, 3-29, 5-16
L nbrbutil 5-29
legacy logs nbsupport 2-9–2-10
description 4-3 nbsvcmon 3-17
disabling 4-23
ndd 8-33
enabling 4-5–4-6, 4-8
extracting data 4-27–4-29 NetBackup Communications Daemon. See
layout 4-26 bpcd.
location of logs 4-4 NetBackup Consistency Checker 1-7
Media Manager 4-7 NetBackup Network Daemon. See vnetd.
Netbackup 4-5 NetBackup relational database 5-3
viewing 4-25 configuration files 5-4
legacy processes 3-4 creation process 5-9
loopback address 8-7 database files 5-3
ltid 3-15 transaction log 5-4
NetBackup Service Monitor. See nbsvcmon.
Veritas NetBackup 6.x Troubleshooting Techniques Index-3

netbackup start 5-5 Policy Execution Manager. See nbpem.
netbackup stop 5-5 port shortages 8-25
NetBIOS 8-17 ports
netstat 8-7, 8-21 13724 3-19, 3-20, 8-39, 8-40, 8-43
13782 3-20
NIC 8-5, 8-16
1556 3-18, 5-13, 8-37, 8-39, 8-40
NIS 8-14, 8-23 1557 3-18
NOM 2-16 2821 5-13
notify scripts 2-18 4032 5-13
nslookup 8-23 processes
nt_ttu.exe 6-12 client 3-8
EMM server 3-6, 3-9
master server 3-5
P media server 3-7
parameters
ALLOW_MEDIA_OVERWRITE 7-18 R
CLIENT_NAME 8-11, 8-17
CONNECT_OPTIONS 8-29–8-31 recorded volume serial number 7-21
DRIVE_ERROR_THRESHOLD 7-19 Registry keys
for bptestbpcd 8-27 HKEY_LOCAL_MACHINE\Soft-
for mail_dr_info script 5-57 ware\VERITAS\NetBackup\Cur-
for nbrbutil 5-27 rentVersion 8-10
for NICs, modifying 8-6–8-7 HKEY_LOCAL_MACHINE\SOFT-
for robotic control configuration 6-28 WARE\VERITAS\NetBackup\Cur-
keep_hours 1-6 rentVersion\Config 8-18
kernel 9-25 HKEY_LOCAL_MACHINE\Sys-
MEDIA_ERROR_THRESHOLD 7-19 tem\CurrentControlSet\Ser-
MONITOR_INTERVAL 3-17 vices\controller_id\Paramete
MONITOR_ON 3-17 rs\Device 9-27
NUMBER_DATA_BUFFERS 9-24 HKEY_LOCAL_MACHINE\SYS-
NUMBER_DATA_BUFFERS_DISK 9-24 TEM\CurrentControlSet\Ser-
NUMBER_DATA_BUFFERS_RESTORE 9- vices\Tcpip\Parameters 8-32
24 HKLMSOFTWAREVeritas etBackupCu
reporting 2-15 rrentVersion ConfigServer 6-
REQUIRED_INTERFACE 8-17, 8-18 30
RESTART_LIMIT 3-17 Registry values
SIZE_DATA_BUFFERS 9-24 MaximumSGList 9-27
SIZE_DATA_BUFFERS_DISK 9-24 Server 8-18
tcp_time_wait_interval 8-33, 8-47 TcpTimedWaitDelay 8-32, 8-47
TIME_WINDOW 7-19 release updates 1-10
PBX 3-4, 3-18, 8-37 reports 2-14
pbx_exchange 3-18 All Log Entries 7-17, 9-7
pbxcfg 8-38 Media List 7-17
performance Media Logs 7-12, 7-16, 7-17
bottlenecks 9-4 Problems 7-11, 7-12, 7-15, 7-16
goals 9-3 resource allocation 3-14
ping 8-7 Resource Broker 5-16

Resource Broker. See nbrb. T
restore process flow 3-26–3-29
tape manager. See bptm.
retention level 7-5, 7-6
TapeAlert 6-39
retention period 7-6
TCP/IP 8-3
robtest 6-13, 6-23, 7-21, 7-22
using 6-13–6-15 tctl 6-11
robust logging TechNotes

description 4-12 234412 7-20
enabling 4-13 234618 8-34
236274 1-6
route print 8-7 265806 1-6
267965 1-7
267977 8-45
S 269177 7-13
scan 6-24 273849 7-12
277242 1-26
scatter-gather list 9-27
278307 1-11
scientific method 2-4 278996 3-6, 7-20
scripts 279038 1-25
available_media 7-5, 7-7, 7-17, 7-24 279929 4-11
mail_bp_reports 2-17 280309 7-9
mail_dr_info 5-57 281578 1-4
mklogdir 4-5 281789 1-20
notify 2-18 282017 1-6
sgscan 6-24 282159 1-20
shared memory buffers 9-10–9-11 282162 1-20
285223 1-13
Shared Storage Option 1-12 285233 1-26
shoe-shining 9-18 286035 8-46
status codes 290185 1-16, 1-26
25 8-9 294899 1-5
41 8-47 document ID 6-33
54 8-25, 8-44, 8-47 telnet 8-7
57 8-44 TIME_WINDOW 7-19
59 8-16, 8-44
83 7-3, 7-11 tpautoconf 1-8, 1-25, 6-19, 6-23, 7-4
84 7-3, 7-12, 7-13 tpconfig 1-8, 6-22, 7-10
85 7-3 tpext 6-33
86 7-3, 7-12, 7-13, 7-14 tpreq 6-17
96 4-19, 7-3, 7-4, 7-7
tpunmount 6-18
98 7-3, 7-9
129 7-3 traceroute 8-8
205 8-25 tracert 8-8
213 8-25 troubleshooting reports 2-14
219 8-25
troubleshooting wizard 2-12
800 5-13
txxcd 3-15, 6-41
Sybase server
dbsrv 5-6 txxd 3-15, 6-41
NB_dbsrv 5-6 typographic conventions Intro-7
Veritas NetBackup 6.x Troubleshooting Techniques Index-5

U V
unified logs virtual tape library 6-4
description 4-3, 4-9 vmadd 7-10
disabling 4-23
vmd 3-15
enabling 4-11
layout 4-31 vmdareq 6-25
message types 4-32 vmglob 6-25, 6-27
naming convention 4-10 vmoprcmd 6-21
viewing 4-30 vmphyinv 7-21
upgrade vmpool 7-10
5.x to 6.x 1-10
clients 1-23 vmquery 7-24
cluster issues 1-11 vnetd 3-19, 3-20
configuration issues 1-25 VTL. See virtual tape library.
database inconsistencies 1-6 vxlogcfg 4-12, 4-14, 4-23, 4-32
EMM database issues 1-26
vxlogmgr 4-21, 4-23
guidelines 1-15
master server 1-18 vxlogview 4-30, 4-31, 4-33, 5-15, 5-17, 5-
media servers 1-22 19, 5-22
mixed version environments 1-14
NetBackup Remote Administration con-
sole 1-21 W
network issues 1-9 WINS 8-14, 8-17, 8-23
portal 1-16
prerequisites 1-4 work list 3-11, 3-12
process 1-17 working with technical suppot 5-56
recovering from problems 1-20
Shared Storage Option 1-12
supported servers 1-13
testing 1-21
version 5.x media servers 1-12
using robtest 6-13–6-15
utilities
bprd 3-13
bptestbpcd 8-27, 8-29
NBCC 1-4, 1-7
nbpemreq 3-13
nbsupport 2-9–2-10
nslookup 8-23
pbxcfg 8-38
perfmon 9-16
ping 8-7
tpconfig 6-22
traceroute 8-8
tracert 8-8
vmstat 9-16


Veritas Netbackup 6.X Troubleshooting Techniques For Unix (Lessons)

Hochgeladen von

Dokumentinformationen

Originaltitel

Copyright

Verfügbare Formate

Dieses Dokument teilen

Dokument teilen oder einbetten

Freigabeoptionen

Stufen Sie dieses Dokument als nützlich ein?

Sind diese Inhalte unangemessen?

Copyright:

Verfügbare Formate

Veritas Netbackup 6.X Troubleshooting Techniques For Unix (Lessons)

Hochgeladen von

Copyright:

Verfügbare Formate

Veritas NetBackup 6.

Lesson 1: Ensuring a Successful Upgrade

Lesson 2: Troubleshooting Methods and Tools

Lesson 3: NetBackup Process Flow

Lesson 4: Using Debug Logs

Lesson 5: Understanding the NetBackup Database

Lesson 6: Troubleshooting Devices

Lesson 7: Troubleshooting Media

Lesson 8: Troubleshooting Network Issues

ii Veritas NetBackup 6.x Troubleshooting Techniques

After completing this course, you will be able to:

Intro–2 Veritas NetBackup 6.x Troubleshooting Techniques

Before attending this course, you should have:

Course Introduction Intro–3

Additionally, you should know:

Additional Course Prerequisites

Intro–4 Veritas NetBackup 6.x Troubleshooting Techniques

Synopsis of the Lessons in the Course

Course Introduction Intro–5

Intro–6 Veritas NetBackup 6.x Troubleshooting Techniques

Typographic Conventions in Text and Commands

Convention Element Examples

Typographic Conventions in Graphical User Interface Descriptions

Convention Element Examples

Course Introduction Intro–7

Lesson Topics and Objectives

After completing this lesson, you will be

Topic 2: Upgrading from List the general procedure for upgrading

1–2 Veritas NetBackup 6.x Troubleshooting Techniques

A successful NetBackup upgrade depends upon:

Topic 1: Preparing for an Upgrade

Lesson 1 Ensuring a Successful Upgrade 1–3

Database Resolve any issues before attempting an

Failing backups Resolve any problems with failing

No NBAR or Remove NBAR and GDM before starting

Capture global and client attribute

1–4 Veritas NetBackup 6.x Troubleshooting Techniques

Lesson 1 Ensuring a Successful Upgrade 1–5

If you want to … Then use …

Scan the catalog and report bpdbm -consistency 2

Reclaim disk space by deleting bpimage -cleanup –allclients

Discard completed jobs from the bpdbjobs -clean

Consistency Issues or Catalog Inconsistencies

1–6 Veritas NetBackup 6.x Troubleshooting Techniques

The NetBackup Consistency Checker (NBCC)

Lesson 1 Ensuring a Successful Upgrade 1–7

Server Inventory Checklist:

Inventorying the Servers

1–8 Veritas NetBackup 6.x Troubleshooting Techniques

• Ensure that there are no connectivity issues.

Short Name FQDN

Note: Network Address Translation (NAT) is not supported.

Lesson 1 Ensuring a Successful Upgrade 1–9

1–10 Veritas NetBackup 6.x Troubleshooting Techniques

Note for VCS Clusters

Lesson 1 Ensuring a Successful Upgrade 1–11

Clustered nodes must run consistent

Shared Storage Mixed versions are not recommended,

Version 5.x nbpushdata -modify_5x_hosts

Shared Storage Option (SSO)

Version 5 Media Servers

1–12 Veritas NetBackup 6.x Troubleshooting Techniques

NBU 5.0 MP4 NBU 6.5