Sie sind auf Seite 1von 478

Carrier VoIP

Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures


Release: (I)CVM12 Document Revision: 06.02

www.nortel.com

NN10450-900 .

Carrier VoIP Release: (I)CVM12 Publication: NN10450-900 Document release date: 3 October 2008 Copyright 2008 Nortel Networks All Rights Reserved. Printed in Canada, the United States of America, and the United Kingdom LEGAL NOTICE While the information in this document is believed to be accurate and reliable, except as otherwise expressly agreed to in writing NORTEL PROVIDES THIS DOCUMENT "AS IS" WITHOUT WARRANTY OR CONDITION OF ANY KIND, EITHER EXPRESS OR IMPLIED. The information and/or products described in this document are subject to change without notice. Nortel, the Nortel logo, Business Made Simple and the Globemark are trademarks of Nortel. All other trademarks are the property of their respective owners.

Contents
New in this release
Features 9 Other changes 9

Introduction Disaster recovery fundamentals Disaster recovery configuration Preparation for disaster recovery

11 13 55 57

Recovering the SPFS system if updisk daemon dies 58 Installing the DDMS 59 Installing optional software on a Core and Billing Manager 64 Installing the OM Data Delivery application 68 Installing the Passport Log Streamer application 71 Installing the SBA and AFT software packages 73 Installing the GR740PT application server 74 Configuring RTB for a billing stream 76 SBRM core access 81 Configuring core access for SBRM through the Nortel Communication Server 2000 Core Manager 84 Creating the backup user ID on the core for SBRM 86 Applying software packages on a Core and Billing Manager using the CBMMTC interface 88 Cloning the image of the active node to the inactive node 91 Installing the Backup Restore Manager software 94 Configuring SSH between the backup restore manager servers 96 Configuring SBA backup volumes on the core 102 Starting or stopping the automated synchronous backup restore manager service 107 Invoking the network-level backup (SBRM-bkmgr) through telnet 110 Invoking the device-level backup (DBRM-bkmgr) through telnet 115 Backing up an XA-Core office image from disk to tape 119 Backing up Call Agent 126 Backing up the SST 129
Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

4 Rescheduling backup time 132 Backing up security related files manually 134 Saving the Ethernet Routing Switch 8600 boot configuration file 136 Configuring the automated synchronous backup restore manager service 138 Configuring Automated INI file backup on a Media Server 2000 node 142 Changing SNMP community string passwords 145 Backing up INI files for all nodes 147 Displaying Media Server 2010 current node configuration 148 Backing up the APS-specific Oracle database and application files 150 Using the command line interface to perform backups 153 Performing a backup of Oracle/MySQL data on an SPFS-based server 156 Backing up UAS configuration files 160 Performing a manual backup of the Policy Controller database 162 Performing a backup of file systems on an SPFS-based server 164 Taking a full IEMS backup 169 Backing up an SPFS-based security client 171 Backing up a Multiservice Switch or Media Gateway device 172 Performing a backup operation 178 Creating system image backup tapes (S-tapes) manually 181 Displaying actions a user is authorized to perform 193 Logging in to the Nortel CS 2000 Core Manager 195 Preparation for SBA installation and configuration 197 Configuring SBA backup volumes on a billing stream 220 Accessing the MATE 222 Administration: Creating Disaster Recovery Floppy Disks 224 Power-On and boot a Policy Controller unit 225 Viewing the operational status of the NCGL platform 226 Viewing the operational status of the SIP Gateway application 228 Verifying synchronization status 232 Booting the XA-Core from a Reset Terminal 233 Resetting the Ethernet Routing Switch 8600 using a saved configuration file 236 Restoring a Call Agent 237 Restoring Session Server Trunks 240 Restoring a database to a Session Server unit 241 Restoring audio files to a UAS node 242 Restoring audio files to a Media Server 2000 Series node 243 Restoring the APS-specific Oracle database and application files 244 Restoring a Multiservice Switch or Media Gateway device 247 Performing a full system restore 253 Restoring a system configuration from a data snapshot 254 Restoring Core Element Manager data 260 Restoring the central security server 262 Performing a full restore of the software from S-tape 265 Performing a partial restore of the software from S-tape 270
Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

5 Recovering backup files from lost backup volumes 275 Restoring the Oracle/MySQL data on an SPFS-based server 277 Clearing the MATE alarm 281 Stopping the SAM21 Manager server application 282 Stopping the SESM server application 284 Stopping the NPM server application 286 Stopping the APS server application 288 Starting the APS server application 290 Starting the SESM server application 293 Starting the SAM21 Manager server application 295 Starting the NPM server application 297 Performing a full system restore on an SPFS-based server 299 High-availability configuration (two servers) 302 Locking a GWC card 305 Locking the Call Agent 308 Installing the remote backup server 310 Scheduling automatic backups on the remote server 315 Viewing configuration information for remote server backups 318 Performing a manual backup of the target server 320 Viewing logs from a remote backup 322 Initiating a recovery back to the cluster 323 Initiating a switch over to the remote backup server 325 Disabling Ethernet Routing Switch 8600 WAN Edge routing 327 Disabling and re-enabling the router ports on the Ethernet Routing Switch 8600 329

Network power down


Powering down an MG 9000 device 335 Powering down the IPmedia 2000 shelves 337 Powering down the SAM16 shelves 338 Powering down a Border Control Point 339 Powering down the MCS servers 340 Powering down the MDM workstation 341 Powering down an SPM device 342 Powering down a Media Gateway/Multiservice Switch 7400 345 Powering down a Media Gateway/Multiservice Switch 15000 346 Powering down a USP 347 Powering down the SAMF frame 348 Powering down the Call Control Frame 350 SAM21 Shelf Controller security and administration fundamentals 352 Powering down the Border Control Point Manager 355 Powering down a Netra 240 server 356 Powering down the SDM 357 Powering down a Ethernet Routing Switch 8600 358

331

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

6 Shutting down the IEMS server 359 Powering down the XA-Core 360 Performing a partial power down of the MG 9000 363 Performing a partial power down of the Media Gateway/Multiservice Switch 15000 364 Performing a partial power down of the USP 368 Performing a partial power down of the Ethernet Routing Switch 8600 369 Performing a partial power down of the Netra 240 server 370 Performing a partial power down of the XA-Core 371

Network power up
Powering up the XA-Core 380 Powering up the Message Switch 396 Powering up the Enhanced Network 399 Powering up an Ethernet Routing Switch 8600 401 Powering up the Link Peripheral Processors 402 Powering up the SDM 405 Powering up a Netra 240 server 406 Powering up the Border Control Point Manager 407 Powering up the SAMF frame 408 Powering up the Call Control Frame 410 Powering up a USP 412 Powering up a Media Gateway/Multiservice Switch 15000 414 Powering up a Media Gateway/Multiservice Switch 7400 415 Powering up the MDM workstation 417 Powering up an SPM device 418 Powering up the MCS servers 420 Powering up a Border Control Point 424 Powering up the IPmedia 2000 shelves 425 Powering up the SAM16 shelves 426 Powering up an MG 9000 device 427 Restoring certificates and Pre-shared Keys for IPSec 432 Restoring the MG 9000 manager system 436 Restoring files from a DVD-RW 437 Reloading or restarting a SAM21 Shelf Controller 440 Restarting or rebooting a GWC card 441 Performing a partial power up of the Netra 240 server 443

375

Disaster recovery maintenance


Performing a partial Performing a partial Performing a partial 8600 451 Performing a partial Switch 15000 power down recovery of the USP 447 power down recovery of the XA-Core 448 power down recovery of the Ethernet Routing Switch power down recovery of the Media Gateway/Multiservice 452

445

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

7 Performing a partial power down recovery of the SPM 453

Disaster recovery power up


Unlocking the Call Agent 456 Unlocking a GWC card 457

455

Reference

461

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

New in this release


The following sections detail what is new in Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures ((NN10450-900)) for software Release (I)CVM12.

Features

Features (page 9) Other changes (page 9)

There were no feature changes made to this document.

Other changes
There are no other changes in this document.

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

10

New in this release

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

11

Introduction
This document provides an overview of the Disaster Recovery process for Carrier VoIP solutions. A disaster is defined as one of the following events:

fire flood lightning storm explosion tornado earthquake hurricane terrorism any other incident causing damage beyond normal repair to telecommunications facilities

Use this document to restore communication capability within the affected central office or network.

Navigation
Disaster recovery fundamentals (page 13) Disaster recovery configuration (page 55) Preparation for disaster recovery (page 57) Network power down (page 331) Network power up (page 375) Disaster recovery maintenance (page 445) Disaster recovery power up (page 455) Reference (page 461)

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

12 Introduction

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

13

Disaster recovery fundamentals


This chapter describes disaster recovery basics specific to the Nortel Carrier VoIP solutions.

Navigation
Power conservation (page 14) Backup and restore (page 15) Synchronized Backup Manager fundamentals (page 15) Sequence of events to run the Synchronized Backup Manager (page 17) Hardware and software requirements (page 18) Requirements and restrictions (page 19) CNM network element backup (page 21) Administration: OAMP workstation backup (page 22) Restoring Multiservice Switch or Media Gateway service data (page 23) Operational Measurement Delivery (page 24) Geographical survivability (page 30) Site failure in a geographic survivable configuration (page 32) Fresh pre-install of CBM 850 cold u0 (page 32) Maintaining sites in a geographic survivable configuration (page 33) Geographical survivability impacts to CS 2000-Compact (page 33) Protocols BGP and OSPF activation (page 41) USP mated pairs (page 42) SSL geographic survivability (page 47) Geographic survivability limitations and restrictions (page 49) Real Time Billing fundamentals (page 50)

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

14 Disaster recovery fundamentals

Power conservation
You can partially power down several components to conserve power but keep the network in an operational state. Power conservation procedures typically involve powering down inactive or spare units, or keeping only a certain set of hardware or cards active. The following table lists the components and types of power procedures.
Table 1 Power procedures Component or network element XA-Core Ethernet Routing Switch 8600 SDM (CS 2000 Core Manager) Netra 240 Border Control Point Services Application Module Frame Call Control Frame USP Media Gateway/Multiservice Switch 15000 Media Gateway/Multiservice Switch 7480 MDM SPM MCS Border Control Point Manager Media Server 2000 Series UAS MG 9000 Full X X X X X X X X X X X X X X X X X X X X X X Partial X X

When you perform a partial power down for a component, you must follow the corresponding partial power down recovery procedure. Do not use a partial power down recovery procedure to recover from a full power down situation. Conversely, do not use a full power down recovery procedure to recover from a partial power down situation.

ATTENTION
Perform a recovery from a partial power down of the MG 9000 by using the same procedure as recovery from a full power down.

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

Synchronized Backup Manager fundamentals

15

Backup and restore


Perform backup and restore operations for CVoIP solutions at the component level. Perform the operations on the components at different intervals during periods of low service activity. Do not make provisioning or configuration changes during the backup and restore window. Back up all related devices in the solution during the same window of time. You cannot make changes to any related devices during this period until all backups are complete or those changes are not captured as part of the coordinated backup and are lost in case of the need to restore the system. For example, the CS 2000, CS 2000 GWC Manager, and MG 9000 Manager all share line service data and you must back them up in the same window. You must restore all related devices together to return the system to a known state. You cannot make changes to the system until the restoration is complete or the changes result in data mismatches.

Synchronized Backup Manager fundamentals


Throughout this document, the backup and restore functionality delivered under this feature is referred to in the following three distinct ways:

Synchronous Backup Restore Manager (SBRM) is a network backup utility that is installed in the /opt/bkresmgr/nbm directory. The SBRM controls and synchronizes the activities related to backup across other devices that contain related backup software. The SBRM is invoked from the IEMS Backup Manager. Device Backup Restore Manger (DBRM) refers specifically to the device-level backup software that is installed in the /opt/bkresmgr/cbm directory. The DBRM is controlled by the SBRM and it is not recommended running the DBRM directly from a CLUI interface. There are few exceptions and they are specified explicitly in this document if it requires to be run at DBRM level. WARNING:Status commands executed at the DBRM cli may be idle even if a NBM (network backup) is in progress. To ensure that no backup is in progress when required to backup at DBRM level, please check the status of both NBM and DBRM.

Backup Restore Manager is a generic term encompassing both SBRM and DBRM functionality that refers to the software that is common to both.

The SBRM provides a centralized mechanism that you use to initiate a synchronous backup for the core components of the CVoIP Call Server. This capability reduces the possibility of error caused by manual backup procedures on the various component platforms spread across the Call Server solution.
Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

16 Disaster recovery fundamentals

From Release (I)SN08, the Backup Manager provides synchronous centralized control of the backup functionality resident within the following Call Server components:

CS 2000 Management Tools (CSMT) server MG 9000 Manager (MG9K) server Supernode Data Manager (SDM)/Core Billing Manager (CBM) server. The software on the SDM/CBM only allows control of the XACore/3PC backup. Backup Restore Manager does not back up SDM/CBM data. Integrated Element Management System (IEMS) server

ATTENTION
The IEMS can reside with the CSMT (and other applications) on the CSMT server.

From the IEMS GUI, the user launches the network backup utility (SBRM) command line interface and initiates a backup for CVoIP Call Server components. The network backup utility (SBRM) receives the user request, and initiates a backup command sequence (using CLI commands over SSH) to execute a synchronous device-level backup of the CVoIP Call Server components. Once the backup is complete, the backup files are stored locally on each component. The following figure shows a high-level functional overview of the SBRM.
Figure 1 High-level functional overview of the SBRM

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

Sequence of events to run the Synchronized Backup Manager

17

Sequence of events to run the Synchronized Backup Manager


The component for which the SBRM is installed, SDM or the Core and Billing Manager, determines the sequence of events.

Installation and conguration of SBRM on the SDM


Installation and configuration of SBRM on the SDM requires execution of the following procedures in the order given:

"Installing and configuring the Backup Restore Manager application on the CS 2000 Core Manager" in Nortel Carrier Voice over IP Upgrade and Patches ( (NN10440-450)) . SBRM core access (page 81) Configuring SSH between the backup restore manager servers (page 96) Configuring core access for SBRM through the Nortel Communication Server 2000 Core Manager (page 84) Creating the backup user ID on the core for SBRM (page 86) Configuring the automated synchronous backup restore manager service (page 138) Common procedures for all components (page 17)

Installation and conguration of SBRM on the Core and Billing Manager


Installation and configuration of SBRM on the Core and Billing Manager requires execution of the following procedures in the order given:

"Installing and configuring the Backup Restore Manager application on the CS 2000 Core Manager" in Nortel Carrier Voice over IP Upgrade and Patches ( (NN10440-450)) . Creating the backup user ID on the core for SBRM (page 86) Configuring SSH between the backup restore manager servers (page 96) Configuring core access for SBRM through the Nortel Communication Server 2000 Core Manager (page 84) Common procedures for all components (page 17)

Common procedures for all components


The following procedures are required to configure the SBRM on all components and are executed in the order given.

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

18 Disaster recovery fundamentals

To configure SSH between the backup manager restore servers (if not already completed), see the procedure, Configuring SSH between the backup restore manager servers (page 96). The following procedures provide additional information about the SBRM:

To configure automated execution of backups, see the procedure, Configuring the automated synchronous backup restore manager service (page 138). To control the automated execution of backups, see the procedure, Starting or stopping the automated synchronous backup restore manager service (page 107).

To configure the IEMS to support the SBRM, see the following documents:

To add the SBRM to the IEMS using the Java Web Start Client or the Web Client, see Nortel IEMS Configuration ( (NN10330-511)) . To launch the CLI for the SBRM using the IEMS Java Web Start Client, see Nortel IEMS Fundamentals ( (NN10329-111)) . You cannot launch the SBRM using the IEMS Web Client.

Invoke a network backup (where several components are backed up at once) from the IEMS Backup Manager. In rare instances, it can be necessary to run various portions of the backup process on individual components or servers. This capability is implemented as a CLUI interface.

Execution of a backup at the component-level ATTENTION


Run a backup at the component-level only in rare circumstances. Take care to keep the SBRM and the DBRM in sync.

Invoke the CLUI interface to perform a backup at the component-level by logging on to the appropriate component using a Telnet session. See the procedure, Invoking the network-level backup (SBRM-bkmgr) through telnet (page 110).

Hardware and software requirements


Backup Restore Manager functionality requires that the appropriate software is resident and configured in all platforms which require synchronized imaging. The Backup Restore Manager software is included in various platform software packages, including the SPFS and SDM. SPFS Backup Restore Manager functionality is used for integration of Call Server Management Tools (CSMT), the Media Gateway 9000 Element Manager (MG9KEM), the IEMS, and the CBM. The software on the SDM/CBM only allows control of the XACore/3PC backup. Backup Restore Manager does not back up SDM/CBM data.
Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

Requirements and restrictions

19

Requirements and restrictions


The following requirements and restrictions apply to Synchronized Backup Manager functionality from (I)SN08:

All platforms requiring Backup Manager software functionality must be Java compatible and must provide a resident Java runtime system. The SBRM software does not support restoration of generated synchronized backup images. You must follow existing manual restore procedures to utilize the backup images produced by the Backup Restore Manager. The Backup Restore Manager system does not perform automatic spooling of backup data or image files to secure or redundant servers. You must manage your secure server archiving capabilities. The Backup Restore Manager system publishes data or image files in predetermined and configurable directories on various servers:

SPSF Platform data/image file location: /data/bkresmgr/backup XACore/3PC data/image file location: Configurable

Backup of program store and associated patches is not part of the functionality associated with Synchronous Backup Restore Manager. If you need to completely restore a server or component, you must reinstall your applications and reapply the associated patches. The backup abort command can require manual cleanup or removal of some backup files. You receive notification at abort command completion. You cannot run the backup process while certain constructive or destructive system audits are in progress, for example, SESM CS 2000 Data Integrity Audit. In general, if you attempt a backup while these audits are in progress, the backup attempt is rejected. However, some constructive or destructive audits are postponed when a backup request is received, for example, MG9KEM Network Element Audit. The architecture of the system in question determines the appropriate behavior. In general, any system rejections of the backup request result in an appropriate message displayed to the Synchronous Backup Restore Manager client user. No system-wide coordination between Synchronous Backup Restore Manager and other applications, for example, Upgrade Manager, CS 2000 Audits, and CM dump scheduling, is delivered. You must manually manage all application activity interdependencies. For example, you must stop scheduled activities such as audits during system backup. Also, you must cancel automated CM image dumps because the Synchronous Backup Restore Manager drives the dump within a synchronized system-wide backup window.

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

20 Disaster recovery fundamentals

APS and NPM queries are not allowed during backup. This limitation is due to the application shut down during backup. Active clients are informed of the application shutdown. Nortel does not recommend that you backup the MG 9000 EM in isolation due to the CSMT platform data dependency. The MG 9000 EM database is hosted on the CSMT server. Synchronous Backup Restore Manager user authorization only controls the launch of the CLI. CLI command-specific authorization is not provided, therefore only users authorized to launch the Synchronous Backup Restore Manager CLI can perform query commands at the Synchronous Backup Restore Manager CLI. The software on the SDM/CBM only allows control of the XACore/3PC backup. Backup Restore Manager does not back up SDM/CBM data.

During the backup, use of the various GUI and command line provisioning interfaces is restricted from attempting the following actions which result in configuration and provisioning data changes:

Add, delete, or update GWCs at the Call Server Management Tools (CSMT) GUI or OSSGate interface. Add, delete, or update line service at the CSMT OSSGate interface. Manually invoke (or run by using the scheduler) the various audits at the CSMT GUI. Add, delete, or update objects in the IEMS topology by using the IEMS client. Add or modify collection jobs by using the IEMS client. Use the Runtime Administration interface at the IEMS client to perform OSS configuration. Use the Security Administration interface at the Integrated client to add or delete users, groups, or operations. Use the Security Administration interface at the IEMS client to change a user password. Add, delete, or update trunks or carriers by using the CSMT. Node provisioning or deprovisioning by using the SAM21Manager. ATM connection set provisioning or deprovisioning by using the SAM21 Manager.

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

CNM network element backup

21

CNM network element backup


The Network Element Backup application provides a user-friendly solution to backup the network element system and configuration data. Some of the key features of this application include:

A single operational user interface for on-demand or scheduled backups. Automatic discovery of network elements. Consolidated view of backup configurations and the status of supported network elements. Policy-based backup for each type of network element. Multiple simultaneous backups. A framework to support additional network elements. The status of backup jobs can be monitored during any phase of the backup.

Network elements can include:

networking devices element managers network management servers

Some network element backup images are stored at the network element (native storage). Some network element backups are stored on the local network management server (central storage). Some network element backup images can be stored on either native storage or central storage. The Network Element Backup application launches and controls the backup process but some pre-backup and post-backup tasks must be performed manually (for example, inserting a tape in a tape drive before the backup and removing the tape from the tape drive after the backup). If the backup process requires tasks to be performed manually, an overview of these tasks is provided in the backup description for the network element and task details are provided in this document. Network Element Backup software is delivered with default backup parameters for each supported network element. You must update these backup parameters for any customizations applied to your network elements. Some backup parameters apply to all instances of that network element type, while other backup parameters apply only to selected instances of a network element. For more information, see CDMA Network Manager Administration -Network Element Backup (NN20000-344) .

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

22 Disaster recovery fundamentals

Administration: OAMP workstation backup


You can perform two types of backup on your OAMP workstation, manual and automatic. Perform manual backups from the desktop by manually initializing the backup process. Perform automatic backups according to the settings defined in the tape drive application.

ATTENTION
Nortel recommends that you use the automatic backups to ensure that the data that is stored on the tapes is up-to-date.

The OAMP workstation is equipped with a tape drive, software to support tape backup, and five blank data tapes. These tapes provide approximately one month of backups to be kept on-site.

ATTENTION
Nortel recommends that you label the tapes to indicate the OAMP workstation associated with the backups to ensure that any recovery operations are performed from the correct tape.

Backup schedule
Create a schedule for your automatic backups to ensure up-to-date storage of the system configuration of your OAMP workstation.

ATTENTION
Nortel recommends that you perform a full system backup once a week and modified (differential) backups once a day.

Manual backup
Nortel recommends that you perform a manual backup of your OAMP workstation after initial installation. To perform a manual backup of the system configuration for your OAMP workstation, see the documentation provided with your Colorado or Veritas software.

Automatic backup conguration


There are two types of automatic backup, full system backup and modified (differential) system backup. The full system backup saves all of the files contained in the hard disk on your OAMP workstation. The modified (differential) system backup saves only the files on the hard disk that have been modified since the last full system backup.

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

Restoring Multiservice Switch or Media Gateway service data

23

ATTENTION
When you use automatic backup, you must leave your OAMP workstation turned on with Windows running and ensure that there is a tape in the tape drive.

The initial installation of your system configures the tape drive application to perform an automatic full-system backup once a week, every Saturday at 1:00 a.m. However, you can modify the settings in the tape drive application.

ATTENTION
If you do not change the default setting for automatic full system backup, you will need to change the tape every Friday.

ATTENTION
For automatic backups to work, the tape drive scheduler icon must be active in the notification box in the taskbar with the automated daily backups option enabled. Your OAMP workstation must remain on with Windows active and a tape must be in the tape drive.

Verify the current modified system backup settings


There is a tape drive configuration file associated with the modified system backup. To verify that the current settings in the configuration file matches the system requirements, see the tape drive documentation provided with the OAMP workstation.

View the current automatic backup schedule


To view the current settings for automatic backup in the tape drive application, see the tape drive documentation provided with the OAMP workstation.

Restoring Multiservice Switch or Media Gateway service data


This section describes using Service Data Backup and Restore to restore backed up service data from the backup site to the Multiservice Switch 7400/15000 or Media Gateway 7400/15000. The Service Data Backup and Restore tools provide the following three types of restore:

A full restore restores all backed up service data to the selected devices. An incremental restore restores service data based on a specified date. Like the full restore, you can perform an incremental restore on either one or multiple devices. A selective restore restores specific service data that you select. Like the full restore, you can perform a selective restore on either one or multiple devices.

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

24 Disaster recovery fundamentals

ATTENTION
Nortel does not recommend that an active file (current view) be restored on a Multiservice Switch 7400/15000 or Media Gateway 7400/15000. When you restore the current view, you may overwrite the existing current view with different content. This action will cause an outage of the Backup and Restore tool.

Operational Measurement Delivery


The Operational Measurement Delivery (OMD) application collects customer-defined operational measurement (OM) data from the DMS switch, and stores the data in OM report files on the core manager in comma-separated value (CSV) format. Configure the OMD application using the OM user interface (OMUI). An OM report file is a collection of OM groups that are monitored at selected reporting intervals. Secure File Transfer (SFT) or File Transfer Protocol (FTP) sends OM report files from the core manager to an operations support system (OSS). A data browser such as a spreadsheet program provides access to the contents of the files.

Report elements
Report elements define the content of OM report files, and combine content of related OM groups for monitoring and analysis. A report element contains a user-defined report element name, a reporting interval for a report element (five minutes, or the office transfer period of 15 or 30 minutes), and names of the OM groups and registers.

Subtraction proles
The subtraction profile determines the change in the value of an OM group register between five-minute OM reports, as defined in a report element. The subtraction profile applies only after the reporting interval is set to five minutes. The following table lists the types of subtraction profiles.
Table 2 Subtraction profiles Type Single Double Non-subtracti on Description A single register represents a running total Two registers (base and extension) represent a running total Subtraction is not performed on selected registers

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

Operational Measurement Delivery

25

Data collection schedules


A data collection schedule defines start and stop times for OM report collection. The collecting interval determines how often in the time period an OM report collection occurs. The data is collected to the same report file for schedules with collecting intervals after midnight. The following table lists the data collection schedule types.
Table 3 Data collection schedule repetition types Repetition Daily Schedule information Daily start and stop time. Format: hhmm, where hh = hour (00 to 24), and mm = minute (00 or 30). Specifies only a single time period; for multiple time periods in the same day, you must define multiple schedules. Weekly start and stop time. Values: Sunday, Monday, Tuesday, Wednesday, Thursday, Friday, and Saturday. Format: hhmm; multiple days can be specified in same schedule. Monthly start and stop time. Values:1 to 31. Format: hhmm; multiple days can be specified in the same schedule.

Weekly

Monthly

File rotation schedules


File rotation schedules specify when to rotate report files. File rotation closes an open report file and moves it to the /omdata/closedNotSent directory on the core manager. Each file rotation schedule contains

a user-defined file rotation schedule name a repetition rate for the rotation schedule based on either the number of report records collected or the number of hours to collect records a schedule that defines the time to rotate the report file

The data collection and file rotation schedules operate independently of each other. If a file rotation schedule event occurs during a scheduled data collection period, the file rotation schedule closes and rotates the OM report file, and a new OM report file with the same name is opened. The new file starts collecting immediately and continues until the end of the collection period. The open OM report file remains in the /omdata/open directory until the file rotation schedule closes it and rotates it to the /omdata/closedNotSent directory.

File transfer destinations


File transfer destinations define remote downstream destinations of OM report files. Each destination entry contains

a user-defined file transfer destination name the valid IP address of a remote destination host (xxx.xxx.xxx.xxx)

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

26 Disaster recovery fundamentals

the FTP port address of the remote host (default: 21) the remote host logon ID and password

The core manager does not authenticate the IP and port addresses or the logon ID and password. An invalid destination causes the file transfer to fail. After a file fails to transfer, log entries are written to the customer log file at /var/adm/custlog. The file is not re-sent, and the report file must be transferred manually using either the OMFTP command, SFT or standard FTP.

File transfer schedules


File transfer schedules specify when to transfer OM report files downstream. Each file transfer schedule contains a

user-defined file transfer schedule name repetition rate for the transfer schedule schedule defining when to transfer the report file (if using a repetition rate) remote file transfer destination host system (<16 destinations/schedule) destination storage directory for each defined transfer destination

The files are transferred downstream using FTP, and move from the /omdata/closedNotSent directory to the /omdata/closedSent directory. If a scheduled file transfer fails, a log is raised and the report file that could not be transferred moves to the /omdata/closedSent directory. The OMDD keeps track of the destination to which the report file could not be transferred. Then, at the next scheduled file transfer, the OMDD attempts to send the report file to the destination again. The OMDD will repeat this activity until one of the following situations occurs:

the file is transferred successfully the file exceeds the retention period for the closedNotSent directory the file gets deleted during an audit because the omdata filesystem usage has exceeded the allowable limit the file is deleted by the omdelete utility

Report registrations
A report registration links information from the report element and schedules for data collection, file rotation and file transfer to collect OM data. The user can create up to 32 report registrations. After a report registration has been created, it can be deleted but not modified. Each

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

Operational Measurement Delivery

27

report registration contains user-defined names for the report registration, report elements and each schedule type. The schedules become active immediately after the creation of the report registration. An OM report file opened by the data collection schedule in the /omdata/open directory uses the name of the report registration as part of the OM report file name. Linking a file transfer schedule into a report registration provides regular and automatic transfers of OM report files to remote downstream destinations. Unless you link a file transfer schedule to a report registration, you must manually transfer your OM report files downstream.

Report registration limit


The report registration limit is the maximum number of report registrations that can be configured on a core manager without affecting processing performance. The number of report registrations range from 1 to 32 (default value: 32). To set the limit, use the Set Report Registration Limit option from the OMUI main menu.

File retention periods


A cleanup of OM report files that have been sent downstream automatically occurs every night at midnight (00:00 or 24:00). Files in the /omdata/closedSent directory are deleted at an interval based on the file retention period defined in the OMUI (range: 1 to 14 days). The default interval is set to 7 days at OMD installation. Unsent OM report files older than 32 days in the /omdata/closedNotSent directory are deleted. This 32-day default value is read from a configuration file set up after the core manager is commissioned.

OMD data collection capacity


Collection of more than 10,000 tuples reduces core manager performance and the retention period for OM report files. To determine the number of tuples in an OM group, either monitor the OM group and count the tuples in the report file or use the OMSHOW command from the MAP (maintenance and administration position) on the DMS switch. Use the formulas in the following table to calculate the limit for OMD data collection.

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

28 Disaster recovery fundamentals Table 4 Formulas for calculating the limit for OMD data collection OMD data capacity transfer type 5- and 15-minute 5- and 30-minute Formula x+ y/3 = n <= 10,000 tuples (without loss of data) x+ z/6 = n <= 10,000 tuples (without loss of data)

where x y z n = = = < the number of OM tuples collected every 5 minutes the number of OM tuples collected every 15 minutes the number of OM tuples collected every 30 minutes 10,000 tuples

The following table lists the current OMD data collection capacity.
Table 5 OMD maximum data collection capacity Transfer type 5-minute 15-minute 30-minute Capacity (number of tuples) 6000 12,000 24,000

OM report le naming
Report files are named according to the report registration name, file creation date and time, name of the switch generating the OMs, and reporting interval. See the following example file name and explanation.

OM report le contents
Tuple information for an OM group can be viewed in CSV format from the OM report file on the core manager, and by entering the OMSHOW command on the MAP. The following table shows an OM report file.

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

Operational Measurement Delivery Table 6 Contents of an OM report file Re g1 Va Reg lu 2 Na e me 0 0 0 0 0 0 0 ANF ANF ANF ANF ANF ANF ANF Re Re g g2 31 Va Na lu m e e 0 0 0 0 0 0 0

29

Date 2/23/00 2/23/00 2/23/00 2/23/00 2/23/00 2/23/00 2/23/00

Tim e 3:35 :00 3:35 :00 3:35 :00 3:35 :00 3:35 :00 3:35 :00 3:35 :00

Switc h Na mes 250U 250U 250U 250U 250U 250U 250U

Gro up Na me TRK TRK TRK TRK TRK TRK TRK

Key/Info Field ISU_GWC.2W.0.0 ESADGTR.OG.0.0 HSET.OG.3.0 JACK.OG.2.0 LTU.OG.2.0 MONTALK.OG.0.0 OCKT.OG.0.0

Reg 1 Na me AOF AOF AOF AOF AOF AOF AOF

Re g3 1 Va lu e

Audits
The OM report files are stored in the omdata filesystem. This filesystem is audited every 30 minutes for the amount of usage. To ensure that the omdata filesystem usage does not reach 100% at any time, the system performs the following actions:

After the filesystem usage reaches 60%, Major trouble log SDM338 is raised indicating that OM report files will be deleted at the time of the next audit, if usage exceeds 90%. Then, if usage exceeds 90% at the time of the next audit log SDM639 is raised indicating all report files in the closedSent directory will be deleted, and the report files are deleted. If omdata filesystem usage does not fall below 80% after deletion of all report files in the closedSent directory, report files from the closedNotSent directory will be deleted, starting from the oldest file, until the usage is at 80% or less. As each report file is deleted from the closedNotSent directory, log SDM631, which describes the action, is raised.

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

30 Disaster recovery fundamentals

Geographical survivability
Geographic survivability is the distribution of components across a geographic area to ensure services continue in the event of a disaster. A disaster can include fire, flood, tropical storm, or act of terrorism. The following figure shows the topology for CS 2000 - Compact support for geographic survivability.
Figure 2 Geographic survivability topology

The CS 2000 - Compact supports geographic survivability through the failover and sparing functionality of individual nodes. The configuration for the CS 2000 Management Tools (CMT) and Integrated Element Management System (IEMS) servers and the Core and Billing Manager (CBM) 850 is as follows:

Site A is configured with the CMT and IEMS high-availability (HA) server pairs Site B is configured with the CBM 850 HA server pairs (referred to as CBM 850 HA u0 and u1) Site A is configured with an additional CBM in to be used in the event of disaster at Site B (referred to as CBM 850 standby u0) Site B is configured with an additional server for the CMT and IEMS in the event of disaster at Site A

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

Geographical survivability

31

For a communication server, geographic survivability requires that redundant components (other than OAM components) reside at different sites (referred to as Site A and Site B). For OAM components, the Automatic Backup and Accelerated restore feature (known as remote backup) remotely backs up all data on the target unit. This provides a standby backup system ready to provide service should the primary system or cluster be unavailable for an extended period of time (for example, catastrophic site loss). The remote backup can assume the identity of the target system with data and files accurate to the last sync and will be located at a different site from the target system. Remote backup performs the backup with a TCP/IP connection, and stores an exact copy on the standby server which can be quickly and remotely activated. This remote backup copies all files in each file system marked for backup using the same behavior as a full system backup. A remote backup configuration tool is provided to set the necessary parameters for backup to automatically occur from one to four times a day. This tool also provides a facility for manually initiating a backup and monitoring its progress. The standby server has an identical copy of files from the last backup, so it can become the primary system by changing the boot pointer and rebooting. When the primary site becomes available, the remote backup feature can be reused to transfer current system configuration back to the primary site and system.

CAUTION
If configuration, provisioning, patching or other write-type operations occurred since the last remote backup, the remote backup system can be out of sync compared to the data in network elements and/or the primary OAM system. When initiating a switchover to a remote backup OAM server, do not execute configuration, provisioning, patching or other write-type operations through the remote backup OAM system until out-of-sync conditions are cleared. Take actions before initiating the switchover to a remote backup OAM server (that is, response to a geographic or other prolonged outage of the primary OAM system) to halt or prevent write-type operations by OSSs and operations personnel until an in-sync status is achieved. If a site outage is imminent (for example, threatening severe weather conditions), consider precautionary preparations to discontinue write-type operations. Make manual backups to the remote backup OAM servers to ensure data synchronization of the remote OAM server before performing a switchover.

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

32 Disaster recovery fundamentals

Apply similar precautions when initiating a recovery back to the primary cluster. In this more controlled scenario, discontinue write-type operations before initiating this procedure

Site failure in a geographic survivable conguration


There are two scenarios for recovery of a geographic survivable network configuration:

Site A failure (page 32) Site B failure (page 32)

Site A failure
A loss of Site A would include a loss of the CMT and IEMS servers. This loss would result in the following losses of functionality:

all OAM&P functions of the CMT and IEMS, including GUI access, element management, alarms, and non-Core logs access to the CS 2000-Compact Core Manager on the CBM 850

The responsive action in this scenario is to initiate a switchover to the standby CMT and IEMS servers at Site B.

Site B failure
A loss of Site B would include a loss of the CBM 850 servers. This loss would result in the following losses of functionality:

transfer of billing records from the Core to the CBM 850 billing records resident in the CBM 850 that had not been offloaded to an OSS ability to receive scheduled Core OMs and logs (resulting in them being discarded) access to the CS 2000-Compact Core Manager backup bootp load repository

The responsive action in this scenario is to initiate a switchover to the standby CMT and IEMS servers at Site A.

Fresh pre-install of CBM 850 cold u0


To reduce the duration of downtime during an outage, it is recommended that the CBM 850 cold u0 be pre-installed. Contact your next level of support for this task.

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

Geographical survivability impacts to CS 2000-Compact

33

Maintaining sites in a geographic survivable conguration


The following procedures contain information for OA&M auto backup and accelerated restore capability:

Installing the remote backup server (page 310) Scheduling automatic backups on the remote server (page 315) Viewing configuration information for remote server backups (page 318) Performing a manual backup of the target server (page 320) Initiating a recovery back to the cluster (page 323) Initiating a switch over to the remote backup server (page 325) Disabling Ethernet Routing Switch 8600 WAN Edge routing (page 327) Disabling and re-enabling the router ports on the Ethernet Routing Switch 8600 (page 329)

Geographical survivability impacts to CS 2000-Compact


The Geographic Survivability feature allows services to continue in the event of a natural or man-made disaster. A sub-panel to the Call Agent Card View Provisioning panel (tab) allows you to enable or disable the feature. The following figure shows an example configuration with Geographic Survivability enabled.

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

34 Disaster recovery fundamentals Figure 3 Call Agent Card View Provisioning panel: Geographic Survivability enabled

Failure scenarios
The following table provides summaries of system responses during various failure scenarios when Geographical Survivability is enabled. The scenarios assume redundant configurations are located in two separate sites.

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

Geographical survivability impacts to CS 2000-Compact Table 7 Failure system response summaries Scenario Ethernet Routing Switch 8600 failure at one site Response At the site with the failed routing switch: All nodes lose mate connectivity by using Ethernet.

35

The Call Agent loses WAN backup connectivity. FC or GigE call data sync link remains up until the Call Agent resets. The Call Agent resets itself approximately 1 minute after losing network connectivity and tries to boot from the network. The Call Agent keeps trying to boot until the network recovers and the boot is successful. The Call Agent detects IST loss, but cannot disable Ethernet Routing Switch 8600 routing. USPc detects isolation and takes down SS7 links at the site with the Ethernet Routing Switch 8600 failure. USP mated pair configurations: Mated pairs connected to pair of gateway POP Ethernet Routing Switch 8600s: No impact to both USPs from a failure. Mated pairs split across a CS-LAN site (each connected to the Ethernet Routing Switch 8600 of local CS 2000 CS-LAN): For CS 2000 - Compacts that share the same CS-LAN, the USP detects isolation and takes down its SS7 links. An Ethernet Routing Switch 8600 failure does not impact USP mated pair behavior in other CS 2000 - Compact configurations. Mated pairs split across a CS-LAN site (each connected to one Ethernet Routing Switch 8600 connected to an edge site): No impact from a failure.

At the site with the in-service routing switch: The Call Agent remains active if it is already active. No outage occurs. If the Call Agent was not active, it takes activity within 2 seconds.

SOS goes through a warm or restart (approximately 20 seconds for the NTRX51HZ card with the MCPN905 processor and 30 seconds for the NTRX51GZ card with the MPCN765 processor). Other nodes go active and follow the Call Agent example after losing mate connectivity.

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

36 Disaster recovery fundamentals

Scenario

Response

If Site B has the in-service router, the standby CMT server must be brought into service. (See the following ATTENTION statement.) If Site A has the in-service router, the standby CBM must be brought into service. (See the following ATTENTION statement.)

ATTENTION
Consider using this option based on the estimated time to recover the fault or failure of the CS LAN versus time and effort to bring up the cold standby system and recover the HA pair afterward. For example, if the time to recover the CS LAN is estimated to be 10 hours, consider if it is worth activating the cold standby CMT server or CBM. See procedure Initiating a switchover to the remote backup server. Optical frame failure at one site At the site with the failed optical frame: All nodes lose mate connectivity by using Ethernet.

Call Agent FC or GigE call data link connectivity is lost. WAN backup remains up. The active Call Agent remains active, but without sync. No outage occurs. The remainder of this scenario assumes the active Call Agent is at this site. If not already active, expect other nodes co-located with the active Call Agent to become active after losing mate connectivity. If not already inactive, expect other nodes co-located with the inactive Call Agent to become inactive after losing mate connectivity. (Done without mate connectivity.) USP mated pair configurations: Mated pairs connected to pair of gateway POP Ethernet Routing Switch 8600s: No impact to both USPs from a failure. Mated pairs split across a CS-LAN site (each connected to the Ethernet Routing Switch 8600 of local CS 2000 CS-LAN): For CS2K-1, only the USP at the active side provides service. For CS2K-2, both USPs provide service, but over a packet network. There are no impacts to other CS 2000s. Mated pairs split across a CS-LAN site (each connected to one Ethernet Routing Switch 8600 connected to an edge site): No impact from a failure.

At the site with the in-service optical frame: The inactive Call Agent detects IST loss. For an Ethernet Routing Switch 8600 CS LAN using OSPF, the Call Agent
Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

Geographical survivability impacts to CS 2000-Compact

37

Scenario

Response disables OSPF. For an Ethernet Routing Switch 8600 CS LAN using Border Gateway Protocol (BGP) or for a third-party CS LAN using either OSPF or BGP, execute manual action to disable routing at the site with the inactive Call Agent. See section Disabling Border Gateway Protocol (BGP) for instructions of disabling and enabling BGP. See section Enabling protocols BGP and OSPF.

Other nodes become inactive and follow the Call Agent example after losing mate connectivity. (Done without mate connectivity.) USPc detects isolation from the active Call Agent, and takes down SS7 links at the site with the inactive USP. If Site B has the failed optical frame, the standby CBM must be brought into service at Site A. (See the following ATTENTION statement.) If Site A has the failed optical frame, the standby CMT must be brought into service at Site B. (See the following ATTENTION statement.)

ATTENTION
Consider using this option based on the estimated time to recover the fault or failure of the CS LAN versus time and effort to bring up the cold standby system and recover the HA pair afterward. For example, if the time to recover the CS LAN is estimated to be 10 hours, consider if it is worth activating the cold standby CMT server or CBM. See procedure Initiating a switchover to the remote backup server. One site is destroyed in a catastrophic event At the site that is destroyed: There is no activity.

The SS7 network takes down the links to the destroyed building. USP mated pair configurations: Mated pair connected to pair of gateway POP Ethernet Routing Switch 8600s: No impact occurs to either USP as they are located in different sites from a failure. Mated pair split across a CS-LAN site (each connected to the Ethernet Routing Switch 8600 of local CS 2000 CS-LAN): The SS7 network takes down all links to the destroyed site. Mated pair split across a CS-LAN site (each connected to one Ethernet Routing Switch 8600 connected to an edge site): If the USPs are located at the same site where the failure occurred, the SS7 network takes down all links at the

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

38 Disaster recovery fundamentals

Scenario

Response destroyed site. Otherwise, no impact occurs to either USP from the failure.

At the site that is not destroyed: All nodes lose mate connectivity. The Call Agent loses all mate connectivity, including the backup WAN link, and drops sync. If the Call Agent is active, it remains active. No outage occurs. If the Call Agent is not active, it takes activity within 2 seconds. If the Call Agent is inactive, it takes activity within 2 seconds followed by a warm or SOS restart (approximately 20 seconds for the NTRX51HZ card with the MCPN905 processor, and 30 seconds for the NTRX51GZ card with the MCPN765 processor). Other nodes go active, and follow the Call Agent example after losing mate connectivity. If Site B was destroyed, initiate a switchover to the standby CBM at Site A. (See procedure Initiating a switch over to the remote backup server. If Site A was destroyed, initiate a switchover to the standby CMT at Site B. (See procedure Initiating a switch over to the remote backup server.


Active Call Agent card fails at one site

At the site with the failed Call Agent: All nodes at the site can communicate with their mates. At the site with the mate Call Agent: All nodes in the site can communicate with their mates.

The (inactive) Call Agent detects loss of connectivity with the mate, detects local and WAN connectivity, takes activity, and restarts the SOS. Other nodes experience disconnection from the SOS for approximately 20 seconds for the NTRX51HZ card with the MCPN905 processor, and 30 seconds for the NTRX51GZ card with the MCPN765 processor (normal restart behavior)

Inactive Call Agent card fails at one site

At the site with the failed inactive Call Agent: All nodes in the site can communicate with their mates. At the site with the mate Call Agent:

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

Geographical survivability impacts to CS 2000-Compact

39

Scenario

Response


Recovery from isolation split brain (Active/Inactive)

All nodes in the site can communicate with their mates. The (active) Call Agent detects loss of connectivity with the mate, detects local and WAN connectivity, and stays active. No SWACT or restart is required.

At both sites: After the Call Agents can communicate with their mates, they recognize that both are active. The Call Agent that was inactive backs down, leaving the other Call Agent active. The fallout is to force Unit 0 active and Unit 1 inactive.

Other nodes can communicate with their mates, negotiate activity, and resume normal operations.

System impact of failures


Failures could cause the following system impacts:

When negotiating activity, the Call Agent preference is always to remain on the same side, if that side supports activity. If failover of the Call Agent is necessary, the Call Agent switches activity in less than 2 seconds. When the Callp Application performs a restart, call processing is interrupted for approximately 20 seconds for the NTRX51HZ card with the MCPN905 processor, and 30 seconds for the NTRX51GZ card with the MCPN765 processor (normal restart behavior). Failovers of other Callp nodes follow the Call Agent failover within 2 seconds. In Enterprise-only configurations where the solution contains Message Controller (MC) cards connected to Message Switches (MS), ENET and TDM peripherals, the MC and MS cards are co-located in one of the main geographically redundant sites. During site isolation, when determining the appropriate master site, preference is given to the site that contains the MC cards. This assumes that the site is able to take activity. If necessary, activity is switched to this side during the activity negotiation.

Recovery scenarios
The following table provides a summary of system responses during recovery.

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

40 Disaster recovery fundamentals Table 8 Recovery system response summary Scenario Recovery from isolation split brain (Active/Inactive) Response

When Call Agents can communicate with their mates, they recognize that both are active. The Call Agent that was inactive before the failure backs down, leaving the other Call Agent active. The fallout is to force Unit 0 active, and Unit 1 inactive. Other nodes can communicate with their mates, negotiate activity and resume normal operations.

General recovery behavior

All elements: continually monitor connections with their mates, and with other network elements with which they normally communicate. When connectivity is not present, they continue to monitor the connections for restored connectivity. (The elements continue monitoring regardless of their activity state.)

negotiate activity and services when connectivity recovers, and resume normal operations. USP mated pair configurations: Mated pair connected to pair of gateway POP Ethernet Routing Switch 8600s: No impact to USP mated pair. Mated pair split across a CS-LAN site (each connected to the Ethernet Routing Switch 8600 of local CS 2000 CS-LAN): USP mated pair does not negotiate activity during recovery. This is a result of USP mated pair operating in a load sharing mode; both are active during normal conditions. Mated pair split across a CS-LAN site (each connected to one Ethernet Routing Switch 8600 connected to an edge site): No impact from a failure.

System impact of recovery


Recovery could cause the following system impacts:

When negotiating activity, the Call Agent preference is always to leave activity on the same side. When recovering to a full system configuration, activity remains on the same unit, without impact. During recovery from a split system (caused by incorrect message routing), node activity resolves in a few seconds. Call processing could require up to 15 minutes to recover completely. If an inactive call agent is not in service when the IST goes down, OSPF on the Ethernet Routing Switch 8600 is not disabled on the site of the inactive call agent, potentially resulting in failed calls due to misdirected messages. This condition can occur on site recovery after

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

Protocols BGP and OSPF activation

41

the Ethernet Routing Switch 8600 returns to service and the optical ring is still recovering. In the event that this outage occurs, manually disable OSPF on the Ethernet Routing Switch 8600 or disconnect the links to the WAN on the site of the inactive call agent until it returns to its operational state.

Protocols BGP and OSPF activation


This section applies only to CS-LAN routers configured with Ethernet Routing Switch 8600s in which BGP is the default routing protocol. Protocols OSPF and BGP serve as an interdomain protocols to distribute routing information between endpoints in an Geographic Survivability configuration. Recall that the CS-LAN consists of dual Ethernet Routing Switch 8600s running as layer 2/3 switches. For redundancy, a pair of upstream routers must be deployed on the Core network edge as the entrance to the CS-LAN. Nortel supports a square CS-LAN topology configuration, in which each Ethernet Routing Switch 8600 is connected to one of two upstream routers. The following figure shows an example of the Ethernet Routing Switch 8600s configured for Geographic Survivability in a square topology.
Figure 4 Ethernet Routing Switch 8600s configuration

In this example, each Ethernet Routing Switch 8600 includes two BGP neighbors. One BGP neighbor is the eBGP peer to the wide area network (WAN) Core router. The other BGP neighbor is the iBGP peer to its Ethernet Routing Switch 8600 mate.

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

42 Disaster recovery fundamentals

The BGP functionality also supports the following features:

can be used with both MCPN 905 and MCPN 765 type Call Agent cards selection of either BGP or OSFP as the protocol during Geographic Survivability installation script setgeoconfig supports either BGP or OSPF input Call Control Agent (CCA) sends control commands (Disable/Enable BGP or OSPF) to the Ethernet Routing Switch 8600 based on protocol selection during installation Inactive CCA triggers command BGP disable command to the Ethernet Routing Switch 8600 when optical connectivity is lost between two sites. When recovering optical connectivity, the inactive CCA triggers command BGP enable to the Ethernet Routing Switch 8600. Existing routing protocol disable or enable event log is used with the corresponding OSPF or BGP protocol.

USP mated pairs


To support Geographic Survivability, a mated pairs of USPs can be configured with up to six CS 2000 - Compact call servers. Nortel supports the following USP mated pair configurations in a Geographic Survivability deployment:

mated pair of USPs connect to a pair of gateway POP Ethernet Routing Switch 8600s, which is co-located with the CS-LAN or in a remote site from the CS 2000 or CS 2000 - Compact mated pair of USPs are split across a CS-LAN site, whereby each USP connects to an Ethernet Routing Switch 8600 in a local CS-LAN configuration mated pair of USPs are split across a CS-LAN site and each connects to a single Ethernet Routing Switch 8600, which also is connected to an edge site

For details on mated USP pairs, see NTP Nortel Universal Signaling Point Configuration ( (NN10093-511)) .

Mated pair connected to a pair of gateway POP Ethernet Routing Switch 8600s
While configuration also applies to a non-Geographic Survivability environment, in a Geographic Survivability environment, the Cores can share the same optical ring providing the distance is close. The mated USP pair is co-located and connect to two Ethernet Routing Switch 8600s, which also connect to a POP gateway.

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

USP mated pairs

43

The following figure shows an example of this configuration.


Figure 5 Mated pairs connected to a pair of gateway POP Ethernet Routing Switch 8600s

The following conditions apply to this configuration:

Two RTC cards cross-connect to two Ethernet Routing Switch 8600s on each USP. Dedicated pairs of IPS7 cards serving as inter-USP mated links cross-connect to two Ethernet Routing Switch 8600s on each USP. The mated link transfers SSNM messages. (The mated link transfers traffic during some failure conditions.) Other IPS7 card pairs dedicated to CS 2000 Cores connect to local Ethernet Routing Switch 8600 on each USP. All the CS 2000s share SS7 cards or have dedicated SS7 cards. Only ABS stand-alone servers can be used. Two ABS servers are co-located with two USP pairs and connected to two Ethernet Routing Switch 8600s. Both USP pairs are connected to only one IEMS server.

Mated pair of USPs split across a CS-LAN site (each connected to the Ethernet Routing Switch 8600 in a local CS 2000 CS-LAN)
In this configuration, a mated pair of USPs are split across a CS-LAN site, whereby each USP connects to an Ethernet Routing Switch 8600 in a local CS-LAN configuration.

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

44 Disaster recovery fundamentals

The following figure shows an example of this configuration.


Figure 6 Mated USP pair split across a CS-LAN site (each connected to the Ethernet Routing Switch 8600 of local CS 2000 CS-LAN)

The following conditions apply to this configuration:

Two RTC cards connect to different Ethernet switch boards on each mated USP pair. Dedicated pairs of IPS7 cards serving as inter-USP mated links cross-connect to two Ethernet Routing Switch 8600s on each USP. The mated link transfers SSNM messages. (The mated link transfers traffic during some failure conditions.) Other IPS7 cards dedicated to CS 2000 Cores connect to local Ethernet Routing Switch 8600 on each USP. All the CS 2000s share SS7 cards or have dedicated SS7 cards. The CBM and CMT can be used as an ABS server for mated USP pairs. An ABS server is co-located with one half of CS2Kc-1. Two UPS pairs connect to one IEMS server.

Mated pair of USPs split across a CS-LAN site (each connected to a single Ethernet Routing Switch 8600)
In this configuration, a mated pair of USPs are split across a CS-LAN site. Each pair connects to a single Ethernet Routing Switch 8600, which also could be connected to an edge site.

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

USP mated pairs

45

The following figures show examples of this configuration. Both figures reflect connections for RTC and IPS7 cards.
Figure 7 Mated USP pair split across a CS-LAN site (each connected to a single Ethernet Routing Switch 8600)

The following figure shows the same configuration but with the Ethernet Routing Switch 8600 connected to an edge site.
Figure 8 Mated USP pair split across a CS-LAN site (each connected to a single Ethernet Routing Switch 8600, which connects to an edge site)

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

46 Disaster recovery fundamentals

VLAN assignment
All USP mated pair configurations support the following VLANs:

Both RTC cards of USP A are assigned to OAM_USP_A_VLAN. Both RTC cards of USP B are assigned to OAM_USP_B_VLAN. IPS7 cards serving as mated links on USP A are assigned to CallP_USP_A_Mate VLAN. IPS7 cards serving as mated links on USP B are assigned to CallP_USP_B_Mate VLAN. IPS7 cards dedicated to CS2Kc-1 on USP A are assigned to CallP_USP_A_CS2K1. IPS7 cards dedicated to CS2Kc-1 on USP B are assigned to CallP_USP_B_CS2K1. IPS7 cards dedicated to CS2Kc-2 on USP A are assigned to CallP_USP_A_CS2K2. IPS7 cards dedicated to CS2Kc-2 on USP B are assigned to CallP_USP_B_CS2K2.

Limitations and restrictions


The following limitations and restrictions apply to this USP configuration.

Customers must guarantee a secure connection between the USP and the remote CS 2000 - Compact. For USPs to communicate with remote CS 2000 - Compacts over the Core network, the CCA, 3PC or XA-Core HIOPs must be advertised in the carriers network, although they have been configured in the call processing private VLAN. To reach the USPs, the VLAN must be advertised outside of the Ethernet Routing Switch 8600 CS-LAN. Other devices that have been provisioned in the same VLAN (for example, STORM-IA) can be exposed. To reduce the risk of DoS attack, provide new filtering rules and new traffic limitation rules on the GW-POP. For example, the new filtering rules on the GW-POP must accept traffic to the USP only if it emanated from the GWC. If IPsec is enabled, apply the following filtering rules to implement the GW-POP firewall.

Discard all traffic to the CCA except traffic from the GWC subnets. Discard all non-ESP or non-IKE traffic (UDP traffic to and from port
500).

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

SSL geographic survivability

47

Ensure that USP mated pair-specific VLANs are not disabled during a ring outage because the USPs are not split across the ring as in other geographic-configured VLANs. Nortel recommends that mated USPs installations not exceed 120 kms from the CS-LAN.

SSL geographic survivability


Before CVM11, SSL components supported network interface failure, whereby they could detect when all network interfaces connecting them to the LAN had failed. When Session Server Lines (SSL) components detected a network isolation condition, they ensured that an isolated component did not remove activity from its mate when the isolation condition was repaired. This behavior mirrors fault tolerant behavior for a component that detects itself to be in the non-surviving site of a Geographic Survivable configuration. The affected components are completely isolated from the rest of the network while call processing continues. SSL components detect themselves network isolated under the following conditions:

the Geo mode is enabled in the CS 2000 Call Agent window no active CS 2000 Compact Call Agent exists when an SSL System Manager or Session Manager starts

The SSL component should detect the isolation before it attempts activation. The component shuts itself down and waits until there is an active CS 2000 Compact Call Agent present before it restarts itself.

Non-surviving site network interface isolation


SSL Geographic Survivability supports a geographic distribution of redundant SSL components across a second site to preserve call processing, billing and element management functions in case of a failure isolating one of the sites. In addition, call capacities do not degrade from the failure of one of the sites. Network requirements and facilities interconnecting the two sites is efficient and cost effective. SSL components that detect themselves to be in a non-surviving site of a Geographically Survivable configuration behave as if they do when they detect a network interface failure. That is, they consider themselves to be network isolated. This feature introduces a mechanism to detect the condition of a non-surviving site of a Geographically Survivable configuration. Detecting network isolation of a non-surviving site of a Geographically Survivable

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

48 Disaster recovery fundamentals

configuration is based on the reachability of the active IP address of the CS 2000 Call Agent. Internet Control Message Protocol (ICMP) echo request (ping) messages are sent to the active IP address of the CS 2000 Compact Call Agent. SSL components that detect this IP address "unreachable" consider themselves to be in the non-surviving site (that is, network isolated). See NTP SSL Configuration (NN10023-511) for details on how to enable or disable pinging on the active IP address of the CS 2000 Call Agent.

Network isolation detection for a non-active network element instance (NEI)


When a non-active NEI detects itself to be network isolated, it shuts itself down and waits for the network isolation condition to clear before restarting itself. This action avoids a dual-active scenario when a stand-by NEI detects itself to be network isolated (whereby its mate is active) before falsely detecting its active mate as failed. In this case, it also shuts itself down. The NEI restarts itself when the network isolation condition is cleared. This measure prevents the active mate from having to decide if it should cede activity.

Network isolation detection for an active NEI


An active NEI does not deactivate itself unless it detects that its mate is also active. Thus, an active NEI remains active when it detects itself to be network isolated.

SST geographic survivability


Geographic survivability is a mechanism that allows the continuation of service during and after a catastrophic event such as fire, flood, tropical storm, or an act of terrorism at a call server site. Thus, the key components of the system must be geographically distributed between locations that are some distance apart, interconnected over a fault-tolerant network. In the event that the components at one site are lost, the system will survive under the control of the components at the other site. In a geographically survivable configuration, the SST is configured across two sites. If the SST at one site loses connectivity, the SST at other site takes over the activity and is able to process the new calls. In a geographically survivable configuration

the deployment of split session-server pairs is supported the SST maintains 99.999 % availability for Callp

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

Geographic survivability limitations and restrictions

49

the time for fail-over is equivalent, on the same order of magnitude, to the fail-over time for a non-geographically survivable configuration stable calls and billing are maintained during a fail-over

Geographic survivability limitations and restrictions


The following limitations and restrictions apply to the Geographic Survivability feature:

The physical distance between active and standby sites is limited to 120 KM/75 miles. Each site must have a single Ethernet Routing Switch 8600 with dual switch fabric and CPU blades or third-party equivalent. Synchronization of data between two CCA blades over GigE links applies to the CS 2100 market with reduced capabilities. This configuration applies just to Enterprise solutions. In Enterprise hybrid configurations (with TDM equipment homed at one site), the TDM equipment is not geographically redundant. In determining the master site, preference is given to the TDM side only when either side can support Callp. If necessary, perform a SWACT to the TDM side to allow Callp on that side. Because the Call Agent interacts with the Ethernet Routing Switch 8600, the feature requires that each site have only one routing switch and IST links configured between sites. Dual Ethernet Routing Switch 8600s at each site and SMLT links between sites are not supported. Interactions between the Call Agent and the CS LAN are supported to prevent split brain scenarios (by disabling OSPF) when the Ethernet Routing Switch 8600s are used for the CS LAN. Upgrades from previous releases in an Enterprise geographic survivable configuration (which have dual Ethernet Routing Switch 8600s at each site) require that the dual Ethernet Routing Switch 8600s be migrated to a single Ethernet Routing Switch 8600 site. When a total loss of communication between sites occurs (that is, all three master links are down), the two Call Agents cannot negotiate activity decision. The decision is based on connectivity check from each site the WAN network.

While unlikely, it could be possible to have an active/active (split


brain) scenario, or an inactive/inactive scenario (no processing).

The WAN backup path mitigates the risk of optical ring failure. The
WAN connection check helps resolve activity when the backup path is down.

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

50 Disaster recovery fundamentals

CS 2000 - Compact supports only a single time zone setting. If the two physical sites are in different time zones, Nortel recommends that the time zone be set to either GMT or the time zone of one of the sites. Both Session Server units of a pair are located at the same site. For offices with Message Controllers, Nortel recommends that the Session Servers be located at the same site as the TDM components. For maximum redundancy, the WAN backup path must be configured separately from the optical network, as follows:

special vlans configured on Ethernet Routing Switch 8600 for


backup path use only

vlans route over the WAN network instead of over the optical ring vlans are not disabled with OSPF disable alarm generated for lack of connectivity

Gateways and services node components of CS 2000 - Compact are single units, and are not geographically redundant. Where the nodes are located and how they are connected to the network affects whether they survive a failure. Although the same nodes are supported in configurations with and without Geographic Survivability, there is no change in configuration or connection in the configuration with the Geographic Survivability configuration.

CAUTION
Line data integrity audit impacts the maintenance actions such as GWC status and GWC swact. At times, the bearer network cannot be retrieved from the core due to the bearer network timeout, as it is design intent.

Real Time Billing fundamentals


Real Time Billing (RTB) allows billing records to be available for transfer from the core manager 30 seconds after the time the billing records are generated. RTB downloads a small group of records to the DIRP billing file at the downstream destination as they are added to the open billing file on the core manager. RTB uses file transfer protocol (FTP) or OpenSSH for secure file transfer protocol (SFTP) through an Ethernet connection to deliver billing records.

ATTENTION
Outbound file transfer using FTP requires the downstream system to be running an FTP server that complies with RFC 959. In particular, for Real-Time Billing, the FTP server must support the "APPE" (Append) message.

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

Real Time Billing fundamentals

51

Terminology
To understand how the SBA processes and routes the billing records it receives for RTB, the following terminology must be understood:

Stream - A stream, or billing stream, can be conceptualized as a pipeline through which billing records received from the core pass. For each stream component that exists on the core, a corresponding stream component exists on the core manager. Billing records created by calls pass through the stream from their point of origination on the core to the core manager, where they are stored on disk. Sub-stream - A stream is further divided into Primary and Recovery sub-streams. The Primary sub-stream handles the current records being sent by the core. The Recovery sub-stream is only active after the SBA is unable to transfer records from the core to the core manager and temporarily stores the records on the core. When the core is once again able to re-establish the connection to the core manager, the stored records are sent to the core manager in a Recovery sub-stream while, concurrently, the current records are sent in the Primary sub-stream. Active file state - When records are written to a file that is open on the core manager, the file name on the core manager is prefixed with an "A", which means "active". When a billing files content is being written to a file on a downstream processor, the name of the file on the downstream processor is also prefixed with an "A". Unprocessed file state - After the file on the core manager receives all of its billing records, the file is closed and the name of the file is prefixed with a "U", which means "unprocessed". In the same manner, after the file content has been transferred to a downstream processor, the file receiving this content on the downstream processor is also prefixed with a "U". Processed file state - When a billing file on the core manager is closed and its content has been received by all designated downstream destinations, the file is then eligible for removal in order to free up disk space. The file name prefix then changes from "U" to "P", meaning "processed".

SBA le transfer subsystem


The SBA file transfer system uses a schedule tuple for scheduled file transfers. This schedule tuple is specified by stream name, file format, and destination. For each tuple, different file transfer parameters can be specified, such as start time, stop time, and file transfer interval. There can be only one tuple for each combination of stream, file format, and destination.

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

52 Disaster recovery fundamentals

The tuple contains a field indicating whether it is active. Scheduled file transfers occur only when the tuple is active. An interval setting in the schedule tuple determines how often SBA checks to see whether there are unprocessed files waiting to be sent downstream. When this interval is exceeded, the files are transferred downstream.

Real Time Billing le transfer


The RTB rts (return to service) command, which is issued from the billing maintenance interface (billmtc), is used to initiate the transfer of open billing files to the downstream customer site. The command specifies the stream, file format, and destination. RTB uses the appropriate fields in the schedule tuple corresponding to this stream. RTB attempts to transfer records to the active billing file at the primary destination IP address of the downstream destination specified in the schedule tuple. For the procedure used to perform this command, see "Returning RTB stream instance to service" in Nortel Core and Billing Manager 850 Accounting ( (NN10363-811)) . While RTB is transferring an open file, on the downstream processor the file name is prefixed with an "A" indicating an open, "active" file. When the file transfer is complete, the file prefix on the downstream processor is changed to a "U", the same file prefix used when scheduled file transfers succeed. When RTB is in service (InSV), the RTB Bsy (busy) command stops the current open file transfer by first closing the current open file on the core manager, sending the remainder of the file downstream, and then closing the FTP connection with the downstream processor. The procedure used for querying the current operational state of RTB is "Querying the status of RTB for a billing stream" in Nortel Core and Billing Manager 850 Accounting ( (NN10363-811)) . The schedule tuple must be active for a stream in order for the stream to be processed. When the two file transfer applications, scheduled transfer and Real Time Billing, are configured both must acknowledge an unprocessed file ("U" file prefix) before the file can become processed ("P" file prefix). Thus, after RTB transfers a file, the file state will remain "unprocessed" until the next scheduled transfer event. When that transfer event occurs, the scheduler examines all unprocessed files and treats them according to whether they have already been transferred by RTB. The files that have not been transferred by RTB are transferred and moved to the "processed" file state after a successful transfer. The files that have been transferred by RTB are moved directly to the "processed" file state without retransmission. Nortel suggests to set the Outbound File Transfer active status to Yes when configuring Real Time Billing.
Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

Real Time Billing fundamentals

53

Connection management
In normal operation, open files transferred by Real Time Billing are sent only to the Primary IP destination specified in the schedule tuple for each destination. If a problem occurs with that destination and open file transfer fails, the current file is closed. RTB will be tried again on the next open files based on the RTB MIB value RTBMaxConsecutiveFailures. After all file transfers allowed by the RTBMaxConsecutiveFailures RTB MIB value have been attempted, a critical alarm is raised, a log is issued, and RTB is moved to the SYSB state. In this state, open file transfer is not active. The retry behavior of RTB differs from that of a scheduled transfer. In the case of a scheduled transfer the primary address is tried first, and if it fails, attempts to re-transmit the file are repeated until the number of retries is exhausted. The retry attempts alternate between the primary and alternate destinations indicated in the schedule tuple. In the case of an RTB transfer, however, RTB will not attempt to re-transmit the file since that impacts the ability to send current records. Thus, RTB closes the file and retries transfer on the next file opened. In addition, unlike scheduled transfer, RTB only uses the primary destination. When RTB closes billing files, it cannot send the billing files downstream. The billing files are, however, automatically transferred from the core manager during the next scheduled transfer when the schedule tuple is active. The billing files can also be transferred manually. For the procedure used to transfer the billing files manually, see "Sending billing files from disk" in Nortel Core and Billing Manager 850 Accounting ( (NN10363-811)) . Manual intervention is required to restore RTB when it is in the SYSB state. The problem can often be attributed to a network connection that is no longer functioning properly. The RTB IPTest command can be used to "ping" the primary downstream address indicated in the schedule tuple to determine this. A SYSB state may also occur if the protocol has been changed in the schedule tuple to something other than the required protocol for RTB. When the problem that forced RTB into the SYSB state is resolved, the RTB Bsy and Rts commands can be used to bring RTB into service.

Alarms
For information about RTB alarms, refer to Nortel Core and Billing Manager 850 Fault Management ( (NN10351-911)) .

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

54 Disaster recovery fundamentals

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

55

Disaster recovery conguration


This chapter provides the tasks for disaster recovery for the Nortel Carrier VoIP solutions.

Disaster recovery conguration tasks


This work flow shows you the sequence of tasks you perform to configure an end-to-end disaster recovery. To link to any task, go to Disaster recovery configuration task navigation (page 56).

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

56 Disaster recovery configuration Figure 9 Disaster recovery configuration tasks

Disaster recovery conguration task navigation Disaster recovery configuration (page 55) Preparation for disaster recovery (page 57) Network power down (page 331) Network power up (page 375) Disaster recovery power up (page 455) Device audits (page 473)

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

57

Preparation for disaster recovery


This chapter provides the procedures for preparing for disaster recovery of the Nortel Carrier VoIP solutions. Navigation

Recovering the SPFS system if updisk daemon dies (page 58) Installing optional software on a Core and Billing Manager (page 64) Installing the Passport Log Streamer application (page 71) Installing the DDMS (page 59) Installing the OM Data Delivery application (page 68) Installing the SBA and AFT software packages (page 73) Installing the GR740PT application server (page 74)

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

58 Preparation for disaster recovery

Recovering the SPFS system if updisk daemon dies


Use this procedure if an updisk daemon fails or one of the datasets goes down on either the active or inactive node.

Procedure steps
On the node displaying the updisk daemon or dataset failure

Step 1 2

Action Enter StopCluster Enter StartCluster


--End--

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

Procedure steps

59

Installing the DDMS


Install the DDMS application. After Enhanced Password Control is in effect on the CM or Core, the DDMS software has the ability to manage automatic password changing on the CBM and the CM, before passwords expire. It is not necessary to manually change any of the passwords for the SDM01-SDM04 user IDs on the CBM or CM. After the DDMS software is returned to service, it reads the tables, ofcopt and ofceng, on the CM to determine whether Enhanced Password Control is in effect. If Enhanced Password Control is in effect, the DDMS software reads the password lifetime value and automatically changes the passwords one day before they expire.

Prerequisites

For a successful installation of DDMS, verify that the Log Delivery Service application is in service. If you make manual changes to the password lifetime value, or if you turn the Enhanced Password Control off or on, you must synchronize these changes with the DDMS software by performing a bsy or rts of the DDMS application. If you change any of the SDM01-SDM04 passwords manually, you must apply the same password changes in the DDMS configuration file.

Procedure steps
Step 1 Action At your workstation, at the CLI prompt on the core, configure the Enhanced Password Control for the SDM01 user ID:
permit sdm01 <sdm01_pswd> 4 10000 english all

ATTENTION
If Enhanced Password Control is in effect on the CM, the password must be at least six characters in length.

2 3 4

Configure Enhanced Password Control for the SDM02 user ID:


permit sdm02 <sdm02_pswd> 4 10000 english all

Configure Enhanced Password Control for the SDM03 user ID:


permit sdm03 <sdm03_pswd> 4 10000 english all

Set Enhanced Password Control for the SDM04 user ID:


permit sdm04 <sdm04_pswd> 4 10000 english all

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

60 Preparation for disaster recovery

5 6

Create the first required logical volume for the DDMS application:
makelv /cbmdata/00/osscomms 16

Set the access privileges of the first logical volume for the DDMS application:
chmod 755 /cbmdata/00/osscomms

Set the ownership privileges of the first logical volume for the DDMS application:
chown maint:maint /cbmdata/00/osscomms

Create the second required logical volume for the DDMS application:
makelv /cbmdata/00/ossaps 112

Set the access privileges of the second logical volume for the DDMS application:
chmod 755 /cbmdata/00/ossaps

10

Set the ownership privileges of the second logical volume for the DDMS application:
chown maint:maint /cbmdata/00/ossaps

11

Create the third required logical volume for the DDMS application:
makelv /cbmdata/00/ossapslog 112

12

Set the access privileges of the third logical volume for the DDMS application:
chmod 755 /cbmdata/00/ossapslog

13

Set the ownership privileges of the third logical volume for the DDMS application:
chown maint:maint /cbmdata/00/ossapslog

14

Apply the two software application packages, OSS and Application Svcs and OSS Comms Svcs . Specify /cdrom/cdrom/applications/cbm/packages as the source directory when you perform that procedure. You are prompted automatically to configure the OSS Comms Svcs package.

15

ATTENTION
The OSS and Application Svcs package does not require configuration.

16

If you are prompted automatically to configure the OSS Comms Svcs package then go to 29 OR

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

Procedure steps

61

If you are not prompted automatically to configure the OSS Comms Svcs continue with the next step. 17 18 Access the configuration level of the maintenance interface:
cbmmtc config

In the list of applications, locate the OSS Comms Svcs application and record its application number (located next to the names of the applications). Select the application:
select <application number>

In response to the command, the OSS Comms Svcs application is highlighted on the cbmmtc config screen. 19 20 Invoke the configuration of the OSS Comms Svcs application:
config

When prompted to enter the logroute tool, as shown in the following figure, press Enter:
Figure 10 DDMS logroute tool banner

The Logroute Main Menu appears. 21 Select the Device List menu
1

The Device List Menu screen appears. 22 23 Select 1 to display the Device List screen. If the list includes device /cbmdata/00/logs/ossaps/ossapslog then continue with the next step. OR If the list does not include device /cbmdata/00/logs/ossaps/ossap slog skip the next step. 24 25 26 Press the Enter key. . Begin to add a new device:
2

Select a file device:

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

62 Preparation for disaster recovery 3

Example Response:
Enter file name ==> /data/logs/

27

Complete the path name:


ossaps/ossapslog

Routing for the DDMS is now set up. 28 29 30 31 32 33 When prompted, enter STD log format (from the range displayed). When prompted, set the ECORE option to ON. Select addrep:
a

Enter the log identifier:


DDMS

When prompted to enter more log routing details, enter


N

Save the new device:


y

Response:
Save completed -- press return to continue

34 35 36 37

Press the Enter key and return to the Add Device screen. Return to the Device List Menu screen:
5

Return to the main menu screen:


6

Exit logroute:
6

The CM User Setup screen appears and the required CM users, SDM01-SDM04, for DDMS are added to the DDMS configuration file. The passwords for these users are the same as those entered in step 1.

ATTENTION
The user IDs and passwords are not case sensitive. You can change them after this installation is complete.

38

Add a new user for each of the required user IDs:


1

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

Variable definitions

63

39 40

When prompted, enter the user name (for example, sdm01). When prompted enter the password.

ATTENTION
The first entry of a user name and password generates the following message: Error: file not valid. You can ignore this message.

41 42

If applicable, continue to add other user names and passwords at the prompt, otherwise continue with the next step. Exit the CM User Setup screen:
0

The DDMS Clients Configuration screen appears. 43 Add a new DDMS client.
1

ATTENTION
The DDMS clients are the CS 2000 Management Tools servers with the SESM load.

44

When prompted, enter the IP address for each of the CS 2000 Management Tools servers and press the Enter key, after each entry. If the CS 2000 Management Tools server is a cluster configuration, add the IP address of both the active and inactive units. After you have entered all the IP addresses, type
done

45 46

Exit the DDMS clients configuration screen:


0
--End--

Variable denitions
Variable <sdm0n_pswd> <application number> Value The CM password for user (SDM01,SDM02,SD M03,SDM04) The number associated with the OSS Comms Svcs application, that you noted

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

64 Preparation for disaster recovery

Installing optional software on a Core and Billing Manager


Install optional software on a Core and Billing Manager (CBM). Refer to the Job aid (page 66) for a listing of the optional software packages (filesets) that can be installed with this procedure.

ATTENTION
This procedure must be performed on a pre-cloned system. If the procedure is not performed on a pre-cloned system, clone the image of the active node to the inactive node of the cluster after the software package has been installed and configured, and after the active node has been made patch-current.

Procedure steps
At your workstation

Step 1

Action Open a connection to the active node of the CBM using SSH and log on as the root user: ssh -l root <ip_address> Enter the password for the root user. To install the DDMS application, perform steps 1 and 5 only in Installing the DDMS (page 59). Go to step 18. To install the OMDD application then perform step 1 only, in Installing the OM Data Delivery application (page 68). Go to step 18. To install the log delivery service application, perform Installing the Passport Log Streamer application (page 71). Go to step 24. To install the SBA or AFT applications, perform Installing the SBA and AFT software packages (page 73) . Go to step 24. To install the GR740PT application server, perform Installing the GR740PT application server (page 74). Go to step 24. To install the FTP proxy application, refer to Nortel Core and Billing Manager 850 Administration and Security ( (NN10358-611)) .

2 3 4 5 6 7 8 9 10 11 12 13

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

Variable definitions

65

14 15

Go to step 17. To install the Backup Restore Manager software, perform Installing the Backup Restore Manager software (page 94), and then go to step 17. Go to step 24. To apply any other optional software application, perform Applying software packages on a Core and Billing Manager using the CBMMTC interface (page 88). If required, return to step 3 to continue applying new software applications. Ensure that you create all required logical volumes for the applications that you are install before you continue. If you created any logical volumes, reboot the CBM:
init 6

16 17

18 19 20 21

If required, complete installation of the DDMS application by performing the remaining steps of Installing the DDMS (page 59), starting with step 14. Go to step 24. If required, complete the installation of the OMDD application by performing the remaining steps of Installing the OM Data Delivery application (page 68), starting with step 2. Ensure that your CBMs are patch-current. For patching procedures, see Nortel ATM/IP Solution-level Administration and Security ( (NN10402-600)) . Clone the active node image to the inactive node by performing the procedure, Cloning the image of the active node to the inactive node (page 91). If applicable, return to the higher level task flow or procedure that directed you to this procedure. This procedure is complete.
--End--

22 23

24

25

26 27

Variable denitions
Variable <ip_address> Value IP address of the active node of the CBM cluster.

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

66 Preparation for disaster recovery

Job aid
The following table lists filesets (applications) that the CBM0090 load includes. The table shows which filesets are included with the CBM at the time of installation (Base) and which filesets are optional that you can install.
Table 9 Filesets available for the Core and Billing Manager Fileset SDM_BASE.version_20.81.0.0 CBM_SETUP NT_SIM.tools SDM_ACE SDM_AFT.DMS500 SDM_BASE.base SDM_BASE.comm SDM_BASE.gdd SDM_BASE.logs.client SDM_BASE.logs SDM_BASE.mtce SDM_BASE.omsl SDM_BASE.tasl SDM_BMI.bmi SDM_DDMS_ossaps SDM_DDMS_osscomms SDM_BASE.util SDM_DEBUG.tools SDM_DMA.dma SDM_FTP.proxy SDM_GR740PT.gr740pt SDM_LOGS.mdm SDM_OMDD.omdd SDM_REACHTHRU.rttl1 SDM_SBA.DMS500 Description Load lineup information CBM installation and upgrade tool; available only on CD Patching tools SDM ACE distribution SBA automatic file transfer Platform base Platform maintenance common Generic data delivery Log Delivery Service client Log Delivery Service Platform maintenance OM Access Service Table Access Service Base Maintenance Interface OSS and Application Svcs OSS Comms Svcs Platform Utilities SDM/CBM Debug Helper Tools DMS Maintenance Application FTP Proxy GR740 Pass Through Passport Log Streamer OM Delivery Reach Through SPM SDM Billing Application Type Base Base Base Optional Optional Base Base Base Optional Base Base Base Base Optional Optional Optional Base Base Optional Optional Optional Optional Optional Optional Optional

ATTENTION
Base = included with the CBM

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

Job aid

67

Table 9 Filesets available for the Core and Billing Manager (contd.) Fileset SDM_SCFT.scft SDM_CLP.clp SDM_CLP.clp NTbkupmgr NTdtsv NTprxy NTsaf NTbulk Description SSH Core File Transfer CBM LI Proxy CBM LI Proxy Succession Provisioning Data Synch Manager CEM DMS Data Server CEM Telnet Ftp Handler CEM Store and Forward CEM Bulk Formatter Type Optional Optional Optional Optional Optional Optional Optional Base

ATTENTION
Base = included with the CBM

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

68 Preparation for disaster recovery

Installing the OM Data Delivery application


Use this procedure to install and configure the OM Data Delivery application on the CBM cluster.

Prerequisites

OM Access Service and Table Access Service application filesets must be installed and in service on the core manager. For the wireless market, the Nortel support group must increase the buffer size within the OM Access Service to 2.5 MB to accommodate the amount of data transferred by the front end for a transfer period of every 30 minutes.

Procedure steps
At your workstation

Step 1

Action Create the following logical volume (directory for a file system) required for the OMDD software package you are installing:
makelv /cbmdata/00/omdata 1008

Apply the SDM_OMDD.omdd software package located in the /cdrom/cdrom/applications/cbm/packages directory (refer to Applying software packages on a Core and Billing Manager using the CBMMTC interface for further information).

ATTENTION
You must specify the CD-ROM directory /cdrom/cdrom/application s/cbm/packages as the source directory path when performing the applying software packages procedure.

3 4

Enter cbmmtc config to access CBM maintenance interface config: Enter config <n> to configure OM Data Delivery The system indicates that the Tuple Number option is inactive and prompts you to determine whether you want to activate it.

ATTENTION
The Tuple Number option allows you to activate or disable a tuple number so that it can be included in a CSV file with other OM information.

Enter y to activate or n to disable the tuple number. The system prompts you to confirm whether the Multiservice Data Manager (MDM) and core manager are integrated.

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

Procedure steps

69

To indicate that the MDM is connected to the core manager for collecting Passport 15000 performance measurement data, type
y

7 8 9 10 11 12

When prompted, enter the IP address of the first MDM you want to connect. When prompted, enter the hostname of the first MDM. When prompted, enter the IP address of the second (alternate) MDM you want to connect. When prompted, enter the hostname of the second MDM. When prompted, enter the port for 5-minute performance PM (performance measurement) data. The default port is 1646. When prompted, enter the port for 30-minute PM data. The default port is 1647. You are prompted as to whether you want to use custom connection retry settings. In case of connection failure, OMDD will try connecting to the MDMs, alternatively.

13 14 15 16 17

Enter y to use custom retry settings. Go to step 16. Enter n to use the default retry settings. Go to step 19. To configure the custom retry settings, enter a numeric value (in seconds) for the first connection retry interval.

ATTENTION
Values higher than 300 seconds are not recommended as they can adversely affect recovery time.

18 19 20 21 22 23 24

Enter the number of retry attempts for the first retry interval. Enter a numeric value (in seconds) for the second connection retry interval. Enter the number of retry attempts for the second connection retry interval. Enter a numeric value (in seconds) for the third connection retry interval. To confirm (save) the configuration data, entery, otherwise entern to reenter all of the configuration data. The system indicates that the configuration is complete. Press the Enter key. The system indicates that the changes will take place after the OM Data Delivery application is restarted. Press the Enter key to restart the OM Data Delivery application.

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

70 Preparation for disaster recovery

25 26

Exit the maintenance interface by typing:


quit all

This procedure is complete.


--End--

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

Procedure steps

71

Installing the Passport Log Streamer application


Use this procedure to install and configure the Passport Log Streamer application on the CBM cluster. For full operation, the log delivery application requires installation of the following application filesets:

Log delivery service (base software) Log delivery service client (optional software) Generic Data Delivery (base software) Passport Log Streamer, if the core manager needs to communicate with the Multiservice Data Manager (MDM) for fault data. (optional software)

Prerequisites

Ensure that no disk faults exist on the core manager. Remove any restrictions for the configured MDM ports from all firewalls between the MDM and the CBM to ensure that the Passport Log Streamer can communicate with the configured MDMs and collect logs.

Procedure steps
At your workstation

Step 1

Action Perform the procedure to apply the SDM_LOGS.m dm_21.39.9.0.pkg software package located in the /cdrom/cdrom/applications/cbm/packages directory (refer to Applying software packages on a Core and Billing Manager using the CBMMTC interface for further information). Because you use the CD-ROM to install the application, specify /cdrom/cdrom/applications/cbm/packages as the directory path of the source directory when you perform that procedure. When prompted, enter the IP address for the first MDM node. When prompted, enter the IP address for the second MDM node. When prompted, enter the port number configured for the pserver application on the first MDM node. When prompted, enter the port number configured for the pserver application on the second MDM node.

2 3 4 5

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

72 Preparation for disaster recovery

6 7 8 9 10

When prompted, type n to indicate that you do not want to receive MDM logs. When prompted, type n to indicate that you do not want to specify Passport 15000 log filters. When prompted, type n to indicate that you do not want to specify Ethernet Routing Switch 8600 log filters. When prompted, enter y to confirm the configuration data. Place the log delivery service application and the Passport Log Streamer application into service by accessing the Application (Appl) level of the CBM maintenance interface:
appl

11 12

Busy the application filesets: bsy <fileset_number> <fileset_number> Return the application filesets to service: rts <fileset_number> <fileset_number> After the application filesets are in service, the system retrieves any current log records. To view or store the log records, see Nortel Core and Billing Manager 850 Fault Management ( (NN10351-911)) .

13 14

Exit the CBM maintenance interface:


quit all

This procedure is complete.


--End--

Variable denitions
Variable <fileset_number> Value The number next to each of the application filesets.

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

Procedure steps

73

Installing the SBA and AFT software packages


Use this procedure to install the SuperNode Billing Application (SBA) and Automatic File Transfer (AFT) software packages on the CBM cluster.

Procedure steps
At your workstations

Step 1

Action Using the procedure Applying software packages on a Core and Billing Manager using the CBMMTC interface (page 88), apply the SBA and AFT software packages located in the /cdrom/cdrom/applications/cbm/packages directory. Create the necessary logical volumes (directories for file systems) required for the SBA using procedure "Adding a logical volume through the command line" in Nortel Core and Billing Manager 850 Accounting ( (NN10363-811)) . Configure the SBA for operation (refer to Nortel Core and Billing Manager 850 Accounting ( (NN10363-811)) for the appropriate procedures). Configure AFT for operation (refer to Nortel Core and Billing Manager 850 Accounting ( (NN10363-811)) . for the appropriate procedures). This procedure is complete.
--End--

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

74 Preparation for disaster recovery

Installing the GR740PT application server


Use this procedure install the GR740PT application server.

Prerequisites

The settings for office parameters eadas_dc_interface and eadas_nm_interface in table OFCVAR, and the settings for the EADAS SOCs (OAM00005 and OAM00006) are correct for your configuration. OAM00004 for EADAS/DC is ON and that office parameters eadas_mpc_and_link and netminder_mpc_and_link are appropriately datafilled in table OFCVAR when BX25 connectivity is required.

Procedure steps
At your workstation

Step 1

Action Using the procedure Applying software packages on a Core and Billing Manager using the CBMMTC interface (page 88), apply the GR740PT software package located in the /cdrom/cdrom/applications/cbm/packages directory. You are prompted to enter the mode of security. If you are not configuring GR740PT in non-secure mode do select 1. Non-secure. If you are configuring GR740PT in local secure mode do select 2. Local (SSH) security. This procedure is complete.
--End--

2 3 4

Job aid
The following table lists the supported configurations for EADAS GR740PT application server:
Table 10 CM EADAS TCP/IP configurations Supported configuratio ns DC and NM over BX25 DC and NM over TCP/IP Setting for eadas _dc_interface X25 TCP_IP Setting for eadas_ nm_interface N/A N/A SOC OAM0 0005 ON ON SOC OAM00 006 IDLE IDLE

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

Job aid

75

Table 10 CM EADAS TCP/IP configurations (contd.) Supported configuratio ns DC and Netminder over BX25 DC over BX25 and Netminder over TCP/IP DC over TCP/IP and Netminder over BX25 DC and Netminder over TCP/IP Setting for eadas _dc_interface X25 X25 TCP_IP TCP_IP Setting for eadas_ nm_interface X25 TCP_IP X25 TCP_IP SOC OAM0 0005 IDLE IDLE IDLE IDLE SOC OAM00 006 ON ON ON ON

The following table lists the channel assignments for EADAS. Note that DC EADAS channels 1, 2, and 3 support TR-740/746 compliant header and message. NM EADAS channels 1, 2, and 3 support SR3942 and TR746 to Netminder:
Table 11 EADAS channel assignments Description DC EADAS lc 1 DC EADAS lc 2 DC EADAS lc 3 NM EADAS lc 1 NM EADAS lc 2 NM EADAS lc 3 Service name DC_EADAS_LOG_CHAN1 DC_EADAS_LOG_CHAN2 DC_EADAS_LOG_CHAN3 NM_EADAS_LOG_CHAN1 NM_EADAS_LOG_CHAN2 NM_EADAS_LOG_CHAN3 TCP port 9550 9551 9552 9553 9554 9555 MTS offset 234 235 236 237 238 239

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

76 Preparation for disaster recovery

Conguring RTB for a billing stream


Use this procedure to perform the following real time billing (RTB) functions:

Prerequisites

add RTB to a billing stream change the RTB configuration for a billing stream delete RTB from a billing stream

This procedure has the following prerequisites:

Configure the billing stream. Perform the procedure "Configuring a billing stream on the core manager" in Nortel Core and Billing Manager 850 Accounting ( (NN10363-811)) . RTB requires outbound file transfer and DIRP file format. Configure outbound file transfer for the stream. Perform the procedure Configuring the outbound file transfer schedule" in Nortel Core and Billing Manager 850 Accounting ( (NN10363-811)) . RTB supports Real-time File Transfer Protocol Wrapper (RFTPW), Real-time Key-based Secure File Transfer (RKSFT), and Real-time Secure File Transfer Protocol (RSFTP) protocols.

This procedure requires the following information:

maximum number of retry attempts after RTB fails to transfer a billing file before RTB raises a critical alarm directory location on the data processing and management system (downstream OSS) of the RTB test file and partial file

You must be a user in a role group authorized to perform accounting-mana ge actions.

Procedure steps
ATTENTION
Instructions for entering commands in the following procedure do not show the prompting symbol, such as #, >, or $, displayed by the system through a GUI or on a command line. Conguring RTB for a billing stream

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

Procedure steps At any workstation or console

77

Step 1

Action Log into the core manager as a user authorized to perform accounting-manage actions (refer to Nortel CS 2000 Core Manager Administration and Security ( (NN10170-611)) for information on how to log on to the core manager). Access the BILLMTC interface:
billmtc

2 3 4

Access the schedule level:


schedule

Display and verify the schedule tuple: list <stream_name> Verify the following fields:

File_Format_Type: DIRP Protocol: RFTPW Active: No

Before configuring RTB ensure that the fields contain the values shown in this list. For secure file transfer, you may choose SFTPW or RSFTP (password-based user authentication) or RKSFT (key-based user authentication). 5 6 7 8 9 10 11 Access the RTB level:
rtb

Access the CONFRTB level:


confrtb

To add RTB to a billing stream, go to step 10. To change the RTB configuration for a billing stream, go to step 21. To delete RTB from a billing stream, go to step 33. Add RTB to a billing stream: add <stream_name> <file_format> <destination>

ATTENTION
If auto recovery is turned on, do not configure multiple RTB destinations with the same Test File Location or Partial File Location on the DPMS.

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

78 Preparation for disaster recovery

Enter the desired maximum retry attempts before RTB raises a critical alarm, and press the Enter key. The default value is 3. 12 Enter the directory on the DPMS where the RTB test file will reside and press the Enter key. The default directory is the Remote_Storage_Directory as configured in the Schedule tuple for this stream. 13 Enter the directory on the DPMS where the RTB remote partial file resides, and press the Enter key. The default directory is the Remote_Storage_Directory as configured in the Schedule tuple for this stream. Example response:
You entered: RTB Max Consecutive Failures: 5 RTB Remote Test File Location: /sba/autorec RTB Partial File Location: /sba/autorec Commit? [Save] {Save Edit Abort}:

14 15 16 17 18

If the displayed values are not correct, go to step 16. If the displayed values are correct, go to step 17. Edit and correct the displayed values:
edit

Save the information you entered:


save

Activate the schedule tuple for the stream by performing "Configuring the outbound file transfer schedule" in Nortel Core and Billing Manager 850 Accounting ( (NN10363-811)) . To add RTB to another billing stream, go to step 10. If you do not want to add RTB to another billing stream, go to step 37. Before performing this step, ensure the RTB schedule tuple is in the OFFL state. Change the RTB configuration for a billing stream: change <stream_name> <file_format> <destination>

19 20 21

22

ATTENTION
If auto recovery is turned on, do not configure multiple RTB destinations with the same Test File Location or Partial File Location on the DPMS.

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

Procedure steps

79

Enter the desired maximum retry attempts before RTB raises a critical alarm, and press the Enter key. The default value is 3. 23 Enter the directory on the DPMS where the RTB test file resides, and press the Enter key. The default directory is the Remote_Storage_Directory as configured in the Schedule tuple for this stream. 24 Enter the directory on the DPMS where the RTB remote partial file resides, and press the Enter key. The default directory is the Remote_Storage_Directory as configured in the Schedule tuple for this stream. Example response:
You entered: RTB Max Consecutive Failures: 5 RTB Remote Test File Location: /sba/autorec RTB Partial File Location: /sba/autorec Commit? [Save] {Save Edit Abort}:

25 26 27 28 29

If the displayed values are not correct, go to step 27. If the displayed values are correct, go to step 28. Edit and correct the displayed values:
edit

Save the information you entered:


save

Activate the schedule tuple for the stream by performing "Configuring the outbound file transfer schedule" in Nortel Core and Billing Manager 850 Accounting ( (NN10363-811)) . To change the RTB configuration on another billing stream, go to step 21. If you do not want to change the RTB configuration on another billing stream, go to step 37. Deactivate the schedule tuple for the stream by performing "Configuring the outbound file transfer schedule" in Nortel Core and Billing Manager 850 Accounting ( (NN10363-811)) . Before performing this step, ensure the RTB schedule tuple is in the OFFL state. Delete the RTB configuration from a billing stream: delete <stream_name> <file_format> <destination>

30 31 32

33

34

Confirm the delete command:


y

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

80 Preparation for disaster recovery

35 36

To delete RTB from another billing stream, go to step 33. Quit the BILLMTC interface:
quit all
--End--

Variable denitions
Variable <stream_name> <file_format> <destination> Value Specifies the name of the configured billing stream Specifies the file format of the configured billing stream Specifies the billing file transfer destination

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

SBRM core access configuration procedures

81

SBRM core access


This chapter shows a task flow and the procedures required to configure access to the core for the Synchronous Backup Restore Manager (SBRM).

SBRM core access conguration procedures


This task flow shows you the sequence of procedures you perform to configure core access for the Synchronous Backup Restore Manager (SBRM). To link to any procedure, click on the procedure in the navigation list.
Figure 11 SBRM core access configuration procedures

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

82 Preparation for disaster recovery

Conguration of SBRM core access navigation

Configuring the bkmgrusr user ID and password to enable communication between the DBRM and SBRM (page 82) Configuring core access for SBRM through the Nortel Communication Server 2000 Core Manager (page 84)

Conguring the bkmgrusr user ID and password to enable communication between the DBRM and SBRM
Use this procedure to configure the bkmgrusr user ID and password in order for the Synchronous Backup Restore Manager (SBRM) to communicate with the Device Backup Restore Manager (DBRM) on the Communication Server 2000 Core Manager. This procedure applies only to the Communication Server 2000 Core Manager running on an AIX platform. The procedure does not apply to the Core and Billing Manager (CBM) running on an SPFS-based server. Prerequisites

You must be a user authorized to perform config-admin actions. For information on how to log on to the Communication Server 2000 Core Manager and how to display actions a user is authorized to perform, see Logging in to the CS 2000 Core Manager and Displaying actions a user is authorized to perform in Nortel CS 2000 Core Manager Administration and Security ( (NN10170-611)) .

Procedure steps Step 1 Action At your workstation, log onto the Communication Server 2000 Core Manager on which the DBRM is installed as a user authorized to perform config-admin actions. Create the bkmgrusr user, type mkuser bkmgrusr Create these groups: emsmtc, emsadm, and emsrw, type mkgroup emsmtc mkgroup emsadm mkgroup emsrw Add the bkmgrusr user to the primary group (maint) and to secondary groups, type (on one single line) chuser pgrp=maint groups=emsmtc,emsadm,emsrw

2 3

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

Configuring the bkmgrusr user ID and password to enable communication between the DBRM and SBRM 83

home=/export/home/bkmgrusr admin=true shell=/bin/ksh bkmgrusr When you type this command, make sure that you typed a space between emsrw and home, and between ksh and bkmgrusr. 5 Confirm that the bkmgrusr user was added to the required groups, type groups bkmgrusr The system displays the groups to which bkmgrusr user belongs. 6 Set the password for the bkmgrusr user, type passwd bkmgrusr The bkmgrusr user is disabled until this step is performed. 7 8 Log off from the Communication Server 2000 Core Manager. Log on to Communication Server 2000 Core Manager as the bkmgrusr. When prompted, change the password for the bkmgrusr user. 9 Change to the home directory and create the .ssh directory, type cd /export/home/bkmgrusr mkdir .ssh chmod 700 .ssh
--End--

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

84 Preparation for disaster recovery

Conguring core access for SBRM through the Nortel Communication Server 2000 Core Manager
Use this procedure to configure access to the core for the Synchronous Backup Restore Manager (SBRM). You must perform this procedure to enable the SBRM to automatically backup a core image. Perform this procedure every time that core user password expires or changes. This ensures that the password you set in this procedure matches that set for the user on the core.

Prerequisites

Perform Configuring the bkmgrusr user ID and password to enable communication between the DBRM and SBRM (page 82) before you perform this procedure for the first time. You must be a user authorized to perform config-admin actions. You must know the user name and password that were configured to enable core access in Configuring the bkmgrusr user ID and password to enable communication between the DBRM and SBRM (page 82). For information on how to log on to the Communication Server 2000 Core Manager and how to display actions a user is authorized to perform, see Logging in to the CS 2000 Core Manager and Displaying information about a user or role group in Nortel CS 2000 Core Manager Administration and Security ( (NN10170-611)) .

Procedure steps
Step 1 Action At the Nortel Communication Server 2000 Core Manager, log onto the core manager with the logon ID and password for a user authorized to perform config-admin actions. If you want to perform this procedure through the command line, at the command line prompt, change the directory to the directory containing appropriate configuration script, type cd /opt/nortel/bkresmgr/cbm/scriptssee If you want to perform this procedure through SDMMTC (SDM maintenance interface), run the configuration script, type ./bkmgr_config.sh then go to step 5. 3 Access the config level of the SDM maintenance interface, type # sdmmtc config

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

Procedure steps

85

From the list of filesets, select the Succession Provisioning Data Sync Manager fileset (Backup Restore Manager fileset, SDM_BKM.bkm), and type config As the configuration script runs, you are prompted for the user name. Type the user name enabled to access the core in Configuring the bkmgrusr user ID and password to enable communication between the DBRM and SBRM (page 82).

You are prompted for the password. Type the password that was set up through the procedure, Configuring the bkmgrusr user ID and password to enable communication between the DBRM and SBRM (page 82). You are prompted for the logical volume where the backup is to be stored. This is the device on which the core image dump will be stored. make sure that this device has sufficient space for the backup file.

You are prompted for the core type, xa-Core or Compact. The software uses this information to determine if the core will have a message switch load.
--End--

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

86 Preparation for disaster recovery

Creating the backup user ID on the core for SBRM


Use this procedure to create the user ID on the core to enable the operation of the Synchronous Backup Restore Manager (SBRM). The types of operations this user are can performed are:

set dump_restore_in_progress field in ofcstd table start image dump ability to run itocci command set

Perform this procedure before you perform Configuring core access for SBRM through the Nortel Communication Server 2000 Core Manager (page 84).

Procedure steps
Step 1 Action At the CI prompt on the core, type permit <backupuser> <backupuser_pswd> 4 10000 english all If Enhanced Password Control is in effect on the CM and after the user is permitted on the switch, log onto the core manually with this user first. The core will prompt you to change the password at the first logon after the logon is permitted. Change the password and then perform the procedure, Configuring core access for SBRM through the Nortel Communication Server 2000 Core Manager (page 84) using the <backupuser> user you have created and the changed password. If Enhanced Password Control is in effect on the CM, the password must be at least six characters in length. The SBRM does not have the ability to manage passwords. Therefore, you must re-run the configuration script in Configuring core access for SBRM through the Nortel Communication Server 2000 Core Manager (page 84) to ensure the password for the <backupuser> user.
--End--

Variable denitions
Variable <all> Value The privilege setting.

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

Variable definitions

87

Variable <backupuser>

Value The user name that will be used by SBRM to log on to the core. Maximum length is 16 characters.

<backupuser_pswd>

The password for the <backupuser> user you are creating. Maximum length is 16 characters.

<english> <4> <10000>

The language setting. The priority. The stack size.

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

88 Preparation for disaster recovery

Applying software packages on a Core and Billing Manager using the CBMMTC interface
Use this procedure to install optional software packages on CBM cluster nodes.

Procedure steps
ATTENTION
Instructions for entering commands in this procedure do not show the prompting symbol, such as #, >, or $, displayed by the system through a GUI or on a command line. At your workstation

Step 1

Action In the command line prompt, enter cbmmtc apply to access the apply level of the cbm maintenance interface The system displays the apply level screen of the cbm maintenance interface, which shows a list of the packages (up to 12 software packages are displayed at a time), if any exist, in the default source directory.

2 3 4

To exit the CBM maintenance interface, type no and go to step 13. If the CBM software is on a CD-ROM, insert the CD-ROM into the CD drive if it is not already present in the drive. At the command line located at the bottom of the cbmmtc user interface screen, type:
source <source_directory_name>

Use /cdrom/cdrom/applications/cbm/packages as the <source_directory_name> if the software is on a CD-ROM. The system displays the apply level screen of the cbm maintenance interface, which shows a list of all packages in the source directory that you specified. 5 Locate the packages to be applied from the displayed list and take note of their numbers (located next to the names of the packages). Select the packages that you have decided to apply:
select <package number> ... <package number>

The packages you selected are highlighted on the cbmmtc apply screen. 6 To apply the selected packages:
apply
Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

Variable definitions

89

If there is a prerequisite package for the package(s) you have selected to apply on the system, the system SWIM tool automatically selects and applies the prerequisite package unless the package is currently selected to be applied. The system prompts if you want to continue with applying the selected packages.

8 9

If you do not want to continue the package application, type no and go to step 13. Type yes in response to the prompt. The status of each package application appears on the cbmmtc apply screen.

10

When the application is completed, the installed packages will appear in the list that displays when you enter the cbmmtc packages level. Verify that the status of the new packages indicates Applied under the Status column.

ATTENTION
It is important that packages installed on the system not be left with a Partial status. If any package installed application fails or otherwise shows a Partial status, contact your next level of support for assistance.

11 12 13 14

Go to step 13. Type no in response to the prompt. Exit from the cbm maintenance interface:
quit all

This procedure is complete.


--End--

Variable denitions
Variable <source_directory_name> Value The full pathname of the directory containing the package that you want to apply. Because you are using a CD-ROM for the installation, specify /cdrom/cdrom/applications/cbm/packages as the source_directory_name The number associated with a package, that you noted. Each package number is separated by preceding and succeeding spaces.

<package number>

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

90 Preparation for disaster recovery

Job aid
Example To select the Reach Through SPM application, which is number 6, and OM Delivery, which is number 11 in the sample screen display shown in the previous figure, enter select 6 11 To de-select any packages that you selected, re-enter the select command for the packages you want to de-select. The highlighting on the packages that you de-select will be removed.

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

Procedure steps

91

Cloning the image of the active node to the inactive node


Use this procedure to clone an active node image onto the inactive node of a CBM cluster.

Procedure steps
ATTENTION
Instructions for entering commands in this procedure do not show the prompting symbol, such as #, >, or $, displayed by the system through a GUI or on a command line. At your workstation

Step 1

Action Start the cloning process by typing


startb

and press the Enter key. 2 3 4 If the system prompts you for the Ethernet address, go to step 4. If the system indicates it is using Ethernet address <EthernetAddress>, record the IP address and go to step 9. At the console connected to the inactive node, log on to the inactive node through the console using the root user ID and password. If the system is not already at the OK prompt, bring the system to the OK prompt:
init 0

At the OK prompt, display the Ethernet address of the inactive node:


OK banner

Example response:
Sun Fire V240, No keyboard Copyright 1998-2002 Sun Microsystems, Inc. All rights reserved. OpenBoot 4.8.0.build_04, 2048 MB memory installed, Serial #52964131. Ethernet address 0:3:ba:28:2b:23, Host ID: 83282b23.

7 8

Record the Ethernet address that is displayed. At your workstation (session connected to Active node), enter the Ethernet address of the inactive node you recorded in step 6.

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

92 Preparation for disaster recovery

9 10 11

If the system prompts you to enter the command: boot net image, go to step 11. If the system does not prompt you to enter the command: boot net - image, to step 12. At the console connected to the inactive node, boot the inactive node from the image of the active node by typing:
OK boot net - image

and press the Enter key.

ATTENTION
There must be a space after the dash.

Example response:
SC Alert: Host System has Reset Sun Fire V240, No Keyboard Copyright 1998-2002 Sun Microsystems, Inc. All rights reserved. OpenBoot 4.8.0.build_04, 2048 MB memory installed, Serial #52964131. Ethernet address 0:3:ba:28:2b:23, Host ID: 83282b23. Rebooting with command: boot net - image . . . SC Alert: Host System has Reset

12

At your workstation (session connected to the Active node), monitor the progress of the cloning from the active node. Cloning the inactive node takes approximately one hour to complete. Example response:

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

Procedure steps

93

Waiting for network response from unit1-priv0... received network response from unit1-priv0... Waiting for unit1-priv0 to clone data... waiting...1 waiting...2 waiting...3 unit1-priv0 is cloning: /export/d2 . . . Verifying cluster status of unit1-priv0 waiting for cluster filesystem status to become normal. Deleted snapshot 0. Deleted snapshot 1. Deleted snapshot 2. Deleted snapshot 3. d99: Soft Partition is cleared

--End--

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

94 Preparation for disaster recovery

Installing the Backup Restore Manager software


Use this procedure to install the Backup Restore Manager software. The Backup Restore Manager application functionality requires the appropriate software resident and configured on platforms that require synchronized imaging. Although no Core and Billing Manager 850 data is backed up through the Backup Restore Manager, the Backup Restore Manager software must be installed on the CBM to allow control of the XA-core and 3PC (Compact) backup.

Procedure steps
ATTENTION
Instructions for entering commands in this procedure do not show the prompting symbol, such as #, >, or $, displayed by the system through a GUI or on a command line.

Step 1

Action At the CI prompt on the core, enter the following command:


permit <backupuser> <backupuser_pswd> 4 10000 english all

ATTENTION
If Enhanced Password Control is in effect on the CM, the password must be at least six characters in length.

ATTENTION
If Enhanced Password Control is in effect on the CM and after the user is permitted on the switch, log in to the core manually with this user first. The core will prompt you to change the password at the first logon after the logon is permitted. Change the password and then perform step 1. The SBRM does not have the ability to manage passwords. Therefore, you must rerun the configuration script in step 4.

At your workstation, apply the software application package, NTbkupmgr . Specify /cdrom/cdrom/applications/cbm/packages as the source directory when you perform that procedure. When the installation is complete, exit from the cbmmtc. At the command line prompt, change directory to the directory containing configuration script:
cd /opt/nortel/bkresmgr/cbm/scripts

3 4

Run the configuration script:


./bkmgr_config.sh

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

Variable definitions

95

You are first prompted for the user name. The user name is that used to log in to the core to initiate an image dump. The script restricts the user name to a maximum of 16 characters. The user name entered must first be enabled on the core in step 1 You are prompted for the user name you entered in step 6. You are first prompted for the password. The configuration script restricts the password to a maximum of 16 characters. Use the password set up in step 1 You are prompted for the logical volume where the backup is to be stored. This is the device on which the core image dump will be stored.

7 8

ATTENTION
Verify that this device has enough space to store the backup.

10

You are prompted for the core type, either XA-core or Compact.

ATTENTION
This information is needed in order for the software to know whether the core will also have a Message Switch load.

--End--

Variable denitions
Variable <backupuser> Value The user name for the core, that is up to 16 characters in length, that will be used by SBRM for logon The password for the <backupuser> user you are creating, which can be up to 16 characters in length 4 is the priority

<backupuser_pswd>

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

96 Preparation for disaster recovery

Conguring SSH between the backup restore manager servers


Configure Secure Shell (SSH) between the backup restore manager servers to set up SSH between the Integrated Element Management system (IEMS) server where the synchronous backup restore manager (SBRM) software and the SPFS-based servers where the device backup restore manager (DBRM) software is installed. Throughout the remainder of this procedure, the IEMS server will be referred to as the SBRM server and the SPFS-based servers will be referred to as the DBRM servers. SSH provides authentication and secure communications over insecure channels. This procedure contains the steps to generate the SSH key pair on the SBRM server and transfer the key pair to each DBRM server and the SDM platform in the network. It also contains the steps to transfer an existing SSH key pair from the SBRM server to new or existing DBRM servers and the SDM platform in the network.

Prerequisites

You need the root user ID and password for the SBRM server. You need the password for the bkmgrusr on the DBRM servers to transfer the SSH key pair. The bkmgrusr is created on the DBRM server when SPFS is installed. You need the root password for the DBRM server when the DBRM server is a two-server configuration, to synchronize the SSH data between the two servers.

ATTENTION
Ensure provisioning activities must not be in progress or scheduled to take place during this procedure.

Procedure steps
At your workstation

Step 1

Action Log in to the SBRM server > telnet <server>.

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

Procedure steps

97

In a two-server configuration, log in to the Active side using its physical IP address. 2 3 4 5 When prompted, enter your user ID and password. Change to the root user
$ su -

When prompted, enter the root password. Change directory to the location of the backup restore manager configuration script
# cd /opt/bkresmgr/admin/bin

Run the backup restore manager configuration script


# ./configure

For an example, see Job aid (page 99). 7 Enter the number next to the Backup Restore Manager SSH setup option in the menu. For an example, see Job aid (page 99). 8 9 10 If you are configuring SSH for the first time, or changing the SSH key pair then, go to 10. If you are transferring an existing SSH key to Device Backup Restore Manager servers then, go to 27. Enter the number next to the Generate SSH key and transfer to Device Backup Restore Manager servers option in the menu. For an example, see Job aid (page 99). 11 When prompted, confirm you want to continue by typing:
y

For an example, see Job aid (page 99). 12 When prompted, press Enter to accept the default location and filename /export/home/bkmgrusr/.ssh/id_rsa for the key. If the default location and filename is not provided, enter /export/home/bkmgrusr/.ssh/id_rsa. If the location and filename /export/home/bkmgrusr/ .ssh/id_rsa already exists and you wish to overwrite, enter y. Otherwise, enter n. For an example, see Job aid (page 99). 13 When prompted, enter y to confirm you want to transfer the SSH key pair. For an example, see Job aid (page 99). 14 When prompted, enter the hostname or IP address of a DBRM server in your network.

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

98 Preparation for disaster recovery

For an example, see Job aid (page 99). 15 When prompted, enter the number next to the platform type of the DBRM server. The SPFS platform type is for servers hosting the CS 2000 Management Tools, MG 9000 Manager and Core and Billing Manager (CBM). The SDM platform type is an AIX platform containing core management software. For an example, see Job aid (page 99). 16 When prompted, enter yes to confirm you want to continue connecting.
Warning: Permanently added comp5iems (45.123.456.98) (RSA) to the list of known hosts. Password:

17

When prompted, enter the password for the bkmgrusr on the DBRM server. For an example, see Job aid (page 99). When prompted, enter y if the DBRM server is a two-server configuration (HA), otherwise, enter n. If you have entered y (yes) then go to 21. If you have entered n (no) then go to 25. When prompted, enter y if you want to manually synchronize the SSH data, otherwise, enter n.

18 19 20 21

ATTENTION
The DBRM server root user password is required to complete this section.

22 23 24 25

If you have entered y (yes) then go to step 24. If you have entered n (no) then go to step 25. When prompted, enter the root password for the DBRM server. For an example, see Job aid (page 99). When prompted, indicate that you want to transfer the SSH key pair to the DBRM installation on the SBRM server in your network. The SBRM always makes a SSH connection to the DBRM residing on the local server so that the local IEMS application is notified of the synchronized backup by way of SBRM to DBRM messaging.

26

Enter the number next to the "Transfer existing SSH key to Device Backup Restore Manager servers" option in the menu. For example, see Job aid (page 99).

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

Job aid

99

27

When prompted, confirm you want to transfer the SSH key pair by typing:
y

This returns you to step 14. For an example, see Job aid (page 99). 28 29 Enter x to exit the Backup Restore Manager SSH setup level by typing: Enter x to exit the Backup Restore Manager Configuration level.
--End--

Variable denitions
Variable <server> Value IP address or host name of the IEMS server where the SBRM software is installed.

Job aid
Table 12 Example responses to procedure steps Step 6 Example response Backup Restore Manager Configuration 1 - Backup Restore Manager SSH Setup 2 - Automated Synchronous Backup Restore Manager Service X - exit select - [X, 1-2] SSH Setup for Synchronous Backup Restore Manager 1 - Generate SSH key and transfer to Device Backup Restore Manager servers 2 - Transfer existing SSH key to Device Backup Restore Manager servers X - exit select - [X, 1-2] Generate SSH key for Synchronous Backup Restore Manager and transfer to Device Backup Restore Manager servers. WARNING: Running this script will generate a new SSH key for the bkmgrusr. This new SSH key must be transferred to all existing and new Device Backup Restore Manager servers. Do you want to continue? [y/n]:

10

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

100

Preparation for disaster recovery

Step 11

Example response Generating SSH key for bkmgrusr. Sun Microsystems Inc. SunOS 5.9 Generic May 2002 Generating public/private rsa key pair. Enter file in which to save the key (/export/home/bkmgrusr/.ssh/id_rsa): Your identification has been saved in /export/home/bkmgrusr/.ssh/i d_rsa. Your public key has been saved in /export/home/bkmgrusr/.ssh/id_rs a.pub. The key fingerprint is: 33:99:17:f0:ac:2a:3a:86:a9:d9:76:65:e6:70:4b:6:8 bkmgrusr@com p5iems Synchronizing files IF clustered node Transfer existing SSH key to Device Backup Restore Manager servers WARNING: You must have the password for the bkmgrusr on the Device Backup Restore Manager server to transfer the SSH key. Do you want to transfer the SSH key? [y/n]: Setting up to transfer SSH key to DBRM server. Enter hostname or IP address of DBRM server to transfer the SSH key to: Platform type of DBRM server to transfer SSH key to: 1 - SPFS 2 - SDM Enter platform type of DBRM server to transfer SSH key to [1-2]: Sun Microsystems Inc. SunOS 5.9 Generic May 2002 The authenticity of host comp5iems (45.123.456.98) cant be established. RSA key fingerprint is ff:eb:a8:87:14:d0:82:28:5c:43:70:5a:ab:af:e6:f d Are you sure you want to continue connecting (yes/no)?: Transfer of SSH key to <server> complete. Is DBRM <server> a High Availability (HA) server? [y/n]: WARNING: On HA DBRM servers, SSH setup is not complete unless one of the following steps is performed. a) The HA DBRM server <servername> is cloned. b) The SSH data on HA DBRM <servername> is manually synched. WARNING: Choosing this option requires the root password for <servername> Do you want to manually synchronize the SSH data on DBRM <servername> now [y/n] Synchronization of SSH data on DBRM <servername> complete.

12

13

14

15

17 22

25

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

Job aid

101

Step 27

Example response Transfer existing SSH key to Device Backup Restore Manager servers WARNING: You must have the password for the bkmgrusr on the Device Backup Restore Manager server to transfer the SSH key. Do you want to transfer the SSH key? [y/n]: Setting up to transfer SSH key to DBRM server. Enter hostname or IP address of DBRM server to transfer the SSH key to:

28

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

102

Preparation for disaster recovery

Conguring SBA backup volumes on the core


Configure backup volumes on IOP, 3PC, DDU, or SLM disks on the core for a billing stream. The maximum number of volumes that you can configure for a billing stream is 69, or the maximum number supported by the underlying hardware, whichever is less per stream. The Table 13 "Disk drive backup volumes" (page 105) lists the disk drive backup volumes that you can configure for the BRISC and XA-core platforms.

Prerequisites

You must configure additional backup storage to prevent a temporary problem that forces the SBA into long-term backup mode. The billing stream is aware that the replaced volumes exist, and recovers files from both the swapped-out and swapped-in sets of volumes as part of the recovery process. The billing stream loses track of swapped-out volumes when a switch of activity (SwAct) or a restart is performed on the DMS or Communication Server 2000 prior to the completion of the recovery of the files.

There is a risk of losing some billing records when you reconfigure or swap-out backup volumes of a stream that is in backup mode during the transition process. During an emergency situation (outage), you must allow time for the recovery process to finish before you attempt to swap out an active backup volume. If you do not allow time for the recovery process to finish, the billing stream does not recognize the swapped-out volumes. If you are using or migrating to a XAC16 system, your backup volumes must be on IOP volumes. If your current backup volumes are on SLM or DDU volumes and you are running a previous release, you must migrate to IOP volumes before upgrading to this release.

ATTENTION
Ensure the size for backup volumes is sufficient. The absolute minimum size for backup volumes is 30MB.

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

Procedure steps

103

Procedure steps
Step 1 Action At your system, write down the dms_disk_space value recorded in Table 20 "Disk space requirements" (page 208) (answer 31), which shows the amount of disk space required for the backup volumes. Determine the amount of disk space of each disk type in your system to be used for storing the backup volumes. Divide the value recorded in step 1 by the maximum volume size supported for the appropriate disk types for your system, obtained from the table Table 14 "Metrics on various disk types " (page 106). Record these values. Ensure that the backup volumes can fit on the disks in your system. Compare the values recorded in step 2 with the maximum number of volumes supported for the disk types in your system, obtained from the table in the previous step. If the number of volumes obtained in the previous step is less than or equal to the maximum number allowed, go to step 4. OR If the number of volumes obtained in the previous step is greater than the maximum number allowed, contact the next level of support. 4 5 Post the billing stream by typing: mapci;mtc;appl;sdmbil;post <stream_name> Obtain information about the existing backup volumes for the billing stream : conf view <stream_name> SBA does not support the configuration of more than one billing stream at a time from multiple workstations. The last billing stream that is configured is the one that is saved. The system displays the name of each backup volume in the stream. Record each backup volume name for future reference. 6 7 Quit out of the MAPCI level :
quit all

Display and record the size of a volume and its number of free blocks: diskut;lv <volume name> Repeat the previous step for each volume name that you recorded in step 2.

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

104

Preparation for disaster recovery

If you are configuring an IOP disk drive backup volume, create an alphanumeric name, consisting of a maximum of twelve characters, for each of the new backup volumes that you determined in Calculation of core manager Disk Space Requirements (page 209). Record each of these names for future reference. IOP volume names on the IOP disks can be up to twelve alphanumeric characters in length, with the first four characters reserved for the disk prefix. OR Create a twelve-character, alphanumeric name for each of the new backup volumes. Record each of these names for future reference. 3PC volume names on the 3PC disks can be up to twelve alphanumeric characters in length, with the first four characters reserved for the disk prefix.

ATTENTION
Logical volumes must be configured evenly across the disks.

10 11 12

Access the disk administration level: diskadm <disk prefix> Determine the free disk space:
dd

Note the following example, which is a response to the command performed in step 8, choosing the F02L disk name.

13

Determine the size of the largest free segment. If the size of the largest free segment is greater than or equal to the maximum allowable volume size for the disk type, go to the next step. OR If the size of the largest free segment is less than the maximum allowable volume size for the disk type contact your next level of support before proceeding with this procedure.

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

Job aid

105

14

Create a new logical volume : cv <volume> <size> ftfs For an example to create a new logical volume, see "Example" (page 106) .

15 16 17 18

Exit the disk administration level at the prompt by typing:


quit

Repeat this procedure until all new logical volumes have been created. Exit to the command prompt:
quit all

Exit back to the command prompt :


quit all

You must alert all operating company personnel who are associated with the system. Provide the names of the old and new backup volumes and the procedure you used to swap the volumes. They must be made aware of that any RESTARTs or SwActs that occur before the billing stream returns to normal mode can cause a loss of billing records. Also, it is imperative that the mode of the billing stream must be closely monitored to ensure that it returns to normal mode without an intervening RESTART or SwAct.
--End--

Variable denitions
Variable <disk prefix> <size> <stream_name> <volume> Value One of the prefixes assigned to the two disks; for example, F02L or F17D. The size of the volume. The name of the billing stream. The backup volume name.

Job aid
The following table lists the backup volumes for the various platforms.
Table 13 Disk drive backup volumes Platform BRISC Backup volume(s) DDU or SLM

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

106

Preparation for disaster recovery

Table 13 Disk drive backup volumes (contd.) Platform XA-core (for releases prior to SDM16 or CS2E03) XA-core (for SDM16 or CS2E0 and higher) Backup volume(s) DDU or IOP IOP

The following table provides metrics on various disk types that can be used in this procedure.
Table 14 Metrics on various disk types Disk type Maximum disks per core 2 2 10 2 Maximum volumes per device 32 32 32 32 Maximum volumes configurable for SBA 64 64 69 64 Maximum volume size

IOP 3PC DDU SLM

2GB 2GB 64MB

The following example shows how the user can create a new logical volume. Example cv AMA8 50 ftfs This entry prompts the system to create the logical volume F17LAMA8, consisting of 50 Mbyte (102400 512-byte blocks) of available disk space.

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

Procedure steps

107

Starting or stopping the automated synchronous backup restore manager service


Use this procedure to start or stop the automated synchronous backup restore manager (SBRM) service on the SPFS-based server where the SBRM software is installed.

Prerequisites

You need the root user ID and password for the SPFS-based server where the SBRM software is installed.

ATTENTION
Ensure provisioning activities must not be in progress or scheduled to take place during this procedure.

Procedure steps
Step 1 Action

ATTENTION
In a two-server configuration, log in to the Active side using its physical IP address.

Log on to the SBRM server


> telnet <server>

2 3 4 5

When prompted, enter your user ID and password. Change to the root user
$ su - root

When prompted, enter the root password. Change directory to where the backup restore manager configuration script is located
# cd /opt/bkresmgr/admin/bin

Run the backup restore manager configuration script


# ./configure

For an example, see Job aid (page 108). 7 Enter the number next to the Automated Synchronous Backup Restore Manager Service option in the menu. For an example, see Job aid (page 108).

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

108

Preparation for disaster recovery

8 9 10

If you want to start the automated SBRM service then go to step 10. If you want to stop the automated SBRM service then go to step 12. To start the SBRM service enter the number next to the Start the Automated Synchronous Backup Restore Manager Service option in the menu. For example, see Job aid (page 108). Enter y to confirm you want to start the SBRM service.
y

11

For an example, see Job aid (page 108). Go to step 14 12 To stop the SBRM service enter the number next to the Stop the Automated Synchronous Backup Restore Manager Service option in the menu. For an example, see Job aid (page 108). 13 14 15 Enter y to confirm you want to stop the SBRM service. For an example, see Job aid (page 108). Enter x to exit the Automated Synchronous Backup Restore Manager Service level by typing Enter x to exit the Backup Restore Manager Configuration level by typing
--End--

Variable denitions
Variable server Value IP address or host name of the IEMS server where the SBRM software is installed.

Job aid
Step 6 Example response Backup Restore Manager Configuration 1 - Backup Restore Manager SSH Setup 2 - Automated Synchronous Backup Restore Manager Service X - exit select - [X, 1-2]

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

Job aid

109

Automated Synchronous Backup Restore Manager Service 1 - Configure Automated Synchronous Backup Restore Manager Service 2 - Start the Automated Synchronous Backup Restore Manager Service 3 - Stop the Automated Synchronous Backup Restore Manager Service 4 - View the Automated Synchronous Backup Restore Manager Service Configuration X - exit select - [X, 1-4] Start the Automated Synchronous Backup Restore Manager Service WARNING: Running this script will start the Automated Synchronous Backup Restore Manager Service Starting the service does not initiate a backup until the day and time configured. Do you want to continue? - [y/n]: Registering with SERVMAN. Starting the Automated Synchronous Backup Restore Manager Service. Starting AUTO_BACKUP_MANAGER through servstart AUTO_BACKUP_MANAG ER Started Stop the Automated Synchronous Backup Restore Manager Service WARNING: Running this script will stop the Automated Synchronous Backup Restore Manager Service Stopping the service does not abort a backup in progress. Do you want to continue? - [y/n]: Stopping the Automated Synchronous Backup Restore Manager Service. Stopping group using servstop AUTO_BACKUP_MANAGER Stopped Deregistering with SERVMAN

10

11

13

14

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

110

Preparation for disaster recovery

Invoking the network-level backup (SBRM-bkmgr) through telnet


Use this procedure to run a network backup (SBRM-bkmgr) on an IEMS server using CLUI. The network backup (SBRM-bkmgr) script is installed under the /bkresmgr/nbm directory. The bkmgr should be invoked from this directory to control and synchronize the activities related to backup across other managed devices which contain device-level backup software. Perform this procedure at your workstation.

Prerequisites
You need the root user ID and password for the IEMS server.

Procedure steps
Step 1 2 3 4 5 Action Log in to the IEMS server by entering the following command:
> telnet <server>

When prompted, enter your user ID and password. Change to the root user by entering:
$ su -

When prompted, enter the root password. Change the directory to where the network backup manager script (SBRM-bkmgr) is located by entering:
#cd /opt/nortel/bkresmgr/nbm

Start the network backup manager by entering:


bkmgr> <command>
--End--

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

Example of invoking the network-level backup (SBRM-bkmgr) through telnet

111

Variable denitions
Variable <server> <command> Value The IP address of the IEMS server. Any one of the following commands:

backup abort backup auto backup full query history query inventory query status quit or exit

or any of the following commands to perform a step by step backup for debugging:

activatenew dumpdata precheck startprov stopprov verify

Example of invoking the network-level backup (SBRM-bkmgr) through telnet


Example of invoking the network-level backup (SBRM-bkmgr) through telnet # cd /opt/bkresmgr/nbm # ./bkmgr bkmgr> backup full Beginning step: precheck Completed step: precheck Beginning step: stopprov Completed step: stopprov Beginning step: dumpdata Error during step: dumpdata *** Component wcars2yx *** *** Component DB *** Failed to create backup files

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

112

Preparation for disaster recovery bkmgr> bkmgr> query status Current State: dumpdata Process Status: idle *** Component wcars2yx *** Step dumpdata for wcars2yx is failed. Current State: dumpdata Process Status: idle *** Component DB *** Step dumpdata for DB is failed. *** Component IS *** Step dumpdata for IS is complete. *** Component NPM *** Step dumpdata for NPM is complete. *** Component CEM *** Step dumpdata for CEM is complete. *** Component IEMS *** Step dumpdata for IEMS is complete. bkmgr> bkmgr> query history Last backup failed at Wed May 21 08:51:31 EDT 2008. *** Component wcars2yx *** *** Component DB *** Failed to create backup files bkmgr> bkmgr> backup abort Beginning step: rollback Completed step: rollback Cleaning up subcomponents... bkmgr> query status Current State: No backup in progress. Process Status: idle bkmgr> query history Last backup failed at Wed May 21 08:51:31 EDT 2008. *** Component wcars2yx *** *** Component DB *** Failed to create backup files bkmgr>

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

Procedure job aid Example of invoking the network-level backup (SBRM-bkmgr) through telnet bkmgr> backup auto Beginning step: precheck Completed step: precheck Beginning step: stopprov Completed step: stopprov Beginning step: dumpdata Error during step: dumpdata Auto backup failed. *** Component wcars2yx *** *** Component DB *** Failed to create backup files Restarting provisioning... Beginning step: rollback Completed step: rollback bkmgr> bkmgr> query status Current State: No backup in progress. Process Status: idle bkmgr>

113

Procedure job aid


During a complete backup of all devices/components, the SBRM network backup manager performs the following steps:

precheck: performs a precheck of all devices/components. stopprov: locks the provisioning session of all devices/components. dumpdata: performs a backup of all devices/components. verify: verifies the backup data of all devices/components. startprov: unlocks the provisioning sessions of all devices/compone nts. activatenew: finalizes the backup data (move data to different directory/volume, ) for all devices/components.

To perform a complete backup, you can use one of the following 2 commands provided by the SBRM-bkmgr:

backup full: performs a complete backup of all devices/components. If a failure is encountered, the process stops at completion of the failed step, and you will need to abort or perform manual actions to recover from the failure. backup auto: performs a complete backup of all devices/components. If a failure is encountered, the process restores the provisioning session if it has been stopped.

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

114

Preparation for disaster recovery

To recover from a failure, use the backup abort command.

backup abort: aborts a backup after completion of the current step. The provisioning session will be unlocked if it has been locked by the backup process.

To perform a query, you can use one of the following three commands:

query status: displays the status of the backup at a high level. query history: displays information about the most recent backup. query inventory: displays the components/devices available for backup.

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

Procedure steps

115

Invoking the device-level backup (DBRM-bkmgr) through telnet


Use this procedure to run device-level backup on an SPFS-based server. The device-level backup (DBRM-bkmgr) is installed under /opt/bkresmgr/cbm directory. It is not recommended to run the DBRM directly from a CLUI interface, it should be controlled by the DBRM-bkmgr. In rare instances, it may be necessary to run various portions of the backup process on the device directly. Perform this procedure at your workstation.

Prerequisites
You need the root user ID and password for the SPFS-based server.

Procedure steps
Step 1 2 3 4 5 Action Log in to the SPFS-based server by entering:
> telnet <server>

When prompted, enter your user ID and password. Change to the root user by entering:
$ su -

When prompted, enter the root password. Change the directory to where device-level backup manager script (DBRM-bkmgr) is located by entering:
#cd /opt/nortel/bkresmgr/cbm

Start the backup manager by entering:


bkmgr> <command>
--End--

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

116

Preparation for disaster recovery

Variable denitions
Variable <server> <command> Value The IP address of the server. Any one of the following commands:

backup abort backup auto backup full query history query inventory query status quit or exit

or any of the following commands to perform a step by step backup for debugging:

activatenew dumpdata precheck startprov stopprov verify

Example of invoking the device-level backup (DBRM-bkmgr) through telnet


Example of invoking the device-level backup (DBRM-bkmgr) through telnet bkmgr> backup full RC=fail~reason=*** Component DB Failed to create backup files bkmgr> query history RC=success~status=Last backup failed at Thu May 22 08:28:28 EDT 2008. *** Component wcars2yx *** Component DB Failed to create backup files bkmgr> bkmgr> query status RC=success~status= Current State: dumpdata Process Status: idle

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

Procedure job aid *** Component DB *** Step dumpdata for DB is failed. *** Component IS *** Step dumpdata for IS is complete. *** Component NPM *** Step dumpdata for NPM is complete. *** Component CEM *** Step dumpdata for CEM is complete. *** Component IEMS *** Step dumpdata for IEMS is complete. bkmgr> bkmgr> backup abort RC=success bkmgr> query status RC=success~status= Current State: No backup in progress. Process Status: idle bkmgr>

117

Procedure job aid


During a complete backup of all devices/components, the device-level backup manager performs the following steps:

precheck: performs a precheck of all devices/components. stopprov: locks the provisioning session of all devices/components. dumpdata: performs a backup of all devices/components. verify: verifies the backup data of all devices/components. startprov: unlocks the provisioning sessions of all devices/compone nts. activatenew: finalizes the backup data (move data to different directory/volume, ) for all devices/components.

To perform a complete backup, you can use one of the following two commands provided by the DBRM-bkmgr:

backup full: performs a complete backup of all devices/components. If a failure is encountered, the process stops at completion of the failed step, and you will need to abort or perform manual actions to recover from the failure. backup auto: performs a complete backup of all devices/components. If a failure is encountered, the process restores the provisioning session if it has been stopped.

To recover from a failure, use the backup abort command.

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

118

Preparation for disaster recovery

backup abort: aborts a backup after completion of the current step. The provisioning session will be unlocked if it has been locked by the backup process.

To perform a query, you can use one of the following three commands:

query status: displays the status of the backup and shows the current state of the backup (that is, precheck, stopprov, dumpdata). query history: displays information about the most recent backup. query inventory: displays the components available for backup.

It is recommended that you use the query status and query history commands to verify the backup result. You should use the backup abort command to unlock the provisioning session if the backup failed.

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

Procedure steps

119

Backing up an XA-Core ofce image from disk to tape


Use this procedure to back up an eXtended Architecture Core (XA-Core) office image by copying the office image files from a disk to a digital audio tape (DAT) cartridge in an XA-Core shelf. Perform this procedure each week or as indicated in the routine maintenance schedule for your office.

Procedure steps
Step 1 2 Action Access the MAP CI level display
>QUIT ALL

Access the image table of contents (ITOC) user interfaceX


>ITOCCI

For an example, see Job aid (page 123). 3

ATTENTION
The example of a MAP response identifies the autoload registered (ALR) image file by an asterisk (*) in the ALR column. Each image file has an index number at the beginning of the tuple line. The ALR image in the example of a MAP response has an index number of 0. The XA-Core selects the ALR image file first to boot the switch. If the ALR image file does not boot the switch then the XA-Core selects the next image file. The next image file is by sequence of the index number from the top of the table.

To list the boot file for the XA-Core in ITOC


>LISTBOOTFILE XA

For an example, see Job aid (page 123). 4 To list the boot file for the message switch (MS) in ITOC, type
>LISTBOOTFILE MS

For an example, see Job aid (page 123). 5 Determine if the XA-Core and MS have image files that are autoload registered (ALR). The examples of a MAP response for steps 3 and 4 in the Job aid identify the ALR image files by an asterisk (*) in the ALR column. If the image files are ALR then, go to step 8.

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

120

Preparation for disaster recovery

7 8

If the image files are not ALR then, go to step 29.

ATTENTION
In the example of a MAP response for step 3 in the Job aid, the name of the office image file for XA-Core is IMG0517CY_CM. Image file IMG0517CY_CM is ALR. Image file IMG0517CY_CM is in volume F02LIMAGE.

ATTENTION
In the example of a MAP response for step 4 in the Job aid, the name of the office image file for the MS is IMG0517CY_MS. Image file IMG0517CY_MS is ALR. Image file IMG0517CY_MS is in volume F02LIMAGE.

Record the names of the office image files for XA-Core and MS that are ALR. Also record the volume name that has these office image files. The ALR image file is the file that you copy to the XA-Core tape. 9 10 Quit the ITOCCI user interface
>QUIT

At the shelf;

WARNING
Static electricity damage
Wear a wrist strap connected to the wrist-strap grounding point of the frame supervisory panel (FSP) when you handle the tape and packlet. The use of the wrist strap protects the packlets against damage caused by electrostatic discharge (ESD).

Determine from office records or office personnel if the DAT tape drive is clean. See the XA-Core procedure How to clean the XA-Core tape drive in the Nortel XA-Core Fault and Performance Management ( (297-8991-510)) . 11 Get a tape cartridge that has the approval of Nortel. Determine the tape planned for a backup of an office image. Determine the tape to use from the office records or from office personnel. Make sure the tape write protection is at the position that permits recording (closed). The tape write protection is an entrance on one side of the tape that has a sliding door. The sliding door is open for write protection and closed to allow a write to the tape.

12

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

Procedure steps

121

13

Insert the DAT tape cartridge into the XA-Core tape drive and close the drive door. The XA-Core tape drive is in the input/output processor (IOP) card of the XA-Core shelf.

14

CAUTION
File of tape lost when formatted
If the tape had files, the formatting of the tape loses the files. Make sure the tape has no files that your office needs.

Access the MAP disk utility


>DISKUT

. For an example, see Job aid (page 123). 15 Load the tape into the MAP disk utility >INSERTTAPE snnp TAPE WRITELABEL label_name For an example, see Job aid (page 123). 16 Confirm the command
>YES

For an example, see Job aid (page 123). 17

ATTENTION
A volume can have more files listed by command LISTVOLS than by command LISTFL in the MAP disk utility. The difference in the number of files between the commands is because of directory files not displayed by command LISTFL.

To list the files in the volume that contains the office image, enter >LISTFL vol_name . For an example, see Job aid (page 123). 18 Begin the disk to tape backup process. To create a backup copy of the XA-Core image file, type: >BACKUP FILE file_name snnp TAPE . For an example, see Job aid (page 123). 19 20 If the command is successful, got step 21. If the command is not successful, go to step 29.

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

122

Preparation for disaster recovery

21

Create a backup copy of the MS image fileX >BACKUP FILE file_name snnp TAPE For an example, see Job aid (page 123).

22 23 24

If the command is successful, go to step 24. If the command is not successful, go to step 29. Check the backup copies of the image files on the tapeX >LISTFL snnp TAPE For an example, see Job aid (page 123).

25

Eject the tape from the MAP disk utility after the backup procedure completes >EJECTTAPE snnp TAPE For an example, see Job aid (page 123).

26

Exit the MAP disk utility and return to the MAP CI level
>QUIT

For an example, see Job aid (page 123). 27 Remove the tape cartridge from the tape drive.

WARNING
Static electricity damage
Wear a wrist strap connected to the wrist-strap grounding point of the frame supervisory panel (FSP) when you handle the tape and packlet. The use of the wrist strap protects the packlets against damage caused by electrostatic discharge (ESD).

Set the tape write protection to the position that does not permit recording (open). The tape write protection is an entrance on one side of the tape that has a sliding door. The sliding door is open for write protection and closed to allow a write to the tape.

28 29

Store the tape cartridge according to office procedure. For additional help, contact the next level of support.
--End--

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

Job aid

123

Variable denitions
Variable <label_name> Value The alphanumeric name of the tape label that records the data. The name can be up to 32 characters long. If blank spaces are in the label name then enclose the label name with quotation marks. The number of the slot position on the XA-Core shelf for the IOP that has the tape device. The upper (U) or lower (L) packlet position of the IOP that has the tape device. The front (F) or rear (R) shelf position of the input output processor (IOP) that has the tape device. The name of the disk volume that contains the office image files.

<nn> <p> <s>

<vol_name>

Job aid
Step 2 3 Example response ITOC User Interface is now active. ITOCCI: Image table Of Contents for XA : A Registered Generic Device File L Date Time Name R MM/DD/YYYY HH:MM:SS -- - -------------------- -------------- ----0 * 05/17/1999 19:26:29 F02LIMAGE IMG0517CY_CM Image Table of Contents for MS : A Registered Generic Device File L Date Time Name R MM/DD/YYYY HH:MM:SS -- - -------------------- -------------- ----0 * 05/17/1999 19:26:29 F02LIMAGE IMG0517CY_MS Disk utility is now active. DISKUT:

14

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

124

Preparation for disaster recovery

Step 15

Example response >INSERTTAPE F02UTAPE WRITELABEL IMAGE_1 Example of a MAP response ***** WARNING ***** Writing the label IMAGE_1 to tape volume F02UTAPE on node CM will destroy all files stored on this tape volume. Do you want to continue? Please confirm ("YES", "Y", "NO", or "N")

16

The INSERT operation may take up to 5 minutes to tension the tape. A tape is now available to user on unit 0, node CM. Name IMAGE_1 has been written to the tape label. >LISTFL F02LIMAGE Example of a MAP response File information for volume F02LIMAGE: {NOTE: 1 BLOCK = 512 BYTES } --------------------------------------FILE NAME O R I O O V FILE MAX NUM OF FILE LAST R E T P L L CODE REC RECORDS SIZE MODIFY G C O E D D LEN IN IN DATE C N FILE BLOCKS ---------------------------------------------IMG0517CY_MS I F Y 0 1020 7542 15360 990517 IMG0517CY_CM I F Y 0 1020 165180 329728 990517

17

18 21

FTFS file IMG0517CY_CM on disk volume F02LIMAGE on node CM backed up as file IMG0517CY_CM on tape device F02UTAPE on node CM. FTFS file IMG0517CY_MS on disk volume F02LIMAGE on node CM backup up as file IMG0517CY_MS on tape device F02UTAPE on node CM.

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

Job aid

125

Step 24

Example response >LISTFL F02UTAPE Example of a MAP response File information for tape volume F02UTAPE, node CM: {Note: 1 BLOCK = 512 BYTES} --------------------------------------------CREATE ORG FILE V FILE NUM OF REC FILE NAME DATE TYPE CODE L SIZE IN RECORDS LEN D BLOCKS IN FILE ---------------------------------------------990520 IMAG 0 329070 165180 1020 IMG0517CY_CM 990520 IMAG 0 15026 7542 1020 IMG0517CY_MS

25

The EJECT operation may take up to 5 minutes to position the tape to the beginning. Rewind of tape F02UTAPE, unit 0, on node CM is completed. This tape device is not available to the user now.

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

126

Preparation for disaster recovery

Backing up Call Agent


Perform a Call Agent backup to make a backup of the call processing application. Images to be backed up are created either manually with the DUMP command, or automatically scheduled with entries in table IMAGEDEV and IMGSCHED. The Call Agent uses two software loads. The first software load includes the platform software such as the operating system and system utilities. The second software load provides the call processing application.

Procedure steps
Step 1 Action At the map, list the available images. The image with the asterisk in the ALR column identifies the image that is set to Auto Load Record and is the image to backup.

At the Call Agent Manager, log on to the inactive Call Agent and change directory to the location of the image.
cd /3PC/sd0x/imagex

ATTENTION
Do not modify files at this level. Any modification to files must be completed through the MAP.

Open a file transfer protocol (FTP) session to the Communication Server 2000 Core Manager and transfer the file.

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

Variable definitions

127

4 5

At the SDM, insert a DAT cassette.

ATTENTION
Use the rvf argument to append to the tape. After the backup completes, optionally verify the integrity of the backup by using tar tvf /dev/rmt0 to view the contents of the tape.

At the Communication Server 2000 Core Manager, verify that the size of the transferred file is the same as in step 2 and copy the file to tape.

Use the rm command to erase the oldest image so the volume does not fill up.
--End--

Variable denitions
Variable </3PC> Value Prefixed in all cases.

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

128

Preparation for disaster recovery

Variable <sd0x/> <imagex/>

Value Taken from the first four characters in the Generic Device name. Taken from the remaining characters in the Generic Device name.

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

Procedure steps

129

Backing up the SST


Use his procedure to perform a manual backup of the SST or to change the backup time. By default, the unit performs a backup of the database and critical files every day at 1:00 AM. see Rescheduling backup time (page 132) to alter the schedule. The contents of the backup are stored locally on each unit in the /data/bkresmgr/backup directory in tape archive (tar) format. For more information, see "Performing a rollback of Session Server Trunks software" in (NN10440-450).

Prerequisites

Suspend all SIP Gateway application provisioning activities during the time of the database backup to ensure that an accurate and complete copy of the active unit database is created. Call processing is not affected and there is no enforcement in the database to prevent provisioning.

Procedure steps
Step 1 2 Action At the NCGL CLI or IEMS client, log on to the ACTIVE unit and change to the root user. Start the backup
/opt/apps/db_install/sd/bkup_ngss.sh

The following output is printed to the screen and the backup is recorded to /data/bckresmgr/backup.

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

130

Preparation for disaster recovery Figure 12 bkup_ngss.sh response

The backup filename is located in /data/bkresmgr/backup and is identified by hostname, date, and time as indicated in the following example: Example Response: unit0.backupfile.2005-04-12_17-10.tgz
Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

Procedure steps

131

For security purposes, ensure that a copy of the backup file is transferred to a secure location. Use the scp command to make a secure copy of the backup file to a secure, remote server on your network. This server should be continuously available for cases where a restoration of the unit become necessary, such as during an upgrade rollback.

4 5

If security related files are being backed up, refer to Backing up security related files manually (page 134). Repeat this procedure on the second unit.
--End--

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

132

Preparation for disaster recovery

Rescheduling backup time


Each unit performs an automatic backup at 1:00 AM daily. Use the following procedure to change the schedule.

Procedure steps
At the NCGL CLI or IEMS client

Step 1 2 3

Action Log onto the unit and change to the root user. Remove the existing backup schedule from the crontab
/opt/apps/db_install/sd/uninstallBkres.sh

Edit the backup schedule file


vi /opt/apps/db_install/sd/backup.cron

The file is opened for editing.

Modify the first five fields (indicated by "0 1 * * *" in the example). The fields are identified in order from left to right:

minute -- the minute of the hour that the command will run, values are 0 to 59 hour -- the hour of the day the command will run, values are 0 to 23 with 0 being midnight day -- the day of the month the command will run, values are 1 to 31, an asterisk indicates to run the command all days month -- the month of the year the command will run, values are 0 to 12, an asterisk indicates to run the command all months weekday -- the day of the week that the command will run, values are 0 to 6, with 0 being Sunday, an asterisk indicates to run the command all weekdays

Do not modify the sixth field (/opt/.../bkup_ngss.sh). When done, write and quit the file. 5 Install the new schedule in the crontab
/opt/apps/db_install/sd/installBkres.sh

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

Procedure steps

133

After the scheduled time, verify that a backup file is created in the /data/bkresmgr/backup directory and that it is created at the new time, not the previously scheduled time. Repeat this procedure on the second unit.
--End--

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

134

Preparation for disaster recovery

Backing up security related les manually


The automatic backup does not backup the certificate.keystore or key files. This guards against loss or theft of the backup which would compromise the key without the operating company personnel knowing about the breach. Copy the following files to a safe location:

/opt/base/share/ssl/certificate.keystore /opt/base/share/ssl/gen_cert.txt /opt/base/share/ssl/server.crt /opt/base/share/ssl/server.key /opt/base/share/ssl/trusted.crt /opt/base/synch_local/common/etc/ssh/ssh_host_dsa_key /opt/base/synch_local/common/etc/ssh/ssh_host_key /opt/base/synch_local/common/etc/ssh/ssh_host_rsa_key

Procedure steps
Step 1 Action At the NCGL CLI or IEMS client, log on to either unit (usually the unit where the latest version of security certificates are stored), and change to root user. Navigate to the /opt/base/share/ssl directory:
cd /opt/base/share/ssl

2 3

Create a new directory to store backup copies of the certificate files: mkdir <SNxx_ddmmyyyy> Copy the certificates to the newly created backup directory:
cp * <SNxx_ddmmyyyy>

ATTENTION
Completing this step ensures that you have valid backup copies of the security certificates for restoring in case of an upgrade abort or rollback or for disaster recovery purposes.

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

Variable definitions

135

ATTENTION
If you want to make backup copies of the security certificates to a remote server then do ensure that the remote host that will store the security related files is secure, that the host is unavailable to general users, and offers restricted access to personnel with security related responsibilities.

Secure copy the files to the remote server: cd <SNxx_ddmmyyyy>scp *<user>@<remote_server>:</ path> The files are copied to the remote server.
--End--

Variable denitions
Variable <SNxx_ddmmyyyy> Value is the name of the new directory based on the currently installed release of the system software (for example SN08) and the current date in the format ddmmyyyy is a valid user ID on the remote server is the IP address of the remote server is where the file will be located on the remote server

<user> <remote_server> <path>

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

136

Preparation for disaster recovery

Saving the Ethernet Routing Switch 8600 boot conguration le


Use the Boot Monitor CLI to save the Ethernet Routing Switch 8600 configuration file. You must have access to the Boot Monitor CLI through a direct connection to the switch or a Telnet connection. For more information about accessing the Boot Monitor CLI, see Managing the Ethernet Routing Switch 8000 Series Switch Using the Command Line Interface Release 3.2, 313194-*.

ATTENTION
You must be directly connected to the switch to initiate a Boot Monitor session. You can connect using a Telnet connection only if the Boot Monitor CLI is already active.

Procedure steps
Step 1 Action

ATTENTION
To save a file to the standby CPU, you must enable TFTP on the standby CPU. To enable TFTP, enter flags tftpd true in the Boot Monitor CLI or config bootconfig flags tftpd true in the Run-Time CLI.

At the Boot Monitor CLI, issue the save command


monitor# save <save_type> [file <value>] [verbose] [standby <value>] [backup <value>]
--End--

Variable denitions
Variable backup <value> Value Saves the specified file name and identifies the file as a backup file. Example save config file ralph.cfg backup 2 file <value> A filename in one of the following formats: [a.b.c.d]: <file>

/pcmcia/<file> /flash/<file>

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

Variable definitions

137

Variable <save_type> standby <value> verbose

Value Specifies what to save. Possible values for this parameter are config, bootconfig, log, and trace. Saves the specified file name to the standby CPU (currently not supported). Saves default and current configuration. If you omit this parameter, only parameters you have changed are saved.

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

138

Preparation for disaster recovery

Conguring the automated synchronous backup restore manager service


Use this procedure to configure the automated synchronous backup restore manager (SBRM) service on the SPFS-based server where the SBRM software is installed, for automated execution of backups. You can configure the SBRM service to execute backups on a weekly or daily basis.

Prerequisites

You need the root user ID and password for the SPFS-based server where the SBRM software is installed.

Procedure steps
At your workstation

Step 1

Action Log on to the SBRM server


> telnet <server>

In a two-server configuration, log in to the Active side using its physical IP address. 2 3 4 5 When prompted, enter your user ID and password. Change to the root user
$ su -

When prompted, enter the root password. Change to the directory where the backup restore manager configuration script is located
# cd /opt/bkresmgr/admin/bin

Run the backup restore manager configuration script


# ./configure

For an example, see Job aid (page 140). 7 Enter the number next to the Automated Synchronous Backup Restore Manager Service option in the menu. For an example, see Job aid (page 140). 8 Enter the number next to the View the Automated Synchronous Backup Restore Manager Service Configuration option in the menu to view the configuration settings..

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

Variable definitions

139

For an example, see Job aid (page 140). 9 10 11 If you want to change the configuration settings of the SBRM service then go to step 11. If you do not want to change the configuration settings of the SBRM service then go to step 19. To change the configuration settings, enter the number next to the Configure Automated Synchronous Backup Restore Manager Service option in the menu. For an example, see Job aid (page 140). 12 13 14 15 When prompted, enter the number next to daily or weekly. For an example, see Job aid (page 140). If you select weekly then go to step 15. If you select daily the go to step 16. When prompted, enter the number next to the day of the week on which you want the backup to execute, or press the Enter key to accept the default value. When prompted, enter the time of day at which you want the backup to execute, or press the Enter key to accept the default. For an example, see Job aid (page 140). 17 If you want to start the automated SBRM service now, perform Starting or stopping the automated synchronous backup restore manager service (page 107). If you do not want to start the automated SBRM service now then go to step 11. Enter x to exit the Automated Synchronous Backup Restore Manager Service level. Enter x to exit the Backup Restore Manager Configuration level.
--End--

16

18 19 20

Variable denitions
Variable <server> Value IP address or host name of the IEMS server where the SBRM software is installed.

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

140

Preparation for disaster recovery

Job aid
Step 6 Example response Backup Restore Manager Configuration 1 - Backup Restore Manager SSH Setup 2 - Automated Synchronous Backup Restore Manager Service X - exit select - [X, 1-2] Automated Synchronous Backup Restore Manager Service 1 - Configure Automated Synchronous Backup Restore Manager Service 2 - Start the Automated Synchronous Backup Restore Manager Service 3 - Stop the Automated Synchronous Backup Restore Manager Service 4 - View the Automated Synchronous Backup Restore Manager Service Configuration X - exit select - [X, 1-4] The Automated Synchronous Backup Restore Manager Service is not running Automated Synchronous Backup Restore Manager Service Configuration Settings Backup Enabled : N Backup Day : SUNDAY Backup Hour : 24 Automated Synchronous Backup Restore Manager Service Configuration Configuring the Automated Synchronous Backup Restore Manager Service does not START the service. Return to the previous menu and select Start the Automated Synchronous Backup Restore Manager Service to start the service. Do you want the backup to run 1.daily or 2.weekly [1/2] : Example response if daily is selected: Hour to run the Synchronous Backup Restore Manager based on 24 hour clock: Example 1 is 1 AM, 12 is noon, and 24 is midnight. This is the hour on the <DAY> the Synchronous Backup Restore Manager will execute a full backup. Enter hour to run the Synchronous Backup Restore Manager (default: 24) Example response if weekly is selected: Day of the week to run the Synchronous Backup Restore Manager: This is day of the week the Synchronous Backup Restore Manager will execute a full backup. 1 - MONDAY 2 - TUESDAY 3 - WEDNESDAY 4 - THURSDAY 5 - FRIDAY

11

12

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

Job aid

141

6 - SATURDAY 7 - SUNDAY Enter day of week to run the Synchronous Backup Restore Manager (default: SUNDAY): 16 Automated Synchronous Backup Restore Manager Service Configuration complete. Configuring the Automated Synchronous Backup Restore Manager Service does not START the service. Return to the previous menu and select Start the Automated Synchronous Backup Restore Manager Service to start the service.

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

142

Preparation for disaster recovery

Conguring Automated INI le backup on a Media Server 2000 node


Use this procedure to configure automated INI file backups from the IEMS GUI main window. The automated INI file backup configuration settings apply to all nodes. Therefore, changing the settings on one node affects all nodes. Refer to the NN10329-111 to log into the IEMS GUI.
Procedure steps

Step 1

Action Select Applications -> IEMS -> Network Elements (refer to the figure below).

2 3

Select the Media Server 2000 series node. Right click the Media Server 2000 series node and select Configure MS2000 Automated INI Backup from the pop-up menu. The Automated INI File Backup Configuration pop-up menu displays.

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

Job aid

143

Click on the Enable Automated INI file backup box. The Middle portion of the pop-up window changes from gray to black. If the Weekly radio button is selected, the days of the week listed at the bottom of the screen are black as well.

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

144

Preparation for disaster recovery

5 6 7 8

Enter a start time for the backup. Click the radio button for either daily or weekly automated backups. If the weekly radio button is selected, click the boxes of the day(s) of the week for which you want the backup to occur. Click on the OK button to complete the automatic INI backup configuration.
--End--

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

Job aid

145

Changing SNMP community string passwords


Use this procedure to change SNMP community string passwords for a Media Server 2000 series node from the IEMS GUI main window. By default, the read and read-write SNMP community strings are public and private, respectively. Enter these into the IEMS GUI when adding or changing the community string passwords. Refer to the NN10329-111 to log into the IEMS GUI.
Procedure steps

Step 1

Action Select Applications -> IEMS -> Network Elements (refer to the figure below).

2 3

Select the Media Server 2000 series node icon. Right click the Media Server 2000 series node icon and select the Config and Maintenance Tool from the pop-up menu.

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

146

Preparation for disaster recovery

4 5

Select the Password Management tab. Select the SNMP tab from the Password Management screen.

6 7

Enter the passwords for the read or write community strings. Click the Change Read String or Change Write String button to complete the password change.
--End--

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

Procedure steps

147

Backing up INI les for all nodes


Use this procedure to back up the INI files for all nodes. This procedure pertains to both the Media Server 2010 and the Media Server 2020. The sample screens in this procedure show a Media Server 2010.

Prerequisites

You must be member of the succssn and emsadm groups.

Procedure steps
Step 1 2 3 4 5 Action At the Windows desktop interface Open a Telnet connection to the CS 2000 Management Tool. Log on with the appropriate user name. Invoke the MS 2000 Series CLUI /opt/nortel/NTsesm/bin/ms2000.sh When prompted, enter the appropriate user name and password. The Media Server 2000 Series CLUI Main Menu is displayed. Enter the number for the Back up the INI files for all nodes menu option. Sample response Backup of .ini for node 172.17.40.230 completed and copied to SDM Press <enter> to continue 6 7 8 Press Enter. The Media Server 2000 Series CLUI Main Menu appears. Enter x to exit the menu. Enterexit to end the Telnet session.
--End--

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

148

Preparation for disaster recovery

Displaying Media Server 2010 current node conguration


Use this procedure to display the current configuration of the Media Server 2010 node.

Prerequisites

You must be member of the succssn and emsadm groups.

Procedure steps
Step 1 2 3 4 5 6 7 Action At the Windows desktop interface Open a Telnet connection to the CS 2000 Management Tool. Log in with the appropriate user name. Invoke the MS 2000 Series CLUI /opt/nortel/NTsesm/bin/ms2000.sh When prompted, enter the appropriate user name and password. The Media Server 2000 Series CLUI Main Menu is displayed. Enter the number to select the Node Maintenance and Configuration menu option. When prompted, enter the IP address of the Media Server node. The Main Menu is displayed. Enter the number to select the Configuration Menu, menu option. The Main Menu is displayed. 8 Enter the number to select theDisplay this nodes current configuration menu option. The current configuration data for the selected node is displayed. 9 10 11 Press Enter. The Main Configuration Menu is displayed. Follow the prompts to return to the Media Server 2000 Series CLUI Main Menu. Enter x to exit the Media Server 2000 Series CLUI Main Menu.

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

Procedure steps

149

12

Enter exit to close the Telnet session.


--End--

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

150

Preparation for disaster recovery

Backing up the APS-specic Oracle database and application les


To ensure successful recovery from a system problem that causes database file corruption, it is recommended that you periodically back up the database files that support operation of the APS. These files include:

Oracle database files Root database files non-Root database files

To back up these files, use the following procedures: Performing a backup of Oracle/MySQL data on an SPFS-based server (page 156) to back up the Oracle database. This utility runs automatically each day at 1 a.m. and backs up the database to a 4mm DAT tape. The utility can also be run manually to back up the database to a disk file. Performing a backup of file systems on an SPFS-based server (page 164)to back up the UNIX file system, including all of the "Root" and "non-Root" database files.

Instructions for performing these procedures are found in NN10409-500. Two APS-specific utilities enable you to back up selected files, when only files required for APS operation must be restored. The "ips_export_db.sh" utility backs up the APS Oracle database. The "backup_appl_data.sh" utility backs up only the non-Root application files, "/audio_files," "PROV_data," "/user_audio_files," and Root application file, "/etc/inet/hosts." Both of these utilities can only be run manually. This procedure enables you to perform a complete manual backup of the APS-specific Oracle database files and application files. It is recommended that this procedure be performed once for each week.

Procedure steps
In a telnet connection to the APS server

Step 1 2

Action Open an xterm window, and log on using the "maint" logon and password. Become the "root" user by entering:
su - root

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

Procedure steps

151

Start the backup


ips_export_db.sh -diskonly

The system displays a log of the backup activity. 4 List the content of the tape on the terminal screen to verify the backup by entering the following commands:
cd /audio_files/aps_db_backup ls -l

A listing of the backed-up files displays. Look in this list for "dmp" files.
more README

A timestamp displays, which should show the time and date of this backup.

ATTENTION
At this point, you have completed the disk-only portion of the backup. Continue with the next step to complete the backup of the application files. The application files are backed up on tape.

Insert a write-enabled (white or grey tab is moved to the right where it can be seen) DAT tape into the 4mm DAT drive on the APS server. Rewind the tape
mt -f /dev/rmt/0c rewind

Start a backup of the application file systems on a single tape:


/usr/ntdb/uas/scripts/backup_appl_data.sh

The system displays a log of the backup activity. 7 8 List the content of the tape on the terminal screen to verify that the backup was successful
tar tvf /dev/rmt/0c | more

Rewind the backup tape


mt -f /dev/rmt/0c rewind

Eject the backup tape, label it, and move the write-enable tab to the "read-only" position (white or grey tab is moved to the left where it cannot be seen), to prevent the data on the tape from being accidentally over-written. Store the tape for use later. Insert another write-enabled DAT tape into the drive to be used

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

152

Preparation for disaster recovery

for the automatic Oracle system back up that runs daily at 1:00 a.m.
--End--

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

Procedure steps

153

Using the command line interface to perform backups


You can use the command line to perform full, incremental, or selective backups. You can perform multiple backups on multiple devices in a single command. You have the option of obtaining the backup information from a file. To display online help for this command, use the -h option on the command line.

Prerequisites

Before you perform a backup, you need to start up the Backup Controller and Providers.

Procedure steps
Step 1 Action Enter the following command as one continuous command
/opt/MagellanNMS/bin/nsbck [-full <devtype> <devname> <devaddr> <id> <pw>] [-incr <devtype> <devname> <devaddr> <id> <pw>] [-view <viewname> <devtype> <devname> <devaddr> <id> <pw>][-cdir <backup_dir_path> [-f <backup_info_file>][-chost <controller_address>] [-nolog] | [-log [<logfile>]] [-nc <#_concurrent_connections>]

The service data backup files have the following naming convention:
./<devtype>/<devname>/<timestamp>.<dataset>/ \ <datafiles...>

ATTENTION
The period (.) represents the SNMP Devices Backup and Restore root directory.

--End--

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

154

Preparation for disaster recovery

Variable denitions
Variable <backup_info_file> Value the name of the file containing backup information. Each line in the file has the same format as the -full, incr, or -view options. The backup directory used for this backup operation. the address of the Controller and has the format host[:port] The names of one or more files that are backed up. the dataset name. the IP address of the device and has the format n.n.n.n. the name of the device. The device type (PP4400, or PP4460) is the user ID for a Passport and the READ community string for a Passport 4400/4460. the name of the file to which output messages go. If not specified, the messages go to the file mbrbackup.log on the current directory (where the tool runs). the user password for a Passport and the WRITE community string for a Passport 4400/4460. has the format yyyymmddhhmmss indicates a selective backup. the name of the view file for a selective backup. the number of concurrent backups. indicates the remote Controller to be used in place of the Controller running on the local host. indicates that the backup information is obtained from a file. indicates a full backup. indicates an incremental backup. indicates that output messages go to stdout/stderr or to a log file. The default is stdout/stderr.

cdir <controller_address> <datafiles...> <datatset> <devaddr> <devname> <devtype> <id> <logfile>

<pw>

<timestamp> <view> <viewname> <#_concurrent_connections> -chost -f full -incr -log

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

Variable definitions

155

Variable -nc

Value indicates the number of concurrent backups to be performed. By default, SNMP Devices Backup tries to back up 5 different devices concurrently. This parameter is useful when you are backing up a large number of devices. indicates that output messages are to be discarded.

-nolog

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

156

Preparation for disaster recovery

Performing a backup of Oracle/MySQL data on an SPFS-based server


Perform the following to execute a backup of Oracle/MySQL data on a SPFS-based server (Sun Netra 240). The server can be hosting one or more of the following components:

Multiservice Data Manager (MDM) Communication Server Management Tools Integrated Element Management System (IEMS) Audio Provisioning Server (APS) Media Gateway (MG) 9000 Manager Network Patch Manager (NPM) Core and Billing Manager (CBM)

An Oracle/MySQL data backup is not required when the SPFS-based server is hosting the CBM or the MG 9000 Manager. If the SPFS-based server is hosting the IEMS, it is highly recommended to purge the IEMS event and performance data prior to executing the data backup. This reduces the size of the space used by the IEMS, and therefore, reduces the backup time, and possibility of a backup failure. The purge capability is only available in (I)SN07 onward. For more information, see "Configuring dark office backups on an SPFS-based server" in Nortel ATM/IP Solution-level Configuration ( (NN10409-500)) .

ATTENTION
It is recommended that provisioning activities be put on hold during the time of the data backup.

Prerequisites

For a Sun Netra 240, you need one or more blank DVDs to store the data. The backup utility limits the storage to 4GB on a DVD-R or DVD-RW. If you are using a new DVD-RW or want to reuse a DVD-RW by erasing the contents, perform procedure Preparing a DVD-RW for use in Nortel ATM/IP Solution-level Administration and Security ( (NN10402-600))

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

Procedure steps

157

ATTENTION
The database must be in sync with the Communication Server 2000 and the MG 9000 Manager (if present). Therefore, ensure you have the appropriate image of both before you proceed. Performing a restore from the Oracle/MySQL database alone can cause data mismatches at the Communication Server 2000 and the MG 9000 Manager (if present).

Procedure steps
ATTENTION
In a two-sever configuration, execute this procedure on the Active server.

Step 1

Action At the server, insert the blank DVD into the drive. In a two-server configuration, insert the blank DVD into the drive of the Active server. At your workstation, log on to the server by typing
> telnet <server>

and press the Enter key. 3 4 When prompted, enter your user ID and password. Change to the root user by typing
$ su -

and press the Enter key. 5 6 When prompted, enter the root password. If the server is hosting the IEMS, and you want to purge the event and performance data, go to step 7, otherwise, go to step 10

ATTENTION
This step stops the IEMS server. Ensure it is acceptable at this time to stop the IEMS server application.

Stop the IEMS server


# servstop IEMS

8 9 10

Run the script to purge the data


# /opt/nortel/iems/current/bin/purgeTempData.sh

Start the IEMS server


# servstart IEMS

Back up the data

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

158

Preparation for disaster recovery $ /opt/nortel/sspfs/bks/bkdata

Example response:
[12/30/06:17:35:03] cs2k0bd:/> /opt/sspfs/bks/bkdata Backup behavior is set to 1 ... *** Note: 8X or higher DVD-RW media may result in a lower success rate. *** *** DVD will be ejected after the backup is complete *** Do you wish to continue the backup (yes or no)? [yes] yes

11 12

Reinsert the backup DVD into the drive. List the content of the DVD
# tar tvf /cdrom/*bkdata*/*.tar

Example response:
-rw-rw-rw- 0/1 1874917 Mar 2 10:17 2005 opt/oracle.dmp .gz -rw-rw-rw- 0/1 1007616 Mar 2 10:17 2005 opt/critdata.c pio

Example response for server which hosts MySQL:


-rw-rw-rw- 0/1 1874917 Mar 2 10:16 2005 opt/oracle.dmp .gz -rw-rw-rw- 0/1 1007616 Mar 2 10:16 2005 opt/critdata.c -rw-rw-rw- 0/1 1874917 Mar 2 10:16 2005 opt/mysql.dmp.gz

When a DVD backup spans more than one disk, all the DVDs with the exception of the last one produce a file error during the verification process. This error message does not interfere with the backup process but can reappear several times as the backup spans multiple disks. 13 14 Go to the root directory
# cd /

Eject the DVD


# eject cdrom

If the DVD drive tray does not open after you have determined that the DVD drive is not busy and is not being read from or written to, enter the following commands in the order listed:
# /etc/init.d/volmgt stop

Then press the eject button located on the front of the DVD drive.
# /etc/init.d/volmgt start

15

Remove the DVD from the drive, label it, and store it in a safe place.

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

Variable definitions

159

16

To restore the data from DVD, refer to the procedure "Restoring the oracle data on an SPFS-based server" in Nortel ATM/IP Solution-level Administration and Security ( (NN10402-600)) .
--End--

Variable denitions
Variable <server> Value IP address or hostname of the SPFS-based server on which you want to perform the backup. In a two-server configuration, enter the physical IP address of the active server.

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

160

Preparation for disaster recovery

Backing up UAS conguration les


Back up UAS configuration files to support the operation of the UAS. For more information about UAS configuration files, see Job aid (page 161).

Procedure steps
ATTENTION
At the time of installation, the UAS is configured to automatically back up configuration files each day at 2:00 am. If an APS node is configured in the network, all UAS nodes in the network can be backed up to the APS node. If an APS node is not configured in the network, the configuration files for UAS nodes in the network can be backed up to a remote UNIX server instead. This procedure enables you to set up automatic backup to a remote server. At the Windows desktop interface

Step 1 2 3

Action Open the command interface, select Start -> Run then type cmd in the displayed window. Open a telnet session to the remote UNIX server and log in as the Root user. Enter the following commands to create a new backup directory
cd /;mkdir /opt;chmod 777 opt cd /opt; mkdir uas;chmod 777 uas cd uas; mkdir uas_conf_backup;chmod 777 uas_conf_backup cd / cd /opt/uas/uas_conf_backup

ATTENTION
The commands from this point forward are specific to a Sun Solaris system only.

Configure NFS to share the "/opt/uas" filesystem and start the NFS server
echo "share -F nfs /opt/uas" >> /etc/dfs/dfstab

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

Job aid /etc/init.d/nfs.server start

161

Create a user logon called "Administrator" that does not require a password:
/usr/sbin/useradd -d /export/home/Administrator -g 1 -s /bin/ksh -m -u 1002 Administrator 2> /dev/null passwd -d Administrator 2> /dev/null

At the local system console, enter the IP address of the remote server in the "Backup Storage IP" field of the Local Configuration Interface GUI screen, using the procedure "Modifying configuration parameters through the Local Configuration Interface GUI" in the document (NN10095-511), entitled "UAS Configuration Management."
--End--

Job aid
Back up UAS configuration files are stored in configuration files. The configuration files include:

uas.conf - containing configuration parameters that support the function of the UAS, including CG6000C card settings, Call Agent definition, APS hostname definition, network element settings, and conferencing service state definition ugw.conf - containing trunk configuration information for PRI Solutions snmpd.cnf - containing parameters that support the SNMP function, including management station address, SNMP user names, community names, and trap version hosts - containing parameters that support the function of the APS, including APS hostname and IP address atmhard.con - containing ATM bearer interface settings that link a local port ATM address to a particular ATM interface port atmconn.con - containing ATM bearer connections settings that provide the UAS with a remote gateways name and ATM address mainsa.conf - containing Main Subagent program settings specifying the kinds of error and log messages to be sent to the management station atmSvcProfile.con - containing data on Switched Virtual Channel (SVC) traffic parameters associated with AAL2 SVCs atmhardloop.con - containing information associated with the loopback of SVCs

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

162

Preparation for disaster recovery

Performing a manual backup of the Policy Controller database


Perform a manual backup of the policy controller database to make regular backup copies of the database or as a precautionary activity before starting any type of upgrade to the Policy Controller application. This procedure may be used as a standalone task or as part of a higher level upgrade activity.

ATTENTION
If you perform provisioning changes during this procedure, it can lead to database corruption and possible system outages. To ensure that you create an accurate and complete copy of the active unit database, verify that all provisioning changes are stopped before continuing this procedure.

Prerequisites
ATTENTION
The Policy Controller database and the CS 2000 database must always be in sync. CS 2000 core images, CS 2000 GWC Manager Oracle database backups, and the Policy Controller application backups must always be maintained in sync for emergency recovery. To ensure they are in sync, perform images and backups at the same time.

Procedure steps
Step 1 2 3 Action Log on to the active Policy Controller unit and change to the root user. Change directory to the database directory by typing
# cd /opt/apps/database/solid/dbfiles

ATTENTION
If other backup copies of the database exist with the same filename, you have the option of deleting those files or putting the backup copy into the backup directory under a new file name.

Put a copy of the database for the active unit in the backup directory by typing
# cp solid.db /opt/apps/database/solid/backup

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

Job aid

163

#cp solid.db /opt/apps/database/solid/backup/sol id.db.backup1


--End--

Job aid
For security purposes, ensure that a backup copy of the database file is transferred to a secure location. Use the UNIX scp command to make a secure copy of the backup database file to a secure, remote server on your network. This server should be continuously available for cases where a restoration of the Policy Controller application database become necessary, such as during an upgrade rollback. A root password for the remote server may be required.

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

164

Preparation for disaster recovery

Performing a backup of le systems on an SPFS-based server


Perform the following to execute a backup of the file systems on a SPFS-based server (Sun Netra 240). The server can be hosting one or more of the following components:

Prerequisites

Communication Server 2000 Management Tools Integrated Element Management System (IEMS) Audio Provisioning Server (APS) Media Gateway (MG) 9000 Manager Network Patch Manager Core Billing Manager (CBM)

For Sun Netra 240, use one or more blank DVD-R or DVD-RW disks to store the data. The backup utility limits the storage to 4 GB on a DVD-R and DVD-RW. If you are using a new DVD-RW, or want to reuse a used DVD-RW and need to erase the contents, complete procedure "Preparing a DVD-RW for use" in Nortel ATM/IP Solution-level Administration and Security ( (NN10402-600)) .

For Nortel Core and Billing Manager 850, you must be a user in a role group authorized to perform config-admin actions. For Nortel Core and Billing Manager 850, you must obtain non-restricted shell access.

Procedure steps
ATTENTION
In a two-sever configuration, execute this procedure on the Active server.

Step 1 2

Action Insert the blank DVD into the drive. In a two-server configuration, insert the blank DVD into the Active server. Log on to the server by typing > telnet <server>

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

Procedure steps

165

and pressing the Enter key. 3 4 When prompted, enter your user ID and password. Change to the root user by typing
$ su -

and pressing the Enter key. 5 When prompted, enter the root password In a two-server configuration, ensure you are on the active server by typing ubmstat. If ClusterIndicatorSTBY is displayed in the response, which indicates you are on the inactive server, log off of that server and log on to the other server through telnet using the physical IP address of the other unit. The response must display ClusterIndicatorACT, which indicates you are on the active server 6 Back up the file systems by typing
# /opt/nortel/sspfs/bks/bkfullsys

and pressing the Enter key. When using a DVD, the system will prompt you to insert another blank disk if more than one is needed. 7 When prompted, continue the backup by typing
yes

and pressing the Enter key. Example response


Backup completed. Please verify the data on the DVD using the cli command - select option for "Other", then "bklist" Note: This action does not include backup data inside the Oracle Database. Finish time: Thur Jan 4 11:03:46 HKT 2007. cleaning up files A backup of oracle data is recommended at this time. On a SPFS based server only hosting the CBM or MG 9000 Manager this is not required. Do you want to perform a backup of oracle data? Type "yes" to continue with oracle backup or "no" to skip:

ATTENTION
If you need to abort an in-progress backup, press Ctrl+c and wait for confirmation that the backup has been aborted.

When prompted, backup the Oracle database by typing


yes

and pressing the Enter key.


Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

166

Preparation for disaster recovery

9 10

Insert the backup DVD into the drive. If the backup resides on multiple DVDs, insert the first backup DVD. Access the command line interface by typing
# cli

and pressing the Enter key. 11 Enter the number next to the Other option. Example response
Other 1 - Log Rotation 2 - copylogs (Logviewer - Capture Various Logs For Debugging Purposes) 3 - logextractor (Logviewer - Extracting compressed log files captured by c...) 4 - logbrowser (Logviewer - Browse logs on the system) 5 - logviewer utility cleanup (Logviewer - Clean up space use by Logviewer ...) 6 - capt_rtes (Capture run-time environment for debugging) 7 - free_disk_space (Free Some Disk Space) 8 - sun_explorer (Execute the Sun Explorer Data Gathering Tool) 9 - mount_image (Mount A Generic Iso Image To The SSPFS unit) 10 - umount_image (Un-Mount A Generic Iso Image From The SSPFS Unit) 11 - bklist (List DVD backup archive) 12 - validate_pam_conf (Validate and/or fix module pointers in /etc/pam.conf...) 13 - set_disk_quota (Set maximum disk space allowed for a user)

12

List the content of the DVD by entering the number next to the bklist option. Example response
=== Executing "bklist" cdrin0: no media 15:34:24 NOTE: Please insert DVD volume 1, hit enter to continue, X to exit

If the content resides on multiple DVDs, the system will prompt you to insert each DVD. Example response
-r-xr-xr-x 0/2 209596 2002-03-14 09:29 usr/dt/bin/dtc alc -r-xr-xr-x 0/2 738032 2002-03-14 09:49 usr/dt/bin/dtcm 17:49:30 NOTE: End of volume

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

Procedure steps 17:49:30 NOTE: Ejecting DVD.. 17:49:36 NOTE: Checking DVD cdrom0: no media Please insert the next volume disk and hit x <enter> to abort: 17:52:56 NOTE: Waiting for DVD to mount -r-xr-xr-x 0/2 103616 2002-03-14 09:49 usr/dt/bin/dtc m_delete -r-xr-xr-x 0/2 644920 2003-06-19 02:38 usr/dt/bin/dtf ile

167

13

When prompted, press the Enter key to execute the command. When the command has completed successfully, you will see a system response. Example response
-rw-r--r-- 0/1 600 2007-02-09 16:27 .disk_conf/fslist. out 17:59:11 NOTE: Backup Verification Passed 17:59:11 NOTE: gtar command exited with:0 17:59:11 NOTE: End of backup archive listing ==== "bklist" execution completed

14

Ensure you are at the root directory level by typing


# cd /

and pressing the Enter key. 15 Eject the DVD by typing


# eject

and pressing the Enter key. If the disk drive tray will not open after you have determined that the disk drive is not busy and is not being read from or written to, enter the following commands:
# /etc/init.d/volmgt stop # /etc/init.d/volmgt start

Then, retry the eject cdrom command previously shown and press the Enter key. 16 Remove the DVD from the drive, label it, and store it in a safe place. The label should include the date of backup, disk number of backup, host name, and unit number. If the backup resides on multiple DVDs then insert the next backup DVD in the disk drive and go to step 15.
--End--

17

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

168

Preparation for disaster recovery

Variable denitions
Variable <server> Value The IP address or host name of the SPFS-based server on which you are performing the backup. In a two-server configuration, enter the physical IP address of the active server.

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

Procedure steps

169

Taking a full IEMS backup


Backup the IEMS software which includes the following:

A backup of the oracle database A backup of the servman critical system files A backup of the central security database and files

ATTENTION
Ensure that there is enough space in the /data partition for each of these backups before continuing.

Prerequisites

The user must have root user privileges to perform this procedure.

Procedure steps
Step 1 Action At your workstation establish a connection to the IEMS server through Telnet or SSH, using the host name or IP address of the server. In a two-server configuration, establish the connection to the active server using the physical IP address of the active server, and ensure you are on the active server using the ubmstat command. For detailed steps, refer to . 2 3 Log on using the root user ID and password. Select your next step.
If this server is hosting the CMT and IEMS this server is only hosting the IEMS Do the next step step 6

4 5 6

Verify the status of the SESM server application by entering:


servman query -status -group SESMService

If necessary, start the SESM server application by entering:


servstart SESMService

Enter the backup manager utility by entering:

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

170

Preparation for disaster recovery /opt/bkresmgr/cbm/bkmgr

7 8

Start the backup by entering:


backup full

Select your next step.


If the response is RC=success RC=fail <reason> Do step 11 step 9 and step 10

9 10 11

Unlock the provisioning session by entering:


backup abort

Contact your next level of support. Exit the backup manager utility by entering:
quit

The Security Services configuration settings and data are backed up to the following file: /data/bkresmgr/<date><time>backupSS2.0<host_name>.tar
--End--

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

Variable definitions

171

Backing up an SPFS-based security client


Backup an SPFS-based security client to obtain a list of the files that will be backed up from the client machine. To enable backup and restore of the security client, the files to be backed up are registered with servman during installation. The files backed up depend on the packages installed.

Prerequisites

You must have root user privileges to perform this procedure.

Procedure steps
Step 1 2 3 4 5 Action At your workstation, open A telnet session to the server by typing > telnet <server> When prompted, enter your user ID and password. Change to the root user by typing su - root When prompted, enter the root password. Enter the following command:
cat etc/critdata.conf

The system returns a list of all non-oracle data files that will be backed up from the client machine. 6 Use the Synchronous Backup Restore Manager (SBRM) to backup the central security client data. When SBRM is run, all of the SPFS data, including the security client data and oracle data, is backed up. For details, see .
--End--

Variable denitions
Variable <server> Value IP address or host name of the server where IEMS resides.

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

172

Preparation for disaster recovery

Backing up a Multiservice Switch or Media Gateway device


This section describes using Service Data Backup and Restore to perform a basic back up of service data. Backup copies only the configuration data of the Multiservice Switch 7400/15000 or Media Gateway 7400/15000. The service data views that make up the configuration data are found on the Multiservice Switch 7400/15000 or Media Gateway 7400/15000 disk in a set of files in individual directories under the directory /provisioning. Backup also copies the special files that are found under the directory /provisioning/netsentry. Service Data Backup and Restore provide three types of backup.

A full backup copies all service data on the selected device or devices. An incremental backup copies only service data changed or created since the last backup. Like the full backup, you can perform an incremental backup on either one or multiple devices. A selective backup copies specific service data that you select. You can perform a selective backup on either one or multiple devices.

The following information applies to using the Service Data Backup and Restore tool to backup Multiservice Switch 7400/15000 or Media Gateway 7400/15000 nodes:

Adding Multiservice Switch 7400/15000 or Media Gateway 7400/15000 devices to the backup list (page 173) Removing Multiservice Switch 7400/15000 or Media Gateway 7400/15000 devices from the backup list (page 173) Viewing the Multiservice Switch 7400/15000 or Media Gateway 7400/15000 backup repository (page 174) Defining a specific userid and password for a Multiservice Switch 7400/15000 or Media Gateway 7400/15000 in the backup list (page 174) Changing the Default User Authentication (page 175) Performing a full backup for a Multiservice Switch 7400/15000 or Media Gateway 7400/15000 device (page 175) Performing a selective backup for a Multiservice Switch 7400/15000 or Media Gateway 7400/15000 device (page 176)

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

Removing Multiservice Switch 7400/15000 or Media Gateway 7400/15000 devices from the backup list 173

Adding Multiservice Switch 7400/15000 or Media Gateway 7400/15000 devices to the backup list
Use this procedure to add devices to the list of devices that you wish to backup. Procedure steps Step 1 Action Open the Backup and Restore window. From the MDM window, select Configuration->Passport Devices->Administration->P assport Service Data Backup/Restore. Select the Backup Configuration tab. Select Add to launch the Add Devices Dialog. From the left pane, select a group of devices or expand the group and use the Ctrl key to select multiple devices within a group. From the drop down list, in the right pane, select a backup mode (Incremental or Full). If a specific userid and password is required for the device, enter the values in the user ID and Password fields and uncheck the Use default checkbox. If you wish to use the default userid and password, click the Use defaultcheckbox. Click OK. The devices are displayed in the Device List. 9 This procedure is complete.
--End--

2 3 4

5 6

7 8

Removing Multiservice Switch 7400/15000 or Media Gateway 7400/15000 devices from the backup list
Use this procedure to remove devices from the list of devices that you wish to backup. Procedure steps

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

174

Preparation for disaster recovery

Step 1

Action Open the Backup and Restore window. From the MDM window, select Configuration->Passport Devices->Administration->P assport Service Data Backup/Restore. Select the Backup Configuration tab. In the Device List, select the devices you wish to remove. Click Remove. In the confirmation dialog, select Yes to confirm or No to cancel the removal. This procedure is complete.
--End--

2 3 4 5 6

Viewing the Multiservice Switch 7400/15000 or Media Gateway 7400/15000 backup repository
If the backup server is running locally, use the following procedure to view all the backup files in the backup repository. Procedure steps Step 1 2 3 4 Action Telnet into the remote workstation with the appropriate userid and password. Navigate to the repository directory. Use Unix directory commands to view the contents of the backup repository. This procedure is complete.
--End--

Dening a specic userid and password for a Multiservice Switch 7400/15000 or Media Gateway 7400/15000 in the backup list
A specific userid and password can be defined when you add the node to the node list or you can set it later using the following procedure. Procedure steps

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

Performing a full backup for a Multiservice Switch 7400/15000 or Media Gateway 7400/15000 device 175

Step 1

Action Open the Backup and Restore window. From the MDM window, select Configuration->Passport Devices->Administration->P assport Service Data Backup/Restore. Select the Backup Configuration tab. Select a node in the Device List section. In the Device Details section, in the Authentication tab, enter a userid and password. Clear the Use defaultcheckbox. This procedure is complete.
--End--

2 3 4 5 6

Changing the Default User Authentication


Use the following procedure to define a default userid and password which is used for all node access unless overridden by a specific userid and password for the node. Procedure steps Step 1 Action Open the Backup and Restore window. From the MDM window, select Configuration->Passport Devices->Administration->P assport Service Data Backup/Restore. Select Options ->Set default authentication. Enter the userid and password in the appropriate fields. Click OK. The new userid and password are used on the next node access. 5 This procedure is complete.
--End--

2 3 4

Performing a full backup for a Multiservice Switch 7400/15000 or Media Gateway 7400/15000 device
Use this procedure to perform a complete system backup of a Multiservice Switch 7400/15000 or Media Gateway 7400/15000 device.

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

176

Preparation for disaster recovery

Procedure steps Step 1 Action Open the Backup and Restore window. From the MDM window, select Configuration->Passport Devices->Administration->P assport Service Data Backup/Restore. Select the Backup Configuration tab. Select Add to launch the Add Devices Dialog. From the left pane, select a group of devices or expand the group and use the Ctrl key to select multiple devices within a group. Click in the Mode title and selectFull from the drop-down list. Click Backup. The progress of the backup is displayed in the Messages area. If the backup is unsuccessful, an error dialog is displayed that specifies the device and the reason for the failure. Note: To cancel a backup in progress click Cancel. 7 This procedure is complete.
--End--

2 3 4

5 6

Performing a selective backup for a Multiservice Switch 7400/15000 or Media Gateway 7400/15000 device
Use the following procedure to backup only a single specified file to the repository. Procedure steps Step 1 Action Open the Backup and Restore window. From the MDM window, select Configuration->Passport Devices->Administration->P assport Service Data Backup/Restore. Select the Backup Configuration tab. Select Add to launch the Add Devices Dialog. From the left pane, select a group of devices or expand the group and use the Ctrl key to select multiple devices within a group.

2 3 4

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

Performing a selective backup for a Multiservice Switch 7400/15000 or Media Gateway 7400/15000 device 177

5 6 7

Select a device from the Devices list. In the Mode title, select Selective from the drop-down list. The Configuration column in the table is enabled. Click in the Configuration cell to display a pull-down list that displays all the available views on the Multiservice Switch 7400/15000 or Media Gateway 7400/15000. Note: This step may take a few seconds to complete because the application must access the Multiservice Switch 7400/15000 or Media Gateway 7400/15000 and list all the views names.

Click Backup. The progress of the backup is displayed in the Status area. If the backup is unsuccessful, an error dialog is displayed that specifies the device and the reason for the failure. Note: To cancel a backup in progress click Cancel.

This procedure is complete.


--End--

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

178

Preparation for disaster recovery

Performing a backup operation


Perform this procedure to execute a backup operation. If you change the external IP address of an Real-time Controller (RTC) system node in your system (or perform any other data provision changes on your system), you should immediately perform a backup operation and delete any data snapshots that were made before you changed the IP address. This ensures proper communication with your OAMP workstations.

Procedure steps
At the OAMP workstation

Step 1 2 3

Action Click Administration>backup. Enter a description of the data snapshot in the snapshot-descri ption box. You can enter up to 32 characters. Click the corresponding check box to activate the transfer snapshot to alternate boot server Nortel recommends that you enable this option. If you dont select this option, a warning dialog box appears. Click Yes to continue.

Click the corresponding checkbox to activate the set as active snapshot on alternate boot server Nortel recommends that you enable this option. If you dont select this option, a warning dialog box appears. Click Yes to continue.

5 6 7 8 9 10 11

Click the corresponding checkbox to activate the delete oldest snapshot on alternate boot server if maximum reached (16) Click the corresponding checkbox to activate the delete oldest snapshot on rtc if maximum reached (5) Click Create to create the data snapshot. This can take several minutes, depending on system activity. Click Close to close the Backup Active RTC window and return to the Administration window. If you selected transfer snapshot to alternate boot server on alt boot server, go to step 15. Select Administration>file-manager. The File Manager window appears. Select the active RTC system node from the Destination list in the right window pane.

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

Job aid

179

12

Select the new data snapshot from the Snapshot box in the Destination portion of the window. Make note of the timestamp of the data snapshot. Copy the snapshot to your alternate boot server by clicking <--. An hourglass appears while the data snapshot is copied. The boxes in the Source portion (left window) of the window are updated with the information for the copied data snapshot when the copy operation is complete. These files are large and can take several minutes to copy from an RTC system node to your alternate boot server.

13

14 15 16

When the file is transferred, click Close. Update the Alternate Boot Snapshot using the ABS Settings ABS window: Open the Alternate Boot Server (ABS) Configuration Manager window by clicking ABS Settings on the Administration>alternate-boot-server window. Select an alternate snapshot by clicking the arrow button. Click Modify to save your changes and close the window. Your backup is complete.
--End--

17 18 19

Job aid
The same backup procedure may be performed through the CLI.

Admin backup create syntax


"snapshot-description" abs-transfer-option abs-active-option rtc-delete-oldest-option abs-delete-oldest-option where: snapshot-description can be up to 32 characters and the other four options are set by entering either y (for enabled) or n (for disabled). See Performing a backup operation (page 178) for option descriptions. The following example creates a snapshot named my snapshot, transfers it to the ABS and sets it as active. It also deletes the oldest snapshot on the rtc or abs, if the maximum number of snapshots is reached on either: admin backup create "my snapshot" y y y y The following table lists CLI return codes.

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

180

Preparation for disaster recovery Table 15 CLI return codes Rcode 0x40100001 0x40100002 0x40100003 0x40100004 0x40100005 0x80100001 0x80100002 0x80100003 0x80100004 0x80100005 0x80100006 0x80100007 0x80100008 0x80100009 0x8010000A Rcode text snapshot-description must be string, length (1 - 32), no additional character filter rtc-delete-oldest-option must be string, length (1 - 1), boolean (Y, y, N, n) abs-transfer-option must be string, length (1 - 1), boolean (Y, y, N, n) abs-active-option must be string, length (1 - 1), boolean (Y, y, N, n) abs-delete-oldest-option must be string, length (1 - 1), boolean (Y, y, N, n) missing snapshot-description insufficient disk space on rtc maximum number of snapshots reached on rtc maximum number of snapshots reached on abs unable to delete oldest snapshot on rtc unable to delete oldest snapshot on abs unable to transfer snapshot to abs unable to set snapshot as active on abs unable to connect to abs, run abs test for details invalid combination of command options

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

Prerequisites

181

Creating system image backup tapes (S-tapes) manually


Use this procedure to create a system backup image manually. Create a system image backup tape (S-tape) manually.

ATTENTION
If you want to schedule automatic system image backups, see SDM Security and Administration document.

The system image includes the following:

Prerequisites

boot (startup) files AIX operating system system configuration data SDM CS 2000 Core Manager software

ATTENTION
This procedure must be performed ONLY from a local console by a trained AIX system administrator with root user privileges.

ATTENTION
All volume groups on the SDM CS 2000 Core Manager must be fully mirrored (Mirrored) before performing this procedure. If not, an error message is displayed.

ATTENTION
If your system includes the SuperNode Billing Application (SBA), you must use tape drive DAT0 to perform this procedure.

ATTENTION
The files under the /data file system are temporary files only, and are excluded from system image backup.

Perform a system image backup after the following events:

initial installation and commissioning of the SDM CS 2000 Core Manager changes to the configuration of disks or logical volumes

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

182

Preparation for disaster recovery

installation of a new version of SDM CS 2000 Core Manager platform software installation of new hardware changes or upgrades to existing hardware

A system image backup takes a minimum of 10 minutes to complete, depending on the size of your file systems.

Recommended tapes
To complete this procedure, use one of the digital audio tape (DAT) drive tapes approved by Nortel. The brands approved by Nortel are: Hewlett Packard (HP), Maxell, Verbatim, Imation. The tape lengths approved by Nortel are:

90-meter (90M) 125-meter (125M) 120-meter (120M)

The 125M tape is approved for UMFIOs only, assuming that your system is equipped with DDS3-capable devices to read the content of the tape.

Procedure steps
The following flowchart provides an overview of the procedure. Use the instructions in the procedure that follows the flowchart to perform the procedure.

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

Procedure steps Figure 13 Summary of creating system image backup tapes (S-tapes)

183

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

184

Preparation for disaster recovery

Creating system image backup tapes (S-tapes) At the local VT100 console

Step 1 2

Action Log on to the SDM CS 2000 Core Manager as the root user. Access the maintenance interface:

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

Procedure steps # sdmmtc

185

CAUTION
System mirroring must be MIRRORED
You cannot perform this procedure until disk mirroring of all volume groups is Mirrored. If necessary, contact the personnel responsible for your next level of support. When disk mirroring is Mirrored, continue this procedure.

Access the storage menu level:


> storage

Example response:
Volume Group Status Free(MB) rootvg Mirrored 608 Logical Volume Location Size(MB) %full/threshold 1 / rootvg 20 25/ 80 2 /usr rootvg 192 85/ 90 3 /var rootvg 64 11/ 80 4 /tmp rootvg 24 6/ 90 5 /home rootvg 300 4/ 70 6 /sdm rootvg 300 44/ 90 Logical volumes showing: 1 to 6 of 6

4 5 6 7

If the disks are "Mirrored" then go to step 6 If the disks are not "Mirrored" then contact next level of support Access the administration (Admin) menu level of the RMI:
> admin

Access the System Image Backup and Restore Menu:


> backup

Example response:
Currently there is a backup running on bnode73.Please execute yours later.Exiting . . .

ATTENTION
If another operator attempts to use the Backup and Restore utility when it is in use, an error message is displayed.

From the System Image Backup and Restore Menu, select Create a System Image on Tape (S-tape):
>2

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

186

Preparation for disaster recovery

After you select option 2, you are prompted to select the tape drive. Example response:
Select the tape drive you wish to use: Enter Enter Enter ( 0, 1 0 to return to previous menu 1 for tape drive DAT0 in Main Chassis-Slot 2 2 for tape drive DAT1 in Main Chassis-Slot 13 or 2 ) ==>

ATTENTION
Use tape drive DAT0 (option 1) if your system also includes SBA.

Select the tape drive to use:


> <n>

ATTENTION
If your system includes SBA, and you wish to use tape drive DAT1 (option 2), the following message is displayed:

Example response:
You have selected DAT 1. This is the default DAT drive for the Billing application, and may currently be in use for the emergency storage of billing records. If you continue to use DAT 1, make sure that the correct tape is in the drive, and that billing records will not be lost during the backup restore operation. Do you wish to continue with DAT 1? ( y | n )

If you wish to continue using DAT1 then do enter y press the Enter key If you do not wish to use DAT1 then do enter n press the Enter key The system prompts you to return to the System Image Backup and Restore Menu if you do not wish to use DAT1. After you select the tape drive, you are prompted to insert a tape in the drive you have selected. Example response:
Please insert a 4mm DAT tape into the tape drive DAT0. Caution: This action will overwrite the content on the inserted tape.Do you want to proceed? ( y | n ) ==> At the SDMCS 2000 Core Manager

10

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

Procedure steps

187

CAUTION
System image backup tape
Creating a system image overwrites the contents of the inserted tape. Ensure that you are using the correct tape before starting the system image backup. If your system includes SBA and you are using DAT1, ensure that the tape drive does not contain an SBA tape.

Ensure that the appropriate SDM CS 2000 Core Manager tape drive contains a 4-mm digital audio tape (DAT) either 90 m or 120 m long. This tape will be designated as the system image backup tape (S-tape).

ATTENTION
For the complete list of approved tapes, see the Recommended tapes (page 182). At the local console

11 12

When you are certain you are using the correct tape, enter:
>y

Read the system message to determine if there is enough room on the temporary directory for the system image backup to proceed.

ATTENTION
If there is not enough room on the temporary directory, an error message appears.

Example response:
Rewinding the tape... The /tmp directory is not big enough. Trying to expand /tmp by 6600KB... Failed to expand the /tmp directory because there isnt enough free disk space left on the rootvg. Please erase some files under /tmp directory to create at least 6600KB for the full system image backup. Enter any key and return to exit ==>

13 14 15

If there is enough disk space then do step 14 If there is not enough disk space then do step 11 Erase enough files from the temporary directory to create the required amount of disk space specified in the error message:
> rm -rf /tmp/<filenames>

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

188

Preparation for disaster recovery

ATTENTION
If you have trouble erasing files from the temporary directory to free up disk space, contact the personnel responsible for your next level of support.

16

Execute the system image backup again. The system image backup begins. Example response:
Rewinding the tape... Starting the system image backup on bnode73. The backup takes a minimum of 10 minutes, depending on the size of your file systems. When the backup is complete, you will be asked to remove the tape from the tape drive. System image backup is in progress ...

ATTENTION
This backup process takes approximately 10 minutes to complete, depending on the amount of data stored on the disk.

17 18 19 20

Read the system message. If the backup is successfully completed then do step 16 If the backup fails then do contact your next level of support The system informs you if the backup is successful. When the backup is complete, the system prompts you to remove the tape and label it as an S-tape. Example response:
The tape backup started on Wed Oct 16 08:21:15 EDT 1997 completed successfully on Wed Oct 16 08:37:37 EDT 1997. A log file /tmp/s-tape.log has been created. Please remove the backup tape from the tape drive. Label the tape as shown below and store it in a safe place. System Image Tape (S-tape) The Machine Node Id: bnode73 Date: Wed Oct 16 08:37:37 EDT 1997 Eject the S-tape from the tape drive? ( y | n ) ==>

21

Determine if you wish to eject the S-tape. Enter

y to eject the tape, or n if you do not wish to eject the tape, and wish to list its contents.

If you wish to list the S-tape then do step 28

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

Procedure steps

189

If you protect and label the tape then do step 16 If you eject the tape, the screen displays "Tape ejected." below the information displayed in step 14. You are then prompted to return to the System Image Backup and Restore Main Menu. Response:
Tape ejected. Would you like to return to the previous menu? ( y | n)

22 23

Place the write-protected tab of the S-tape in the open position, to prevent accidental erasure. When you are ready for the system to return to the System Image Backup and Restore Main Menu, enter
>y

24

Determine if the backup is successful. The system informs you if the system image backup is successful, as shown in the response in step 14. You may also wish to view the s-tape.log file or list the files on the S-tape.

25 26 27 28 29 30 31 32

If you wish to view the s-tape.log file then do step 19 If you wish to list the S-tape then do step 28 If the backup is successful then do step 38 If the backup fails then do contact your next level of support Exit the System Image Backup and Restore Main Menu:
>0

Exit the RMI:


> quit all

Access the s-tape.log file:


# cd /tmp

Scroll through the file:


# more s-tape.log

This screen informs you that the system image backup was completed successfully. Example response:
bosboot: Boot image is 5881 512 byte blocks. 0+1 records in. 1+0 records out. Backing up the system... ..................................................... ..... 0512 038 mksysb: Backup Completed Successfully.

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

190

Preparation for disaster recovery The S-tape backup started on Wed Oct 16 09:24:07 EDT 1997 completed successfully on Wed Oct 16 09:36:03 EDT 1997

33 34 35 36 37 38 39 40

Determine if you wish to list the S-tape. If you wish to list the S-tape then do step 24 If you do not wish to list the S-tape then do step 40. Return to the login directory:
# cd

Access the RMI:


# sdmmtc

Access the administration (Admin) menu level of the RMI:


> admin

Access the System Image Backup and Restore Menu:


> backup

From the System Image Backup and Restore Menu, select List Contents of the System Image Tape (S-tape):
>3

41

After you select option 3, you are prompted to select the tape drive. Example response:
Select a tape drive you wish to use: Enter Enter Enter ( 0, 1 0 to return to previous menu 1 for tape drive DAT0 in Main Chassis-Slot 2 2 for tape drive DAT1 in Main Chassis-Slot 13 or 2 ) ==>

ATTENTION
Use tape drive DAT0 (option 1) if your system also includes SBA.

42

Select the tape drive:


> <n>

Example response:
You have selected DAT 1. This is the default DAT drive for the Billing application, and may currently be in use for the emergency storage of billing records. If you continue to use DAT 1, make sure that the correct tape is in the drive, and that billing records will not be lost during the backup restore operation. Do you wish to continue with DAT 1? ( y | n )

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

Procedure steps

191

If you do not wish to use DAT1, the system prompts you to return to the System Image Backup and Restore Menu. 43 44 If you wish to continue using DAT1 then enter
y

If you wish to not continue then enter


n

ATTENTION
If your system includes SBA, and you still wish to use DAT1 (option 2), the following message is displayed:

45

After you select the tape drive, you are prompted to insert the S-tape into the tape drive that you selected in step 30. Example response:
Please insert your System Image Backup tape (S-tape) into the tape drive DAT0 and allow at least 5 minutes to complete the listing. A log file will be saved in /tmp/s-tape.toc. Are you ready to proceed? ( y | n )

At the SDM CS 2000 Core Manager

46

Insert the S-tape into the tape drive.

ATTENTION
Wait until the tape drive stabilizes (yellow LED is off) before you proceed. At the local VT100 terminal

47 48

When you are ready to continue this procedure, enter:


>y

The contents of the S-tape are displayed. When the listing is complete, the system prompts you to return to the System Image Backup and Restore Menu. Example response:
Would you like to return to the previous menu? ( y | n )

49

Return to the System Image Backup and Restore Menu:


>y

At the CS 2000 Core Manager

50 51 52

If you have not already done so, remove the S-tape from the tape drive by pressing the eject button on the tape drive. Label the tape according to your office practices, and store it in a safe location. If you ejected an SBA tape, reinsert the tape.

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

192

Preparation for disaster recovery At the local VT100 terminal

53

Exit the System Image Backup and Restore Menu,:


>0

ATTENTION
If you wish to exit the RMI, enter QUIT ALL.

--End--

Variable denitions
Variable <n> Value is the option (1 or 2) for the tape drive you wish to use

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

Procedure steps

193

Displaying actions a user is authorized to perform


Display the actions a user in a particular role group is authorized to perform.

Prerequisites

You must be authorized to perform security-view actions.

Procedure steps
Step 1 Action At the local or remote VT100 console Log into the core manager as a user authorized to perform security-view actions. 2 3 4 Access the maintenance interface, type sdmmtc Access the Admin level, type admin Access the User level, type user A list of users and authorization groups is displayed. 5 Display the actions a user in a particular role group is authorized to perform, type dispgrpr <group> act Sample response for the secadm group Authorized actions for secadm security-admin security-manage security-view 6 Exit the maintenance interface, type quit all
--End--

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

194

Preparation for disaster recovery

Variable denitions
Variable <group> Value Value can be: maint, secadm, or emsadm

Job aid
Role groups allow you to group individual users according to the task each user has to complete. The following table lists the standard actions for each role group in a standalone system.
Role group emsadm Standard action Authorized to perform these actions: fault-view, fault-manage, fault-admin, accounting-admin, config-view, config-manage, config-admin, accounting-view, accounting-manage, accounting-admin, performance-view, performance-manage, performance-ad min Gr740Oss Authorized to perform these actions: performance-manage maint Authorized to perform these actions: fault-view, fault-manage, config-view, config-manage, accounting-view, acc ounting-manage, performance-view, performance-manage passthru root (user) secadm No authorized actions. Authorized to perform all actions. Authorized to perform these actions: security-view, security-manage, security-admin

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

Procedure steps

195

Logging in to the Nortel CS 2000 Core Manager


Use this procedure in a standalone system to log in to the Nortel CS 2000 Core Manager. For a complete description of logon methods, see Nortel CS 2000 Core Manager Basics, (NN10018-111).

Prerequisites

You must be a user authorized to perform actions associated with the procedure. It is recommended that you log in to the CS 2000 Core Manger through SSH (secure shell) using a password.

Procedure steps
At the local or remote VT100 console

Step 1

Action Log in to the CS 2000 Core Manager using one of the following commands for SSH access:
ssh <userID>@<IPaddress | hostname>

or ssh -l <userID> <IPaddress | hostname> Example response:


Don_secus Password:

2 3

Enter your password. Access a user interface, for example, access the maintenance interface:
sdmmtc

Exit the maintenance interface:


quit all
--End--

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

196

Preparation for disaster recovery

Variable denitions
Variable <userID> <IPaddress> <hostname> Value is your user ID is the IP address of the CS 2000 Core Manager is the host name for the CS 2000 Core Manager

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

General stream information

197

Preparation for SBA installation and conguration


The majority of procedures in this document require you to fill out these questionnaires. Check the prerequisites of each procedure to determine if you require information from this section.

General stream information


The following table contains a list of questions that concern general stream information. Record your answers in the spaces provided.
Table 16 General stream information # 1 Question What is the name of this stream? Explanation stream_name The stream name on the SBA must match the stream name on the DMS Switch. This name must match a stream name in the CM table CRSFMT. Type: string Range: 1 to 4 characters Example: AMA Example: GCDR (not case sensitive) 2 Enter total traffic rate at engineered capacity (ENGCAP) in million BHCA for the total billing traffic rate including the data and voice requirements. Enter Number of retention days See the rest of this section for more information. Answer

The number of days for storing the billing files in the file system and within the file system capacity for the current new stream.

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

198

Preparation for disaster recovery

Table 16 General stream information (contd.) # 4 Question What is the associated stream name? Explanation associated_stream This question applies only to filter streams. The associated stream name parameter specifies the name of the associated CM billing stream. Type: string Range: 1 to 4 characters Example: AMA, OCC (not case sensitive) 5 What is the name of the Filter Criteria file? filter_criteria_file This question applies only to filter streams. Enter the filter criteria file name that contains the expression to be applied for the filtered stream. Type: string Range: 1 to 255 characters (case sensitive) 6 What is the record format of this stream? record_format The stream record format on the SBA must match the record format of the DMS Switch stream. The only record formats supported by this product and release are Answer

BC (Bellcore AMA format) SMDR (Station Message Detail Recording) CDR300 CDR250

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

General stream information

199

Table 16 General stream information (contd.) # Question Explanation Type: enumeration Range: BC, SMDR, CDR300, CDR250 (not case sensitive) 7 What is the file format of this stream? file_format This is the format of the billing files that SBA creates on the core manager. Type: enumeration Range: DNS, DIRP (not case sensitive) 8 What is the name of the logical volume on the core manager for storing the billing files for this stream? logical_volume_name The logical volume is the name of the directory where the billing files are stored for this stream. Type: string Range: 1 to 255 characters 9 Will file transfers for this stream be initiated by file_transfer_mode Billing files always move from SBA to the downstream destination, but the file transfers can be initiated by Answer

SBA (outbound) the downstream destination (inbound)

SBA (outbound) the downstream destination (inbound)

If Outbound is chosen, the SBA must be configured with additional file transfer information. The outbound file transfer questionnaires must be completed. If Inbound is chosen, the outbound file transfer questionnaires are not needed. Type: enumeration Range: Inbound, Outbound Default: Outbound (not case sensitive)

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

200

Preparation for disaster recovery

Table 16 General stream information (contd.) # 10 Question What is the desired state for the stream? Explanation sba_stream_state The stream state controls where the records are sent. Answer

ON: records are sent only to the SBA OFF: records are sent only to an existing DIRP system BOTH: records are sent to both SBA and to an existing DIRP system

The BOTH state is intended for startup verification of SBA processing against DIRP processing. Extended use of the BOTH state can result in SBA performance problems. A system generating more than 175000 CDRs per hour does not support BOTH or OFF mode. Type: enumeration Range: On, Off, Both (not case sensitive) 11 Do you want the files renamed with close date? files_renamed_with_ close_date This question applies only if the file format is DIRP. Type: Boolean Range: Yes, No Default: No (not case sensitive) 12 Do you want to reset DIRP sequence number at midnight? This question appears only when you answer No to the question, "Do you want the files renamed with close date?" Reset_DIRP_sequence_number _ at_midnight This parameter enables resetting the DIRP sequence number to zero after midnight, before opening a new billing file. Type: Boolean Range: Yes, No Default: No (not case sensitive)

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

General stream information

201

Table 16 General stream information (contd.) # 13 Question Do you want the files closed for file transfer and writetape? Explanation files_closed_on_file_ transfer This question applies only if the file format is DIRP. Type: Boolean Range: Yes, No Default: No (not case sensitive) 14 Do you want DIRP blocks closed based on time (applicable only for DIRP file format)? This question appears only when file_type=DIRP and record_format=BAF or CDR250. DIRP_blocks_closed_ based_on_time This parameter specifies whether the DIRP blocks are to be closed after a defined elapsed time. SBA block flushing does not support customized DIRP file formats that do not allow hex AA padding at the end of a block. This type of DIRP file expects CDRs to be of equal size, and each block ends with a special event record. Therefore, GSP and MCI CDR DIRP files are not supported. It is recommended that block flushing be used with real-time transfer mechanisms such as Real-Time Billing (RTB). Type: Boolean Range: Yes, No Default: No (not case sensitive) 15 File DIRP block closure time limit (in seconds) This question appears only when you answer Yes to DIRP_ blocks_ closed_base d_on_time (question 13) DIRP_block_closure_ time_limit This parameter specifies the maximum amount of time in seconds that a DIRP block is kept open before it is closed. Type: Integer Range: 1 through 120 Default: 1 Answer

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

202

Preparation for disaster recovery

This section contains a series of questionnaires that you must complete before you configure the SuperNode Billing Application (SBA) on the core manager for the first time. The following table contains a list of questions concerning general stream information. Record your answers in the spaces provided.
Table 17 General stream information # 1 Question What is the name of this stream? Explanation stream_name The stream name on the SBA must match the stream name on the DMS Switch. This name must match a stream name in the CM table CRSFMT. Type: string Range: 1 to 4 characters. (not case sensitive) 2 What is the desired disk utilization? desired_disk_utilization The percentage of disk utilization that should be maintained under normal conditions. The billing application will attempt to keep the disk utilization for the volume containing this billing data. Type: integer Range: 1090 3 Will file transfers for this stream be initiated by file_transfer_mode Billing files always move from SBA to the downstream destination, but the file transfers can be initiated by Answer

SBA (Outbound), or the downstream destination (Inbound)

SBA (this is called outbound) or the downstream destination (this is called inbound)

If Outbound is chosen, the SBA must be configured with additional file transfer information. The outbound file transfer questionnaires must be completed.

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

General stream information

203

Table 17 General stream information (contd.) # Question Explanation If Inbound is chosen, the outbound file transfer questionnaires are not needed. Type: enumeration Range: Inbound, Outbound Default: Outbound (not case sensitive) 4 What is the file format of this stream? file_format This is the format of the billing files that SBA creates on the core manager. Type: enumeration Range: DIRP (not case sensitive) 5 Do you want the file to be rotated on a regular schedule What do you want the primary file rotation schedule to be? Value: Yes or No Answer

file_rotation_schedule 1. Rotates files every 24 hours daily (that is, close files at midnight each day) 2. Rotates files every 12 hours daily (that is, close files at noon and midnight each day) 3. Rotates files every 6 hours daily (that is, close files at 6 am, noon, 6 pm and midnight each day) 4. Rotates files every hour on the hour daily (that is, close files at 1 am, 2 am, 3 am, daily etc.) Type: integer Range: 14

Do you want file sequence number to be reset at midnight? Do you want to close the file on record criteria?

Value: Yes or No

The number of records, or size in bytes.

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

204

Preparation for disaster recovery

AMADNS lename and header values


The following table contains a list of configuration questions concerning AMADNS filename and header values. The values selected are used in the headers and names of the AMADNS files that SBA creates for this stream. Record your answers in the spaces provided. The source component id and type are not configured per stream and their values will be used by every enabled AMADNS stream on this SBA.
Table 18 AMADNS filename and header values # 16 Question What is the destination component ID for this stream? Explanation destination_id Type: String Range: 0000 to 4095 Default: 0002 17 What is the destination component type for this stream? destination_type Type: String Range: 01 to 15 Default: 03 18 What is the source component id for this SBA? source_id Type: String Range: 0000 to 4095 Default: 0001 19 What is the source component type for this SBA? source_type Type: String Range: 01 to 15 Default: 02 20 What is the standard file type for this stream? standard_file_type Type: Number Range: 1, 6 to 31 Default: 1 (BC), 11 (SMDR) 21 What is the error file type for this stream? error_file_type Type: Number Range: 1, 6 to 31 Default: 2 (BC), 12 (SMDR) Answer

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

File closure limits

205

File closure limits


The following table contains a list of configuration questions concerning limits that control automatic closing of billing files by SBA. Note that the first of these settings that are reached, triggers the closing of the file. Record your answers in the spaces provided.
Table 19 File closure limits # 22 Question Do you want the files for this stream to close after a defined elapsed time? Explanation close_on_timer This controls if SBA closes billing files based on how long the files have been open. If the answer is Yes, SBA leaves a file open no longer than the value specified in question #22. If the answer is No, skip question #22 and go to question #23. Type: Boolean Range: Yes, No Default: No (not case sensitive) 23 What is the maximum time that a file can be open for this stream? file_open_time_limit This controls the maximum time SBA keeps a file open. It is enabled only if Yes is the answer to question #21. If the answer to question #21 is Yes, enter the maximum time that a file can be open for this stream, then go to question #25. If the answer to question #21 is No, skip this question and go to question #23. Type: number Units: minutes Range: 5 to 10080 Default: 10080 Answer

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

206

Preparation for disaster recovery

Table 19 File closure limits (contd.) # 24 Question Do you want Files closed at scheduled intervals from midnight? Explanation file_closed_at_scheduled_i ntervals_from_ midnight The response to this prompt determines if SBA closes billing files at scheduled intervals calculated relative to midnight. If the answer to this question is Yes, you are prompted to choose options, as shown in question #24. If the answer is No, skip question #24 and go on to question #25. Type: Boolean Range: Yes, No Default: No (not case sensitive) 25 What is the scheduled file closure time option for this Stream? scheduled_file_closure_tim e_option Choose a time in response to a prompt to determine when a billing file is closed, if you answer Yes to question #23. Ignore this question if the answer to question #23 is No. Options: 1) Close billing files every 2) Close billing files every 3) Close billing files every 4) Close billing files every 5) Close billing files every 6) Close billing files every 7) Close billing files every 8) Close billing files every 9) Close billing files every Type: number Range: 1 through 9 Default: 5 Answer

24 hours 12 hours 6 hours 2 hours hour 30 minutes 15 minutes 10 minutes 5 minutes

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

File closure limits

207

Table 19 File closure limits (contd.) # 26 Question What is the maximum number of records generated each day for this stream? Explanation records_per_day This is used to calculate the maximum number of Answer

records per file, and bytes per file

Type: number Units: Records per day 27 What is the maximum size of a record? bytes_per_record This is used to calculate a value for the maximum number of bytes per file. Type: number Units: Bytes per record Range: none 28 What is the maximum number of records per billing file for this stream? records_per_file This controls the maximum number of records a billing file can contain before SBA automatically closes the file. The recommended value based on a target of 300 files a day will be calculated and provided as the default value, if the average number of records per day is one or more. Type: number Units: records per file Range: BC 10000 to 50000016384 00 SMDR 1000 to 5000001638400 29 What is the maximum number of bytes per billing file for this stream? bytes_per_file This controls the maximum size (in bytes) of a billing file before SBA automatically closes it. A recommended value is calculated with the following formula: Records per day * average record size / 300 = Bytes per file

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

208

Preparation for disaster recovery

Table 19 File closure limits (contd.) # Question Explanation Type: number Units: bytes per file Range: BC:1000000 to 104857600 SMDR: 100000 to 104857600 30 What is the average record size? (not applicable if the number of records per day is 0) average_record_size This parameter specifies the maximum size of a record. The default value is 80, but depends on the record type and the record size as defined on the CM. This prompt appears when the Number of records per day parameter is set to a value other than zero (0) Answer

Disk space requirements


The following table contains a list of configuration questions related to core manager and DMS-switch disk space required by the SBA. Record your answers in the spaces provided. Disk space sizing requirements are calculated using the DMS switch value billable Busy Hour Call Attempts (BBHCA). This value is the total number of billing-record-generating calls that are processed within the busiest one hour window of a typical 24-hour day. For information on the BBHCA estimation factor and its use in calculating required disk space, see Calculation of core manager Disk Space Requirements (page 209) and Calculation of Disk Space Requirements (page 210).
Table 20 Disk space requirements # 31 Question How much disk space on the core manager is needed for the billing files for this stream? Explanation logical_volume_size If the core manager is unable to send the billing files to the downstream processor, they accumulate on the core manager disk space. The allocated disk space must be capable of holding at least 5 days of SBA billing files. Answer

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

Disk space requirements

209

Question

Explanation The formula for calculating SBA-required disk space on the core manager is described in Calculation of core manager Disk Space Requirements (page 209). Type: number Units: Mbytes Range: NA Default: none Space is allocated in 16 Mb increments.

Answer

32

How much disk space is needed for backup of billing records on the DMS Switch for this stream?

dms_disk_space If the DMS switch is unable to send the billing records to the core manager, they are backed up to the DMS disk space. The allocated DMS disk space must be capable of holding at least a one day accumulation of SBA billing records. The formula for calculating SBA-required disk space on the DMS switch is described in Calculation of Disk Space Requirements (page 210). Type: number Units: Mbytes Range: NA Default: none

Calculation of core manager Disk Space Requirements


The formula for calculating megabytes of disk space needed for SBA billing streams is: BBHCA * ALCR * 10 hours * CRRD --------------------------------------------------/ disk utilization 1048576

BBHCA (Billable busy hour call attempts), multiplied by the ALCR ALCR (average length of a call record in bytes), multiplied by 10 hours, multiplied by CRRD (Call-record retention days), divided by 1048576 (the number of bytes in a megabyte), divided by the desired disk utilization.

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

210

Preparation for disaster recovery

For this calculation, the desired disk utilization is a percentage that is expressed as a decimal from 0.1 and 0.9. This formula must be applied to each billing stream with the total of all streams representing the total megabytes of disk space required. The maximum number of files to hold billing records for a billing stream is 15,000. The maximum number of files in closedSent directory is 4,000. The calculation of 10 hours multiplied by BBHCA is an experience-based factor that can be used to estimate 24 hours of call traffic. If you know that the stream has a high BBHCA for more or less than 10 hours per day, increase or decrease the hours value. Calculation Example Assumptions:

BBHCA = 150000 Average length of call records = 85 bytes Call retention days = 10 Desired disk utilization = 0.6 (60%)

Calculation: 150000*85*10*10/1048576/.6=2026 Megabytes (2 Gbytes)

Calculation of Disk Space Requirements


Regardless of the volume size determined in this procedure, VSE users cannot configure a backup volume size greater than 2GB. For non-VSE users, the maximum volume size that can be configured is limited to the size of the physical disk. BBHCA * ALCR * 10 hours * CRRD

BBHCA (Billable busy hour call attempts) multiplied by ALCR (Average length of a call record in bytes), multiplied by 10 hours, multiplied by CRRD (Call-record retention days)

This formula must be applied to each billing stream with the total of all streams representing the total DMS Switch disk space required. The calculation of 10 hours multiplied by BBHCA is an experience-based factor that can be used to estimate 24 hours of call traffic. If you know that the stream has a high BBHCA for more or less than 10 hours per day, increase or decrease the hours value.

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

Outbound file transfer destinations 211

Calculation Example Assumptions:

BBHCA = 150000 Average length of call records = 85 bytes Call retention days = 2

Calculation: 150000*85*10*2/(1024*1024) = 243 Mbytes of disk space

Outbound le transfer destinations


The following table contains a list of stream configuration questions relating to transferring files from SBA to one or more destinations. This table requires specific configuration information for the destinations, IP addresses, user IDs, passwords, and directories. The SBA uses this configuration information to log on , and to transfer the files to the downstream destination. Record your answers in the spaces provided. The downstream destination (billing server) must comply with the following FTP commands and successful return codes, in order for the destination to successfully receive billing files:
FTP states/commands OPEN USER PASS TYPE STRU CWD STOR CLOSE RNTO QUIT Table 21 Outbound file transfers # 33 Question What is the destination to transfer the billing files? Explanation destination The combination of the values for stream name, file format type, and destination acts as the key to the schedule tuple. The destination cannot contain unprintable characters or blanks. Answer Return Codes 220 230 230 200 200, 250 250 226, 200, 250, 150 221 250 221

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

212

Preparation for disaster recovery

Question

Explanation Type: numeric String Range: 1 to 15 characters Default: none Example: Eventure

Answer

34

Which protocol is to be used to transfer billing files from the SBA?

protocol FTPW uses the File Transfer Protocol RFTPW (real time file transfer protocol wrapper) is used for the Real-Time Billing (RTB) application. RFTPW is supported only if the RTB application is configured. SFTPW (secure file transfer protocol wrapper) provides secure outbound file transfer using the OpenSSH SFTP client. SFTPW is supported only if OpenSSH is installed on the Core Manager. RSFTP (password-based user authentication) used for secure Real-Time Billing application when configured with OpenSSH SFTP client on the Core Manager. The initial host key acceptance of the downstream processor must be performed manually before the SFTP is used to transfer files. This must be performed for each downstream destination. KSFTP (key-based secure file transfer protocol wrapper) provides secure outbound file transfer with public key authentication, using the OpenSSH SFTP client. KSFTP is supported only if OpenSSH is installed on the Core Manager. RKSFT (key-based authentication) used for secure Real-Time Billing application when configured with OpenSSH SFTP client on the Core Manager.

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

Outbound file transfer destinations 213

Question

Explanation sendtape does not support SFTPW/RSFT P. Type: enumeration Range: FTPW, RFTPW, SFTPW, RSFTP, KSFTP , RKSFT, Default: FTPW (not case sensitive)

Answer

35

What is the IP address of the primary destination for this stream?

primary_destination The primary destination is the IP address that the SBA logs on to, and transfers the billing files. Type: IP Address Range: 0.0.0.0 to 255.255.255.25 5 Example: 47.202.35.189

36

What is the port for the primary destination?

primary_port The primary port number is associated with the primary IP address. Type: numeric Range: SFTPW, RSFTP, KSFTP, or RKSFT: 22, 1025 to 65535 Default: 22, for SFTPW, RSFTP, KSFTP, or RKSFTSFTPW or RSFTP Example: 22 FTPW or RFTPW: 21, 1025 to 65535 Default 21, for FTPW or RFTPW Example: 22

37

What is the IP address of the alternate destination for this stream?

alternate_destination The alternate destination is the IP address that the SBA logs into and transfers the billing files if SBA encounters problems in connecting to the primary destination. If there is no alternate destination, make this entry identical to the primary IP address. Type: IP Address Range: 0.0.0.0 to 255.255.255.25 5 Example: 47.202.35.189

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

214

Preparation for disaster recovery

# 38

Question What is the Port for the alternate destination?

Explanation alternate_port The alternate port number is associated with the alternate IP address. Type: numeric Range: SFTPW or RSFTP: 22, 1025 to 65535 Default: 22 for SFTPW or RSFTP SFTPW, RSFTP , KSFTP, or RKSFT: 22, 1025, 65535 FTPW or RFTPW: 21, 1025 to 65535 Default: 21 for FTPW or RFTPW Default: 22 for SFTPW, RSFTP, KSFTP, or RKSFT Example: 22 FTPW or RFTPW: 21, 1025 to 65535 Default 21, for FTPW or RFTPW

Answer

39

What is the logon for the downstream destination for this stream?

remote_login The SBA user ID needed to log on to the downstream destination, and to transfer the billing files. Type: string Range: 1 to 20 alphanumeric characters Default: none Example: amadns (case sensitive)

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

Outbound file transfer destinations 215

# 40

Question What is the password for the logon ID in question 24 for this stream?

Explanation remote_password This is the SBA password used to log on to the downstream destination to transfer the billing files. Type: string Range: 1 to 20 alphanumeric characters Default: none Example: abracadabra (case sensitive) The prompt for this entry is not generated if the RKSFT/KSFTPRSFTP (real-time/key-based file transfer authentication) protocol is selected)

Answer

41

What is the directory path on the downstream destination where the transferred billing files are to be stored?

remote_storage_ directory This is the full path to the directory on the downstream destination to which SBA transfers billing files. If this value is a period (.), the SBA FTP client does not issue a change working directory (CWD) command when a file transfer occurs. Type: string Range: 1 to 255 characters Example: /users/amadns/billing (case sensitive)

42

What is the desired field separator character for this stream?

field_separator This is a single character that the SBA uses to separate the components of billing file names when they are transferred to the downstream destination. If the downstream destination is a UNIX system, the recommended field separator is a period (.); this results in a file name such as 020001.030002.00001.01.2. If the downstream destination is a system that does not allow more than one period (.) in the filename, the recommended field separator is an underscore (_); this results in a file name such as 020001_030002_00001_01_2.

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

216

Preparation for disaster recovery

Question

Explanation Type: character Range: any printable character Default: . (period) (case sensitive)

Answer

43

What is the desired filename extension for this stream?

file_extension The filename extension is a short character string that SBA appends to the billing file names when it transfers them to the downstream destination. The extension can be set as a part of the schedule add command by providing a value to the following prompt: File extension can have between 0 and 3 characters. Enter Blank for no file extension. Press Enter to accept Blank. Enter File_Extension: Nortel does not recommend the use of the .gz file extension when you configure the outbound file transfer schedule. Use of the .gz extension can make it difficult for you to distinguish between compressed and uncompressed files on the downstream system.

Outbound le transfer protocol


The following table contains a list of configuration questions relating to transferring files from SBA to the downstream destination. This table requires specific configuration limits information to control how the SBA reacts when it encounters problems in connecting to the downstream destination. Record your answers in the spaces provided.

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

Outbound file transfer schedule Table 22 Outbound file transfer protocol # 44 Question What is the maximum number of times SBA attempts to complete a failed session with the downstream destination for this stream? After a session for this stream fails, what is the maximum time in seconds that SBA must wait before attempting re-connection to the downstream destination? Explanation protocol_max_retries Type: number Range: 0 to 10 Default: 3 Answer

217

45

protocol_retry_wait _time Type: number Units: seconds Range: 1 to 60 Default: 1

Outbound le transfer schedule


The following table contains a list of stream configuration questions relating to transferring files from SBA to the downstream destination. This table specifically addresses configuration information concerning when SBA initiates a connection to the downstream destination to transfer billing files. Record your answers in the spaces provided.
Table 23 Outbound file transfer schedule # 46 Question Are scheduled file transfers to the downstream destination required for this stream? Explanation schedule_active Determines if SBA automatically initiates file transfers to the downstream destination. If set to Yes, SBA automatically transfers files to the downstream destination at the times defined by the answers to questions 46, 47 and 48. If this value is set to No, manual file transfers can be made using the sendfile command. Type: Boolean Range: Yes, No Default: No If No, use 0:00 for Answers 46 and 47 and 120 for Answer 48.
Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

Answer

218

Preparation for disaster recovery

Table 23 Outbound file transfer schedule (contd.) # 47 Question When should SBA start initiating file transfers to the downstream destination each day? Explanation schedule_start_time Determines the time of day when SBA starts file transfers to the downstream destination. See the examples following this table for more information. Type: Time of Day Units: hh:mm Range: 00:00 to 23:59 Default: none 48 When should SBA stop initiating file transfers to the downstream destination each day? schedule_stop_time Determines the time of day when SBA ends file transfers to the downstream destination. See the examples following this table for more information. Type: Time of Day Units: hh:mm Range: 00:00 to 23:59 Default: none 49 Within the daily time window defined in questions 46 and 47, how often should the SBA transfer files to the downstream destination? schedule_interval This specifies the interval, in minutes, at which SBA is to initiate billing file transfers to the downstream destination. This interval is only active during the window of time specified by the start time (question 46) and stop time (question 47). See the examples following this table for more information. Type: Number Units: Minutes Range: 5 to 1440 Default: 120 Answer

The following are some examples that show different answers to questions for the start time (question 46), stop time (question 47), and the interval (question 48) and the resulting SBA file transfer times. If your start time and stop time are identical, then SBA is setup for continuous outbound file transfer.

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

Outbound file transfer schedule

219

Start Time 0:00

Stop Time 0:00

Interval 240

SBA Actions The SBA transfers files every four hours, at the beginning of the hour, starting at midnight. The SBA transfers files every thirty minutes at 10 minutes and 40 minutes after the hour, between 10:10 pm and 2 am. The SBA transfers files every five hours at 15 minutes after the hour, starting at 3:15 am.

Resulting Transfers The SBA initiates file transfers at 12:00 midnight, 4:00 am, 8:00 am, 12:00 noon, 4:00 pm and 8:00 pm The SBA initiates file transfers at 10:10 pm, 10:40 pm, 11:10 pm, 11:40 pm, 12:10 am, 12:40 am, 1:10 am and 1:40 am SBA initiates file transfers at 3:15 am, 8:15 am, 1:15 pm, 6:15 pm and 11:15 pm.

22:10

2:00

30

3:15

3:15

300

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

220

Preparation for disaster recovery

Conguring SBA backup volumes on a billing stream


Add new SBA backup volumes to a billing stream, or remove SBA backup volumes from a billing stream.

Procedure steps
Step 1 Action At the MAP Access the billing level, type mapci;mtc;appl;sdmbil 2 To add volumes, type addvol <stream_name> <volume1> ... <volume5> Repeat this step to add as required. Example Command to add five volumes addvol AMA S00DAMA1 S01DAMA2 S00DAMA3 S01DAMA4 S00DAMA5 3 To remove volumes, type remvol <stream_name> <volume1> ... <volume5> Repeat this step as required. Example Command to remove five volumes remvol AMA S00DAMA1 S01DAMA2 S00DAMA3 S01DAMA4 S00DAMA5

--End--

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

Variable definitions

221

Variable denitions
Variable <stream_name> Value The name of the billing stream to which to add the volume. The volume name. You can add, or remove, a maximum of five volumes at one time. Separate each entry with a space.

<volume1> ... <volume5>

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

222

Preparation for disaster recovery

Accessing the MATE


Use this procedure to access the MATE.

Prerequisites

You must have the root user ID and password to log on to the server. You may log on to the server as root or emsadmin.

Procedure steps
Instructions for entering commands in the following procedure do not show the prompting symbol, such as #, >, or $, displayed by the system through a GUI or on a command line.
At the workstation UNIX prompt or VT-100 terminal prompt:

Step 1 2

Action Log onto the CBM. Get the current hostname by entering:
GetCurrentHostName

Example response: <CBM hostname>-<unit0 / unit1> 3 Access the Report Registration Menu: If the hostname returned in step 2 contains "unit0" then the MATE hostname is "unit1" If the hostname returned in step 2 contains "unit1" then the MATE hostname is "unit0" 4 Determine if the MATE is running by entering:
ping <mate hostname>

The <mate hostname> is the one determined in step 3. 5 6 If the MATE is not Active, you must access the MATE through a local VT100 terminal. Go to step 7. Access the MATE using SSH by typing: ssh root@ <mate hostname> You can log on to the MATE without a password. To exit the MATE, type > exit to return to the local system.

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

Procedure steps

223

This procedure is complete


--End--

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

224

Preparation for disaster recovery

Administration: Creating Disaster Recovery Floppy Disks


The disaster recovery operation is used when it is necessary to restore the programs and data stored on your OAM&P workstation. The disaster recovery operation depends on the use of disaster recovery floppy disks. You can use the tape drive application to create the two disaster recovery floppy disks required to perform the disaster recovery operation. To create the disaster recovery floppy disks, perform the following steps:

Procedure steps
At the OAM&P workstation

Step 1 2

Action Label two formatted floppy disks as Disaster Recovery Disk 1 and Disaster Recovery Disk 2. Insert Disaster Recovery Disk 1 into the A: drive in your OAM&P workstation and follow the directions.

ATTENTION
The disaster recovery floppy disk creation process does not format the disks for you, but it does erase all of the data on the disks. Ensure that you have copied important data from the disks before proceeding.

ATTENTION
For more information about creating disaster recovery disks, see the tape drive documentation provided with the OAM&P workstation.

--End--

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

Procedure steps

225

Power-On and boot a Policy Controller unit


This procedure is used to power on a Policy Controller unit that has been installed as a replacement, or was shutdown for any other reason. This procedure may be used as a standalone task or as part of a higher level activity such as part of a dead office recovery activity or software upgrade activity.

Prerequisites

If the unit was a replacement unit recently installed, ensure that all power cabling connections have been properly installed and secured at the rear of the chassis and SAM-F frame.

Procedure steps
At the front panel of the Policy Controller unit

Step 1

Action If necessary, power on the Policy Controller using the main power switch located on the front panel.

If desired, at the Policy Controller console, monitor the boot progress of the unit.
--End--

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

226

Preparation for disaster recovery

Viewing the operational status of the NCGL platform


View the operational status of the Nortel Carrier-grade Linux (NCGL) platform to view the service status of the hardware and NCGL operating system using the CS 2000 NCGL Platform Manager. This procedure provides instructions to determine the service status of the SST NCGL platform only. Although you can accomplish some activities described in this procedure using the CS 2000 Session Server Manager, they are described instead using the more complete CS 2000 NGCL Platform Manager.

ATTENTION
This procedure does not describe how to view customer logs or alarms. For detailed instructions about viewing customer logs or alarms, see Nortel Session Server Trunks Fault Management ( (NN10332-911)) .

Procedure steps
At the CS 2000 NCGL Platform Manager or IEMS client

Step 1

Action At the Communication Server 2000 Session Server Launch point, click the Carrier VoIP Communication Server 2000 NCGL Platform Manager link.
The Platform Main Page menu appears.

ATTENTION
The platform (System) Information page does not update automatically. Click the System Information link to update it.

Review the System Information page. 3 4 5 Review the Node Maintenance page. The Node Maintenance page refreshes every 45 seconds. Review the System Status page. The Chassis Information section. Review the Network Connectivity page. The Network Connectivity page refreshes every 45 seconds.

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

Procedure steps Figure 14 Crossover and LAN Ethernet cable connections for SST units

227

Review the Disk Services page. The Disk Services page does not update automatically. Click the Disk Services link to update it.

Review the Services page. The Services page does not update automatically. Click the Services link to update it.

Review the Administration page. The Administration page does not update automatically. Click the link to update it.

To logout from the CS 2000 NCGL Platform Manager, click Logout.


--End--

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

228

Preparation for disaster recovery

Viewing the operational status of the SIP Gateway application


Use the following procedure to view the service status of the SIP Gateway application.

Procedure steps
At the CS 2000 Session Server Manager or IEMS client

Step 1

Action At the Communication Server 2000 Session Server Launch point, click the Carrier VoIP Communication Server 2000 Session Server Manager. Click to open the Session Server, Maintenance, Application folder. Click the SIP Gateway link. Monitor the status of the SIP Gateway application.

2 3 4

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

Procedure steps

229

The view refreshes according to the value shown in the box at the bottom of the status section. 5 6 To increase or decrease the refresh rate, select a different value and click Refresh Rate. Optionally, you can perform the following service affecting actions:

Lock the SIP Gateway application Unlock the SIP Gateway application Suspend the SIP Gateway application Unsuspend the SIP Gateway application Cold SwAct the SIP Gateway application

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

230

Preparation for disaster recovery

To view the number of active calls currently handled by the application, and the synchronization status of the units, click QueryInfo.
--End--

Job aid
The following table provides a description of the Node Status fields.
Table 24 Node status field descriptions Field Unit Connection Status Bar Unit Number Activity State Description Indicates which unit in the node the CS 2000 Session Server Manager is connected to. Identifies the two units in the node, labeled 0 and 1. Indicates which unit is active and which is inactive (standby). Acts as an indirect indicator of fault-tolerant status; when both units have an Operational status of enabled, the node is fault-tolerant. Indicates the service status of each unit, enabled or disabled.

Operational State

The following table provides a description of the SIP Gateway application Status fields.
Table 25 SIP Gateway application Status field descriptions Field Administrative State Operational State Procedural Status Control Status Indication Locked, Unlocked, or ShuttingDown Enabled or Disabled Terminating or Suspended or -

Use the following table to interpret the SIP Gateway areas CCITT X.731-style and related DMS-style status indicators.

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

Job aid Table 26 SIP Gateway Maintenance field descriptions and interpretation of service states Administrative State Locked Locked Locked Operational State Disabled Enabled Enabled Procedural Status in-service in-service Terminating Control Status Suspended in-service in-service

231

DMS styleServic e States Offline (OFFL) Manual Busy(MA NB) Manual Busy Transitioning(M ANBP) In Service(INSV) System Busy(SY SB) Going out of service(INSVD)

Unlocked Unlocked Shutting Down

Enabled Disabled Enabled

in-service in-service in-service

in-service in-service in-service

(-) indicates a status of in-service

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

232

Preparation for disaster recovery

Verifying synchronization status


Verify synchronization status to determine the synchronization status of the two units.

Procedure steps
Step 1 Action At the Communication Server 2000 Session Server Manager or IEMS client, select Communication Server 2000 Session Server Manager from the launch point menu. Select Session Server > Maintenance > Application > SIP Gateway from the left side menu: At the bottom of the SIP Gateway Maintenance panel, locate and click the QueryInfo button. The synchronization status of the units is displayed at the bottom of the query results panel. If the units are not in sync, check for alarm conditions.

2 3 4

If applicable, return to the higher level task flow or procedure that directed you to this procedure.
--End--

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

Procedure steps

233

Booting the XA-Core from a Reset Terminal


Boot the XA-Core from a Reset Terminal to charge the XA-Core from a reset terminal display.

ATTENTION
Booting the switch causes the switch to drop all calls.

ATTENTION
An image loads from a small computer systems interface (SCSI) device. The SCSI device can be in a disk or a digital audio tape (DAT).

CAUTION
Call your next level of support
Contact your next level of support before performing this procedure.

CAUTION
Extended service interruption
A longer recovery time occurs for a switch boot from tape than a switch boot from disk. To minimize recovery time, boot from disk.

CAUTION
Extended service interruption
Log on procedures can vary depending on your office configuration. If you need additional help, contact your next level of support.

Procedure steps
Step 1 Action See the office records and determine the name of the XA-Core recording device that contains the last office image file.

ATTENTION
The XA-Core recording device is a disk drive or a tape drive for a digital audio tape (DAT). Record the name of the XA-Core device.

At the XA-Core shelf make sure that the disk drive or the tape drive that you recorded in step 1 is in-service.

ATTENTION
The device is in-service when the green light-emitting diode (LED) is on (illuminated).

If the device is not in service, contact your next level of support.

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

234

Preparation for disaster recovery

If the device is in service, at the XA-Core reset terminal override the XA-Core reset terminal by typing:
>\OVERRIDE

Example of a reset terminal response


NOW IN SERVICE AFFECTING MODE

Boot the XA-Core by typing: >\BOOT <nn> <s> <p> Example Response:
>\BOOT 4 F L

Example of system response:


Warning: Boot command will take it out of service. Please confirm ("YES", "Y", "NO", or "N")

Type Y to confirm the command. 6 Monitor the reset terminal display to determine if the switch has booted.

ATTENTION
The reset terminal displays a response to indicate a boot in progress. The response also displays diagnostic messages and alphanumeric addresses. When the switch has completely booted, a prompt appears on the display.

One possible example of a reset terminal response


CI: > Another possible example of a reset terminal response FWCI>

7 8 9

If the response is a prompt then go to step 9. At the MAP terminal If the response has no prompt after approximately 15 minutes then contact your next level of support. Press the <BREAK> key to determine if you have to log in.

ATTENTION
The log on message indicates that you have to manually log in. An automatic log in can occur if the office parameters have automatic log in.

Example of a MAP response


Please Login.

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

Variable definitions

235

10 11 12

If log in is not automatic then go to step 11. if the log in is automatic then go to step 15. Login to the MAP terminal by typing:
>LOGIN

Example of a MAP response


Enter User Name

13

Enter your user name by typing: ><user_name> Example of a MAP response


Enter Password

14

Enter the password by typing: ><password> Example of a MAP response


SuperNode_1 Logged in on 1997/01/15 at 20:37:17

15 16 17

Access the MS Clock level of the MAP display by typing:


>MAPCI;MTC;MS;CLOCK

Synchronize the clocks by typing:


>SYNC

If the MAP response is not a successful completion, contact your next level of support.
--End--

Variable denitions
Variable <nn> <p> <password> <s> Value The slot number parameter value to indicate the number of the physical shelf slot - 0 to 18. The position parameter value that indicates the IOP bay - either upper (u) or lower (l). The name of the password for the account. The side parameter value to indicate the circuit pack or packlet location in the physical shelf front (f) or rear (r). The name of the user for the account.

<user_name>

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

236

Preparation for disaster recovery

Resetting the Ethernet Routing Switch 8600 using a saved conguration le


The Ethernet Routing Switch 8600 boot command allows you to reset or reboot the system using a saved configuration file.

ATTENTION
You must be directly connected to the switch to initiate a Boot Monitor session. You can only connect using a Telnet connection if the Boot Monitor CLI is already active.

Prerequisites

You must have access to the Boot Monitor CLI through a direct connection to the switch or a Telnet connection. For more information about accessing the Boot Monitor CLI, see "Managing the Ethernet Routing Switch 8000 Series Switch" using the Command Line Interface Release 3.2, 313194-*.

Procedure steps
Step 1 Action At the Boot Monitor CLI issue the boot command by typing
monitor# boot [<file>] [config <value>]
--End--

Variable denitions
Variable <file> Value software image device and file name in the format [a.b.c.d:]<file> | /pcmcia/<file> | /flash/<file>. The file name, including the directory structure, can be up to 1024 characters. software configuration device and file name in the format [a.b.c.d:]<file> | /pcmcia/<file> | /flash/<file>. The file name, including the directory structure, can be up to 1024 characters.

config <value>

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

Procedure step

237

Restoring a Call Agent


Restore a Call Agent to restore an archived call processing application image from tape.

CAUTION
Possible service interruption
Do not use this procedure when in an emergency situation with no stable call processing application image. If in a situation without a restartable image, contact Nortel Global Network Product Support (GNPS) immediate. Attempting to use this method without a valid call processing application image could fail due to constant resets on the Call Agent.

Procedure step
Step 1 2 3 Action Insert the DAT cassette with the image to restore. Restore the image from tape. This step requires root privilege. For example, see Job aid (page 238). At the Call Agent Manager log on to the inactive Call Agent and change directory to the location in which to restore the image. Verify that enough disk space exists to hold the image. For example, see Job aid (page 238). 4 Open a file transfer protocol (FTP) session to the Communication Server 2000 Core Manager, and transfer the image. It may be necessary to become the super user to transfer the file. For example, see Job aid (page 238). 5 At the MAP enter the DISKUT level and use the IMPORT command to make the image available to the call processing application. For example, see Job aid (page 238). If additional space is needed to import the image, the IMPORT command offers to expand the volume. 6 7 Set the image in the Image Table of Contents (ITOC). For example, see Job aid (page 238). The restored image is now available for booting.
--End--

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

238

Preparation for disaster recovery

Job aid
Figure 15 DAT cassette

Figure 16 Disk space

Figure 17 File transfer

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

Job aid

239

Figure 18 DISKUT level

Figure 19 User interface

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

240

Preparation for disaster recovery

Restoring Session Server Trunks


Restore Session Server Trunks (SST) to restore a backup copy of critical files to an active unit.

CAUTION
Possible service disruption
Performing a restore of the SIP Gateway application database to the active unit is a service affecting activity and can cause data mismatches at the Communication server 2000.

Prerequisites

You must have secure file transfer access, such as scp, to the unit from a remote host if the backup file is stored on a remote host. Backups are also stored locally on each unit in directory /data/bkresmgr/backup.

CAUTION
Both units must be free of any Session Server Manager patches before the database is restored. This is accomplished by reinstalling the current version of the SIP Gateway application software (performed in step 1 below). It is not necessary to remove NCGL patches.

Procedure steps
Step 1 Action For detailed steps for this procedure, see Nortel Carrier Voice over IP Network Upgrades and Patches (NN10440-450).
--End--

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

Procedure steps

241

Restoring a database to a Session Server unit


Restore a SIP Gateway application database from a backup copy to the active Session Server units. Perform this procedure part of a high-level fault management activity for restoring a backup copy over a corrupted version of the database, or as part of a high level upgrade activity from Nortel Session Server Upgrades, (NN10349-461).

ATTENTION
Service affecting
Perform this procedure on the active unit only. Restoring the SIP Gateway application database to the active unit affects service and can cause data mismatches at the CS 2000 call server.

Prerequisites

Back up the Session Server Trunk before you perform this procedure.

Procedure steps
Step 1 Action At the Session Server CLI or IEMS client Log on to the active Session Server to which you will restore the database. 2 3 4 Change to the root user. Change directories, type $ cd /opt/apps/database/solid_install Run the database restore script, type $ ./restorebackup.sh
--End--

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

242

Preparation for disaster recovery

Restoring audio les to a UAS node


In the event that a re-installation of a UAS node is required due to an error condition, audio files must be restored to the unit when it becomes operational. This procedure allows you to enable audio provisioning to the node and to specify which audio files are to be restored to it.

ATTENTION
For more information about re-installation of a UAS node, contact your Nortel service representative.

Procedure steps
Step 1 Action After the re-installation of the UAS node has been completed, determine whether you want to enable provisioning of the node occur during the next audio distribution cycle or immediately. If you want to enable provisioning of the node to occur during the next audio distribution cycle, go to step 4. If you want audio provisioning of the node to occur immediately, go to step 5. Perform "Enabling provisioning of a UAS node" in the document (NN10095-511), entitled "UAS Configuration Management," in your UAS document suite.

2 3 4

ATTENTION
Provisioning of the node will begin during the next audio distribution cycle. The distribution cycle occurs once for each hour.

Perform "Provisioning a UAS node" in the document (NN10095-511), entitled "UAS Configuration Management," in your UAS document suite.

ATTENTION
Provisioning of the node will begin immediately although as much as a five-minute delay may occur before actual provisioning activity begins.

--End--

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

Procedure steps

243

Restoring audio les to a Media Server 2000 Series node


In the event that a re-installation of a Media Server 2000 Series node is required due to an error condition, audio files must be restored to the unit when it becomes operational. This procedure allows you to enable audio provisioning to the node and to specify which audio files are to be restored to it.

ATTENTION
For more information about re-installation of a Media Server 2000 Series node, contact your Nortel service representative.

Procedure steps
Step 1 Action After the re-installation of the Media Server 2000 Series node has been completed, determine whether you want to enable provisioning of the node occur during the next audio distribution cycle or immediately. If you want to enable provisioning of the node to occur during the next audio distribution cycle then go to step 4. If you want audio provisioning of the node to occur immediately then go to step 5. Perform the "Enabling provisioning of a Media Server 2000 Series node" (see the Media Server 2000 Series Configuration Management document).

2 3 4

ATTENTION
Provisioning of the node will begin during the next audio distribution cycle. The distribution cycle occurs once for each hour.

Perform the procedure "Provisioning a Media Server 2000 Series node" (see the Media Server 2000 Series Configuration Management document).

ATTENTION
Provisioning of the node will begin immediately although as much as a five-minute delay may occur before actual provisioning activity begins.

--End--

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

244

Preparation for disaster recovery

Restoring the APS-specic Oracle database and application les


This procedure enables you to restore the APS-specific Oracle database and application files (/PROV_data, /audio_files, and /user_audio_files). The Server Platform Foundation Software (SPFS) base software provides utilities that enable you to restore these files. The "rsimpora" utility restores the APS Oracle database files. The "ufsrestore" utility restores the UNIX file system, including all of the "Root" and "Non-Root" APS database files. The instructions for performing these backup procedures are found in the ATM/IP Configuration document (NN10276-500). In addition to these two utilities, two additional APS utilities and procedures enable you to restore selected files when only the files required for APS operation must be restored. The "ips_export_db.sh -restore" utility restores only the APS Oracle database. A procedure that utilizes the UNIX "tar" command enables you to restore the non-Root files, "/audio_files," "/PROV_data," "/user_audiofiles," and Root file, "/etc/inet/hosts."

Procedure steps
Step 1 2 3 4 5 6 7 Action Open an xterm window and log in using the "maint" logon and password. Become the "root" user by entering
su - root

If you want to restore Oracle database files from tape then go to step 5. If you want to restore Oracle database files from disk then go to step 10. Insert the appropriate "ORACLE" backup tape into the DDS-3 tape drive. Rewind the backup tape by performing the following command
mt -f /dev/rmt/0c rewind

Start the restoration of the APS Oracle database from the tape by entering the following command
ips_export_db.sh -t /dev/rmt/0c -restore

ATTENTION
This command is entered on a single line.

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

Procedure steps

245

Messages logging the progress of the restoration display on the screen. When you are prompted about continuing the restoration even if the current database will be destroyed, enter "y" (yes). 8 After the restoration of the Oracle database is complete, rewind the backup tape by performing the following command
mt -f /dev/rmt/0c rewind

9 10

Eject the backup tape and store it for possible use later. Go to step 11. Start the restoration of the APS Oracle database from the disk by entering the following command
ips_export_db.sh -diskonly -restore

ATTENTION
This command is entered on a single line.

Messages logging the progress of the restoration display on the screen. When you are prompted about continuing the restoration even if the current database will be destroyed, enter "y" (yes). 11 12 13 14 Insert the appropriate "application file" backup tape into the DDS-3 tape drive. Change directory to the root directory by typing
cd /

Rewind the backup tape by performing the following command


mt -f /dev/rmt/0c rewind

Restore the application files (/PROV_data, /audio_files, and /user_audio_files) by entering the following command
tar xvf /dev/rmt/0c

15

When the restoration of the application files completes, remove the tape from the tape drive and store for possible use later. Insert another write-enabled DAT tape into the drive to be used for the automatic Oracle system back up that runs daily at 1:00 a.m. Verify that the application files have been restored by entering the following commands
cd /PROV_data ls -l

16

The files should display on the screen.


cd /user_audio_files ls -l

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

246

Preparation for disaster recovery

The files should display on the screen.


cd /audio_files ls -l

The files should display on the screen.


--End--

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

Procedure steps

247

Restoring a Multiservice Switch or Media Gateway device


This section describes using Service Data Backup and Restore to restore backed up service data from the backup site to the Multiservice Switch 7400/15000 or Media Gateway 7400/15000. The Service Data Backup and Restore tools provide three types of restore:

A full restore restores all backed up service data to the selected device or devices. An incremental restore restores service data based on a specified date. Like the full restore, you can perform an incremental restore on either one or multiple devices. A selective restore restores specific service data that you select. Like the full restore, you can perform a selective restore on either one or multiple devices.

ATTENTION
It is not recommended that an active file (current view) be restored on a Multiservice Switch 7400/15000 or Media Gateway 7400/15000. When you restore the current view, you may overwrite the existing current view with different content. This action will cause an outage of the Backup and Restore tool.

The following information applies to using the Service Data Backup and Restore tool to perform a restore for Multiservice Switch 7400/15000 or Media Gateway 7400/15000 nodes:

Adding Multiservice Switch 7400/15000 or Media Gateway 7400/15000 devices to the restore list (page 248) Removing Multiservice Switch 7400/15000 or Media Gateway 7400/15000 devices from the restore list (page 248) Defining a specific userid and password for a Multiservice Switch 7400/15000 or Media Gateway 7400/15000 in the backup list (page 174) Performing a full restore for a Multiservice Switch 7400/15000 or Media Gateway 7400/15000 device (page 249) Performing an incremental restore for a Multiservice Switch 7400/15000 or Media Gateway 7400/15000 device (page 250)Performing a selective restore for a Multiservice Switch

7400/15000 or Media Gateway 7400/15000 device (page 251)

Activating the restored Multiservice Switch or Media Gateway data (page 252)

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

248

Preparation for disaster recovery

Adding Multiservice Switch 7400/15000 or Media Gateway 7400/15000 devices to the restore list
Use this procedure to add devices to the list of devices that you wish to restore. Procedure steps Step 1 Action Open the Backup and Restore window. From the MDM window, select Configuration->Passport Devices->Administration->P assport Service Data Backup/Restore. Select the Restore Configuration tab. Select Add to launch the Add Devices Dialog. From the left pane, select a group of devices or expand the group and use the Ctrl key to select multiple devices within a group. From the drop down list, in the right pane, select a restore mode (Incremental or Full). If a specific userid and password is required for the device, enter the values in the user ID and Password fields and uncheck the Use default checkbox. If you wish to use the default userid and password, click the Use defaultcheckbox. Click OK. This procedure is complete.
--End--

2 3 4

5 6

7 8 9

Removing Multiservice Switch 7400/15000 or Media Gateway 7400/15000 devices from the restore list
Use this procedure to remove devices from the list of devices that you wish to restore. Procedure steps Step 1 Action Open the Backup and Restore window. From the MDM window, select Configuration->Passport Devices->Administration->P assport Service Data Backup/Restore.

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

Performing a full restore for a Multiservice Switch 7400/15000 or Media Gateway 7400/15000 device 249

2 3 4 5 6

Select the Restore Configuration tab. In the Device List, select the devices you wish to remove. Click Remove. In the confirmation dialog, select Yes to confirm or No to cancel the removal. This procedure is complete.
--End--

Dening a specic userid and password for a Multiservice Switch 7400/15000 or Media Gateway 7400/15000 in the restore list
A specific userid and password can be defined when you add the device to the restore list or you can set it later using the following procedure. Procedure steps Step 1 Action Open the Backup and Restore window. From the MDM window, select Configuration->Passport Devices->Administration->P assport Service Data Backup/Restore. Select the Restore Configuration tab. Select a device in the Device List section. In the Device Details section, in the Authentication tab, enter a userid and password. Clear the Use default checkbox. This procedure is complete.
--End--

2 3 4 5 6

Performing a full restore for a Multiservice Switch 7400/15000 or Media Gateway 7400/15000 device
Use this procedure to perform a complete system restore of a Multiservice Switch 7400/15000 or Media Gateway 7400/15000 device. Procedure steps

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

250

Preparation for disaster recovery

Step 1

Action Open the Backup and Restore window. From the MDM window, select Configuration->Passport Devices->Administration->P assport Service Data Backup/Restore. Select the Restore Configuration tab. Select Add to launch the Add Devices Dialog. From the left pane, select a group of devices or expand the group and use the Ctrl key to select multiple devices within a group. Click in the Mode title and selectFull from the drop-down list. Click Restore. Activate the restored data by executing the procedure Activating the restored Multiservice Switch or Media Gateway data (page 252). This procedure is complete.
--End--

2 3 4

5 6 7

Performing an incremental restore for a Multiservice Switch 7400/15000 or Media Gateway 7400/15000 device
Use the following procedure to copy files, based on a specific date, from the repository back to the Multiservice Switch 7400/15000 or Media Gateway 7400/15000. Procedure steps Step 1 Action Open the Backup and Restore window. From the MDM window, select Configuration->Passport Devices->Administration->P assport Service Data Backup/Restore. Select the Restore Configuration tab. Select Add to launch the Add Devices Dialog. From the left pane, select a group of devices or expand the group and use the Ctrl key to select multiple devices within a group. Select a device from the Devices list. In the Mode title, select Incremental from the drop-down list. Define a date in the Date column (For example July 4, 2008)

2 3 4

5 6 7

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

Performing a selective restore for a Multiservice Switch 7400/15000 or Media Gateway 7400/15000 device 251

Click Restore. All files with a date and time earlier than the specified date are restored.

Activate the restored data by executing the procedure Activating the restored Multiservice Switch or Media Gateway data (page 252). This procedure is complete.
--End--

10

Performing a selective restore for a Multiservice Switch 7400/15000 or Media Gateway 7400/15000 device
Use the following procedure to copy a specific file from the repository back to the Multiservice Switch 7400/15000 or Media Gateway 7400/15000. Procedure steps Step 1 Action Open the Backup and Restore window. From the MDM window, select Configuration->Passport Devices->Administration->P assport Service Data Backup/Restore. Select the Restore Configuration tab. Select Add to launch the Add Devices Dialog. From the left pane, select a group of devices or expand the group and use the Ctrl key to select multiple devices within a group. Select a device from the Devices list. In the Mode title, select Selective from the drop-down list. The Configuration column in the table is enabled. 7 Click in the Configuration cell to display a pull-down list that displays all the available views on the Multiservice Switch 7400/15000 or Media Gateway 7400/15000. Note: This step may take a few seconds to complete because the application must access the Multiservice Switch 7400/15000 or Media Gateway 7400/15000 and list all the views names. 8 Click Restore.

2 3 4

5 6

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

252

Preparation for disaster recovery

Activate the restored data by executing the procedure Activating the restored Multiservice Switch or Media Gateway data (page 252). This procedure is complete.
--End--

10

Activating the restored Multiservice Switch or Media Gateway data


Restore restores the backup configuration data onto the Multiservice Switch 7400/15000 or Media Gateway 7400/15000 disk. After a successful restore, perform the following procedure to activate the restored data. Step 1 2 Action Establish a telnet session to the Multiservice Switch 7400/15000 or Media Gateway 7400/15000. Download to the Multiservice Switch 7400/15000 or Media Gateway 7400/15000 any required application software missing from the Multiservice Switch 7400/15000 or Media Gateway 7400/15000 disk. Enter provisioning mode. Activate the restored view and confirm the activation If required, commit the activated view. This procedure is complete.
--End--

3 4 5 6

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

Activating the restored Multiservice Switch or Media Gateway data

253

Performing a full system restore


Restore system configuration from a backup. Typical uses of this operation are in the event of one or more of the following:

An emergency situation requires restoration of system configuration from a stored data snapshot. You use the file manager function to ensure that there are copies of the data snapshot to be restored on both RTC system nodes. You make the data snapshot the boot snapshot for both RTC system nodes. You deactivate the linksets and change the path state of the application server process (ASP) paths to Down for your system, as appropriate. You perform a complete office recovery (COR) on your system, which means that you unseat the RTC system nodes, turn off both power switches on the shelf, reseat the RTC system nodes, and then restore power. Your stored data snapshot is now the running data snapshot.

The following procedures must be executed in the order given to restore data snapshots stored on your alternate boot server or on the RTC system nodes.

Restoring a system configuration from a data snapshot (page 254) Procedure 2 Performing a restore operation from a backup CD-RW (page 257)

CAUTION
Risk of service loss
Performing a COR on your system is service affecting. Nortel recommends that you do this during off-peak hours and that you ensure that a mated system is available.

CAUTION
Risk of service loss
Do not power down the shelf when the SCSI disk light on the RTC system node is on. To do so could cause a disk failure.

CAUTION
Risk of equipment damage
Wear wrist straps, and use standard antistatic precautions.

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

254

Preparation for disaster recovery

Restoring a system conguration from a data snapshot


Use the procedures in this section to restore a system configuration from a data snapshot stored on your alternate boot server.

Procedure steps
At the OAMP workstation

Step 1 2 3 4

Action Click Administration>file-manager. If the RTC system node is not locked, click Lock. If the RTC system node is not offline, click Offline. Make note of which RTC system node is the active RTC system node. This information is required later in the procedure. Ensure that the LED is off for the SCSI Disk Drive associated with this RTC system node before unseating the mission card. Unseating the RTC card stops the write and read access to the SCSI card to avoid SCSI failure.

on the inactive RTC (slot15).

To execute a system COR

Unseat the mission card on the inactive RTC by pressing outward on the top and bottom latches of the mission card to release it from the Communications Application Module (CAM) shelf. Grasp the top and bottom latches of the mission card and gently pull it to disconnect it from the associated TM. Do not remove the mission card from the CAM shelf. Repeat steps 2 and 3 for the RTC mission card for the active RTC system node. Turn off both the A and B power switches on the rear of the control CAM shelf of your system. Reseat the RTC mission card for an RTC system by. gently sliding the mission card back into place. Apply pressure to the faceplate until you feel resistance. Snap the top and bottom latches of the mission card inward, toward one another. Two audible clicks can be heard when the mission card is seated properly. Repeat steps 5 through 7 for the RTC mission card for the other RTC system node.

8 9 10

11

12

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

Procedure steps

255

13

Return power to the shelf. To do this, turn on both the A and B power switches on the rear of the control CAM shelf of your system. System start up can take several minutes, depending on the configuration of your system.

To view system node status

14 15 16

Click Configuration>platform>node. Click the Graphical View tab. Confirm that all system nodes are enabled (green status light). Note: Nortel recommends that you do not attempt any provisioning changes until all system nodes recover.If the GUI fails to log in when it is reestablishing the connection, the boot snapshot may not contain the current users user-account. Provide another valid user-account that is configured in the snapshot.

17 18

Log into your system. Open the RTC system node administration tab for the RTC system node in slot 12. To do this, click Configuration>platform>node. Click the Graphical view tab, and select slot 12. If the table is opened before the GUI reconnects to the RTC, the old data is displayed. After the GUI reconnects to the RTC, reload the table.

19

20

If the data snapshot descriptions match, proceed to step 17. If the snapshot descriptions do not match, return to step 10 and attempt the procedure again. If the procedure has failed twice in row, contact your next level of support. For the current active snapshot the snapshot-date and snapshot-version fields are empty.

21

Update the system time settings in the Set Date/Time window, click Administration> Set Date/Time. Note: This step is not required if simple network time protocol (SNTP) is enabled for this system.

22

This procedure is complete


--End--

Use the following procedure to perform a restore backup from a backup tape

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

256

Preparation for disaster recovery Procedure 1 Performing a restore operation

ATTENTION
This procedure can overwrite existing data files on your workstation. Make sure you have a current data backup before you begin this procedure. At the Windows-based OAMP workstation

Step 1 2

Action Insert the tape containing the data that you want to restore. Double-click the Backup Exec icon on the desktop if you are running a Veritas program or the Colorado Backup II icon if you are running a Colorado program. Select Restore files using the Restore Wizard. Click Ok. Click Next. Select from media in the device. Click Next. The system loads the information from the backup tape. Select the backup date and time for the data that you want to restore. Click Ok. The system loads the information from the backup tape. Click the + button on the tree control to expand the tree box next to the C: drive. Select any directories that you want to restore. Click Next. Click Next a second time. Select Always replace the file on my computer. Click Start. The system restores the backup data. Click Ok. The data restore procedure is complete.
--End--

3 4 5 6 7 8 9 10 11 12 13 14 15 16

ATTENTION
This procedure can overwrite existing USP data files on your workstation. Make sure you have a current data backup before you begin this procedure.

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

Procedure steps

257

Before performing this procedure, ensure that the ABS is installed in the workstation. The root privilege is required to perform this procedure.
Procedure 2 Performing a restore operation from a backup CD-RW At the Solaris-based OAMP workstation

Step 1

Action Remove the current USP data by entering the following command at the prompt:
rm -Rf <USP data directory>

Example rm -Rf /opt/usp/ntssgusp/abs/sites 2 3 Insert the CD-RW containing the USP data that you want to restore into the CD-ROM drive. Go to the partition where the ABS is installed by entering the following:
cd <partition>

Example cd / The ABS is installed in the root partition by default. 4 Restore the USP data by entering the following command:
ufsrestore xf /<cdrom directory>/<dumped filename>

where <cdrom directory> is the directory where the CD-ROM is mounted. Example ufsrestore xf /cdrom/cdrom0/usp_data_20051115.dmp The ABS is installed in the root partition by default. The following is displayed; enter what is in bold text when prompted: You have not read any volumes yet. Unless you know which volume your file(s) are on you should start with the last volume and work towards the first. 5 Enter "1" in response to the promp as shown:
Specify next volume #:

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

258

Preparation for disaster recovery 1

If the data files are large, it can take several minutes to restore the data. 6 Enter "n" in response to the promp as shown:
set owner mode for .?[yn] n

Enter "n" in response to the promp as shown:


Directories already exist, set modes anyway? [yn] n

8 9

Return to the root directory by entering the following command:


cd /

Restore the DHCP configuration data by entering the following command:


ufsrestore xf /<cdrom directory>/<dumped filename>

where <cdrom directory> is the directory where the CD-ROM is mounted. Example ufsrestore xf /cdrom/cdrom0/dhcp_20051115.dmp The following is displayed; enter what is in bold text when prompted: You have not read any volumes yet. Unless you know which volume your file(s) are on you should start with the last volume and work towards the first. 10 Enter "1" in response to the promp as shown:
Specify next volume #: 1

If the data files are large, it can take several minutes to restore the data. 11 Enter "n" in response to the promp as shown:
set owner mode for .?[yn] n

12

Enter "n" in response to the promp as shown:


Directories already exist, set modes anyway? [yn]

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

Procedure steps n

259

13

Copy the NFS configuration file back by entering the following:


cp /<cdrom directory>/dfstab /etc/dfs/dfstab

Example cp /cdrom/cdrom0/dfstab /etc/dfs/dfstab 14 Restart the NFS server by entering the following commands:
/etc/init.d/nfs.server stop /etc/init.d/nfs.server start

15 16

Remove the CD-RW from the CD-ROM drive and return it to its regular storage location. This procedure is complete.
--End--

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

260

Preparation for disaster recovery

Restoring Core Element Manager data


Restore Core Element Manager (CEM) data to backup the CEM. Perform this backup procedure through the Synchronous backup restore manager (SBRM). For more information, see . The information needed by the Core Element Manager to restore data is located in /opt/nortel/cem/data/coreEMS/configBackup.

Prerequisites

The user must have root user privileges to perform this procedure.

Procedure steps
Step 1 2 3 4 5 Action At your workstation, open a Telnet session to the server, type > telnet <server> When prompted, enter your user ID and password. Change to the root user, type $ su When prompted, enter the root password. Make sure that the /opt/nortel/cem/data/coreEMS/configBackup directory contains the following files, type # ls /opt/nortel/d ata/coreEMS/configBackup

data_dir ldapConfig.tar nodes.tar server

Make sure that CEM is not running, type


# servquery -status -group CEM

Example response CEM server STOPPED 7 If the CEM server is running, stop the CEM server, type
# servstop CEM

Example response

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

Variable definitions

261

CEM server successfully stopped 8 Run the restore script, type


#/opt/nortel/cem/data/coreEMS/nodes/server/bin/pos tRestore.sh
--End--

Variable denitions
Variable <server> Value The IP address or host name of the server where IEMS resides.

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

262

Preparation for disaster recovery

Restoring the central security server


Restore the central security server to restore the security server from a backup file.

ATTENTION
SSL uses certificates to ensure security. Certificates from one server cannot be used on another server. If you want to take a back up file from an SSL server and restore it to a different server, see , after you complete the restore. Run a script to set up IS authentication, session, and policy traffic to operate under SSL. The servers in the same high availability cluster can use the same SSL certificate. If the restored IS SSL certificate is expired, see after you complete the restore.

Prerequisites

You must have a backup tar file. You must have root user privileges to perform this procedure.

Procedure steps
Step 1 Action At your workstation, open A Telnet session to the server, type telnet <server> If you are upgrading the active server, Telnet to the active server. If you are upgrading the inactive server, Telnet to the inactive server. 2 3 4 5 6 7 8 When prompted, enter your user ID and password. Change to the root user, type su - root When prompted, enter the root password. Stop the security services, type # servstop RADSVR and press Enter. Type # servstop IS and press Enter. Type # servstop WEBSERVICES and press Enter. List the available security server backup files and determine the backup file to be restored by typing

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

Procedure steps

263

# ls /data/bkresmgr and press Enter. Note the name of the security server backup file you intend to restore. You will need this information in a later step. 9 List the available security server backup files and determine the backup file to be restored by typing # ls /data/bkresmgr and press Enter. The format of the security server backup files are<data><time> backupSS2.0<host_name>.tar The name of the security server backup file you intend to restore. 10 Change directories, type
# cd /opt/nortel/applications/security/current_s1is ext/swmgmt/bin

11

Create a backup of the nsswitch.conf file, type # cp /etc/nsswitch.conf /etc/nsswitch.conf.<DATE> Record the name of the nsswitch.conf.<DATE> file. This file is required after the restore operation.

12 13

Disable SAML, type # cp /etc/nsswitch.conf.files /etc/nsswitch.conf Perform the restore operation, type
# ./brr_security.sh -restore /data/bkresmgr/<date><tim e>backupSS2.0<host_name>.tar

14 15 16 17 18

Restart the security services, type # servstart and press Enter. Type WEBSERVICES and press Enter. Type # servstart IS and press Enter. Type # servstart RADSVR Enable SAML by typing # cp /etc/nsswitch.conf.<DATE> /etc/nsswitch.conf and press Enter.

19

If you restored the back up image from a different server with a different certificate, or if the restored certificate is expired, see to replace the invalid or expired certificate on the IEMS server. Clone the active server to the inactive server. For details, see Clone the active server to the inactive server. For details, see .

20

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

264

Preparation for disaster recovery

21

Clone the active server to the inactive server. For details, see .
--End--

Variable denitions
Variable <DATE> <date><time> <host_name>.tar <server> Value The backup file created or should be restored. The backup file to be restored. The name of the security server backup file you intend to restore. The IP address or host name of the server where IEMS resides.

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

Procedure steps

265

Performing a full restore of the software from S-tape


Restore the core manager software load from the system image backup tape (S-tape).

Prerequisites

You must be a trained AIX system administrator who has root user privileges to the core manager. You must mirror all volume groups on the core manager before you perform this procedure. If you perform this procedure when disk mirroring is not at the Mirrored state, the system displays an error message. If your system includes the SuperNode Billing Application (SBA), you must use tape drive DAT0 to perform this procedure.

Procedure steps
Step 1 2 3 Action At the local VT100 console Log into the SDM CS 2000 Core Manager as a root user. Access the maintenance interface, type # sdmmtc Determine if the core manager maintenance interface is available. If core manager maintenance interface is available, go to the next step. If core manager maintenance interface is not available, go to step 10. 4 5 Access the administration level, type admin. Perform a full restore of the core manager, type restore Sample responseSelect the tape drive you want to restore from, or type Abort to abort: Enter 0 for the tape drive in the main chassis slot 2. Enter 1 for the tape drive in the main chassis slot 3. 6 Type the number for the tape drive to use.

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

266

Preparation for disaster recovery

To use the tape drive in slot 2, type 0 To use the tape drive in slot 13, type 1 Sample responseInsert the backup-tape into the tape drive in the main chassis slot 2. When completed press [Enter] to start the restore. 7 When prompted, type y Sample response Insert the backup-tape into the tape drive in the main chassis slot 2. When completed press [Enter] to start the restore. 8 Insert the backup S-tape into the specified tape drive. Wait until the tape drive stabilizes (yellow LED is off) before you proceed. 9 10 Press Enter to start the restore process. Go to step 16. At the core manager The core manager maintenance interface is not available (step 2), make sure that one of the core manager tape drives (slot 2 or 13 in the main chassis) contains the system image backup tape (S-tape). Use tape drive DAT0 (option to perform a full restore from an S-tape) if your system also includes SBA. At the Modular Supervisory Panel 11 Reboot the core manager. If the prompt is available at a local VT100 console, reboot the core manager, type shutdown -Fr. If the prompt is not available, reboot the core manager by turning the power off, then on. Use the MSP breaker that supplies power to the core manager to turn the power off and on. At the local VT100 console 12 When the system COLD Start message appears, interrupt the boot process. Interrupt the process by press Esc twice. The system takes about 4 minutes to initialize. Proceed according to he system prompt. 13 If the prompt is FX-Bug and you are in a menu, type the number to select Go to System Debugger. Go to step 15. 14 If the prompt is FX-Diag, Switch the directory to FX-Bug, type > sd
Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

Procedure steps

267

15

View the input/output devices on the core manager to verify the address of the tape drive from the FX-Bug prompt, type Fx-Bug> ioi If the FX-Bug prompt appears (response from step 12), type Fx-Bug> pboot <address_for_Archive_Python> Valid choices are: pboot 1 50 if the tape drive is located in slot 2 pboot 6 50 if the tape drive is located in slot 13 Wait for the system to reboot. This takes approximately 4 minutes.

16

17

After the system reboots, you are prompted to define the console setting and the language setting. Define the console setting, type 1. The system does not echo your entry. In case of any failures, contact your next level of support.

18

To select the language setting, type 1. The Welcome to Base Operating System Installation and Maintenance menu appears.

19

Type the number to select the Change/Show Installation Settings and Install menu option. The System Backup Installation and Settings menu appears. Sample response System Backup Installation and Settings Either type 0 and press Enter to install with the current settings, or type the number of the setting you want to change and press Enter. Setting: Current Choice(s): 1 Disk(s) where you want to install hdisk0... Use Maps No 2 Shrink File System No >>> 0 Install with the settings listed above. The elipses (...) under Current Choice(s) indicates that more than one disk is currently in use.

20

The default disk hdisk0, which is located in slot 2 of the main chassis. If your core manager contains one disk drive in each domain of the main chassis, accept the default setting. If you have additional disk drives, you may wish to change the settings. If you want to accept the default setting, go to step 27. If you want to change the setting, type

21

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

268

Preparation for disaster recovery

>1 The Change Disk(s) Where You Want to Install menu appears, with a list of all the available disks on which you can install the system backup image. The >>> symbols identify the currently selected disks. 22 You must install the system backup on one disk in each domain to achieve fault-tolerant operation. Valid choices in response to the prompt in the Change Disk(s) Where You Want to Install menu are, are hdisk0 and hdisk5. The rootvg disks for installation must have location codes c1-f2-00-0 for domain 0 c1-f13-00-0 for domain 1 23 Type the number to select one or more disks, and press Enter. To undo a disk you selected, type the number again and press Enter. After you select the disk, the System Backup Installation and Settings menu appears. 24 25 At the prompt. type 0 and return to step 20. To accept the default setting (step 21), type 0 The restore process begins, and takes at least 30 minutes to complete. The progress monitor shows the approximate percentage of the tasks completed, and the elapsed time. If an error message appears at the end of the restore process, datavg did not import successfully. Contact the next level of support. If you are performing this procedure as part of removing an I/O expansion chassis (Upgrades, (NTRX50EC)) you must manually reboot the system. At the FX-bug> prompt, type pboot 1 0. At the core manager 26 Remove the S-tape from the tape drive and store it in a secure location. At the local or remote terminal 27 28 29 30 Log in to the core manager as a root user, press Enter when you see the TERM=(vt100) prompt. Start the core manager maintenance interface, type # sdmmtc Access the storage level, type > storage Determine the mirror status of the disks.

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

Procedure steps

269

ATTENTION
If the status of the disks is Integrating or Not Mirrored, stop this procedure. You cannot continue until the status is Mirrored. If necessary, contact your next level of support.

31 32

Access the LAN level, type > lan Check the state of DCE. Sample response DCE State: SysB

33

Access the application (APPL) level to check the state of any DCE-based applications, type > appl Sample response # Application State 1 Table Access Service . 2 Log Delivery Service . 3 OM Access Service . 4 Secure File Transfer Fail 5 Enhanced Terminal Access Fail 6 Exception Reporting Fail Applications showing 1 to 6 of 6

ATTENTION
DCE and DCE-based applications can fail if the key tab files restored from tape contain obsolete keys.

34

If the DCE state is SysB at the LAN menu level of the RMI (response from step 32), and the logs indicate an error with the security client service in DCE, restore the service. If the state of some DCE-based applications Fail (response from step 32), try to restore them by busying (BSY) and returning to service (RTS) the applications from the SDM APPL level. If that , restore them by performing the procedure to add the application server to the DCE cell in the Configuration Management document. You must create a new system image backup tape.
--End--

35

36

37

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

270

Preparation for disaster recovery

Performing a partial restore of the software from S-tape


Perform a partial restore of the software from S-tape to restore individual files or sets of files from the system image backup tape (S-tape).

Prerequisites

You must be a user authorized to perform fault-admin actions.

CAUTION
Possible loss of data
Use this procedure at the discretion of the system administrator.

Perform a partial restore only if you are familiar with the files, and know exactly which files are to be restored. If you restore the wrong files, you may inadvertently corrupt core manager software.

ATTENTION
This procedure must be performed by a trained AIX system administrator authorized to perform fault-admin actions.

ATTENTION
All volume groups on the core manager must be fully mirrored (Mirrored) before performing this procedure. If you attempt to perform this procedure when disk mirroring is not Mirrored, an error message is displayed on the screen.

ATTENTION
If your system includes the SuperNode Billing Application (SBA), use tape drive DAT0 to perform this procedure.

ATTENTION
Instructions for entering commands in the following procedure do not show the prompting symbol, such as #, >, or $, displayed by the system through a GUI or on a command line.

Procedure steps
Step 1 Action At the core manager insert the S-tape into the tape drive you selected.

ATTENTION
Wait until the tape drive stabilizes (yellow LED is off) before you proceed.

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

Procedure steps

271

At the local or remote VT100 console when you are ready to continue this procedure, enter:
y

The contents of the S-tape are listed on the screen. When the listing is complete, the system prompts you to return to the System Image Backup and Restore Menu. Example response:
Would you like to return to the previous menu? (y|n)

3 4 5 6

Return to the System Image Backup and Restore Menu:


y

If you are listing the contents of the tape to verify that the file has been restored then go to step 14. If you are listing the contents of the tape to verify the file name or directory that you wish to restore go to step 6. From the System Image Backup and Restore Menu, select "Restore Files from the System Image Tape (S-tape)":
4

After you select option 4, you are prompted to select the tape drive. Example response:
Select a tape drive you wish to use: Enter 0 to return to previous menu Enter 1 for tape drive DAT0 in Main Chassis-Slot 2 Enter 2 for tape drive DAT1 in Main Chassis-Slot 13 ( 0, 1 or 2 ) ==>

ATTENTION
Use tape drive DAT0 (option 1) if your system also includes SBA.

Enter the number for the tape drive you want to use (1 or 2).

ATTENTION
If your system includes SBA, and you still wish to use tape drive DAT1 (option 2), the following message is displayed:

Example response:
You have selected DAT 1. This is the default DAT drive for the Billing application, and may currently be in use for the emergency storage of billing records.

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

272

Preparation for disaster recovery If you continue to use DAT 1, make sure that the correct tape is in the drive, and that billing records will not be lost during the backup/restore operation. Do you wish to continue with DAT 1? ( y | n )

if you wish to continue using DAT1, enter y I If you do not wish to use DAT1, enter n

The system prompts you to return to the System Image Backup and Restore Menu if you do not wish to use DAT1. 9 After you select the tape drive, you are prompted to insert the S-tape into the appropriate tape drive. A warning is displayed advising that this procedure must only be completed by qualified core manager system administrators. The warning also advises that files and directories must be entered exactly as they appear in the file listing. Insert the S-tape in the appropriate tape drive.

ATTENTION
Wait until the tape drive stabilizes (yellow LED is off) before you proceed.

Example response:
Are you ready to enter the name of the file or directory? (y|n)

10

Continue this procedure:


y

Example response:
Enter the name of the directory or file that you wish to restore as ./<your-full-path>/<your-file-or-directory>. Note: Tape processing may take a few minutes to complete. A log file /tmp/s-tape.restore will be created. ==>

11

Enter the full path name of the directory or file that you wish to restore, exactly as shown in the file listing, including "/" at the beginning.

ATTENTION
A log file /tmp/s-tape.restore is created when the restore is completed.

ATTENTION
An error message is displayed if the restore is unsuccessful. If this occurs, go to step 25.

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

Procedure steps

273

12

During the restore process, the screen does not display any additional information. When the file restore is complete, the file you have restored is displayed. The system then prompts you to return to the System Image Backup and Restore Menu. Example response:
Would you like to return to the previous menu? ( y | n )

ATTENTION
If the restore has failed, an error message is displayed before the prompt, advising you to list the contents of the tape, and perform the procedure again.

13 14

Return to the System Image Backup and Restore Menu:


y

Determine if the restore was successful. The system displays the file that you have restored. You may also wish to view the s-tape.restore file or list the files on the S-tape. If the restore is successful then go to step 25. If the restore failed then contact your next level of support If you wish to view the s-tape.restore file then go to step 18. Exit the System Image Backup and Restore Menu:
0

15 16 17 18 19 20 21 22 23 24 25

Exit the maintenance interface:


quit all

Access the s-tape.restore file:


cd /tmp

Scroll through the file:


more s-tape.restore

Continue pressing the Enter key until the files that you have restored, and the date of the restore, are displayed. If successful then go to step 26. If failed then go to contact your next level of support. Exit the System Image Backup and Restore Menu:
0

ATTENTION
If you then wish to exit the maintenance interface, type quit all and press the Enter key.

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

274

Preparation for disaster recovery

26

At the core manager remove the S-tape and store it in a secure place.
--End--

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

Procedure steps

275

Recovering backup les from lost backup volumes


Recover backup files from lost backup volumes swaps back old volumes as the primary backup volumes. You must recover backup files from lost backup volumes if the SBA cannot track backed up files when the SWACT and RESTART processes occur when you perform procedure "Configuring SBA backup volumes" in the Accounting document.

Prerequisites

You must have the names of the swapped out volumes. For more information see, "Configuring SBA backup volumes on the core" in the Accounting document.

Procedure steps
Step 1 2 3 Action Post the billing stream. > mapci;mtc;appl;sdmbil;post <stream_name> Quit back to the appl;sdmbil level.
> quit

Confirm that the names of the billing streams existing backup volumes are the swapped in volumes you created earlier. > conf view <stream_name>

ATTENTION
SBA does not support the configuration of more than one billing stream at a time from multiple workstations. The last billing stream that is configured is the one that is saved.

Read the notes you made when you performed the procedure, "Configuring SBA backup volumes on the core" in the Accounting document, to confirm that the backup volumes that you listed in step 3 are the backup volumes you created with that procedure. If the backup volume names are the backup volume names configured when you performed the procedure then continue with step 7. If the backup volume names are not the backup volume names configured when you performed the procedure then determine if someone else reconfigured the backup volumes before you continue with step 7.

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

276

Preparation for disaster recovery

7 8

Access the billing level.


> mapci;appl;sdmbil

Configure the billing stream of the logical volumes you created when you performed the procedure, "Configuring SBA backup volumes on the core" in the Accounting document. > addvol <stream_name> <volume1> ... <volume5> Example Response: To add five volumes, the command is: addvol AMA S00DAMA S01DAMA S02DAMA S03DAMA S04DAMA Repeat this step until all of the volumes have been added to the stream.

Exit back to the command prompt.


> quit all

ATTENTION
The non-empty backup volumes are automatically detected by the SBA audits. In addition, the SBA places the billing stream into recovery mode and the volumes from the original backup volumes are sent to the core manager.

ATTENTION
You must provide, to all operating company personnel who work on the core, the names of the old and new backup volumes and the procedure you used to swap the volumes. They must be made aware that any restarts or switch of activity (SwAct) that occur before the billing stream returns to normal mode can cause a serious loss of billing records. The mode of the billing stream must be closely monitored to ensure that it returns to normal mode without an intervening RESTART or SwAct.

--End--

Variable denitions
Variable <stream_name> <volume5> Value Name of the billing stream. Volume name. Up to five volumes (with each entry separated by a space) can be added at one time.

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

Procedure steps

277

Restoring the Oracle/MySQL data on an SPFS-based server


Restore the Oracle/MySQL data from a backup DVD on a Server Platform Foundation Software (SPFS)-based server, Sun Netra 240). Also use this procedure to restore the data that was automatically backed up to a file on the server by the backup restore manager. The server can be hosting one or more of the following components:

Audio Provisioning Server (APS) Core and Billing Manager (CBM) Communication server 2000 Management Tools Communication server 2000 SAM21 Manager Integrated Element Management System (IEMS) Media Gateway (MG) 9000 Manager Multiservice Data Manager (MDM) Network Patch Manager

Although the MDM, MG 9000 Manager and CBM do not host a database, this restore procedure still applies to these components as it can be used to recover other critical files.

Prerequisites

You need the DVD on which you backed up the Oracle/MySQL data. If the data was backed up by the backup restore manager, you need the name of the file located in directory /data/bkresmgr/backup.

Procedure steps
Step 1 2 Action Insert the DVD on which you backed up the Oracle/MySQL data, into the drive. Log on to the server. > telnet <server> Press the Enter key. 3 When prompted, enter your user ID and password.

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

278

Preparation for disaster recovery

Change to the root user.


$ su -

and press the Enter key. 5 6 When prompted, enter the root password. Verify the permissions on the restore log directory (bkslog).
# ls -alrt /var/opt/nortel

and press the Enter key. Example response:


total 22 lrwxrwxrwx 1 root succssn 28 Dec 22 2003 gwc -> /net/47.141.126.131//swd/gwc drwxrwxr-x 4 drwxr-xr-x 7 drwxr-xr-x 2 drwxrwxrwt 2 root other 512 Dec 22 2003 . root sys 512 Sep 10 12:41 .. oracle oinstall 512 Dec 8 15:58 db root other 6656 Dec 15 19:56 bkslog

7 8

If the permissions of bkslog are drwxrwxrwt then go to step 10 If the permissions of bkslog are not drwxrwxrwt, then change the permissions of bkslog.
# chmod 1777 /var/opt/nortel/bkslog

and press the Enter key. 9 Determine if server applications have been stopped.
# servquery -status all

and press the Enter key. 10 11 If not already done, stop the server applications that run on the server. For Communication server 2000 Management Tools server applications see Stopping the SESM server application, Stopping the SAM21 Manager server application, Stopping the APS server application, Nortel ATM/IP Solution-level Administration and Security ( (NN10402-600)) , . For MG 9000 Manager and mid-tier server application see Nortel MG 9000 Fault Management ( (NN10074-911)) , Nortel MG 9000 Administration and Security ( (NN10162-611)) . For IEMS server application see Stopping the NPM server application, Nortel IEMS Administration and Security ( (NN10336-611)) . Verify the permissions, owner and group of /data/oradata by typing:
#ls -l /data

12

13

14

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

Procedure steps

279

and pressing the Enter key. Example response:


drwxr-xr-x 7 oracle oinstall 512 Oct 16 18:13 oradata

15

Restore the database from backup DVD.


$ /opt/nortel/sspfs/bks/rsdata

and press the Enter key. Example response:


Done patching database DATABASE re-started successfully Restoring critical files from opt/critdata.cpio 2384 blocks !! WARNING !! Restoring /etc/passwd, /etc/shadow or /etc/group files could cause problems with respect to passwords, files ownership. Do you want to restore these files (y/n):[y]n

16 17

The DVD should eject automatically. Remove the DVD from the drive and proceed to the next step. Restore the database from the backup file.
$ /opt/nortel/sspfs/bks/rsdata -f /data/bkresmgr/bac kup/<filename>

and press the enter key. 18 19 20 If an alarm appears, or the database did not restore properly, contact Nortel. Otherwise, continue with the next step. If no alarm appears, and you do not need to restore the certificate, continue with the next step. If no alarm appears, and you need to restore the certificate, determine if WebPKProxy is configured.
# ls /data/pkclient/certificates

and press the Enter key. 21 22 23 If the files CA1_Cert.pem, CA2_Cert.pem, and TRUSTED_Cert.p em do not exist, continue with the next step. If the three files listed in step 17 are associated with this machine, reconfigure the WebPKProxy as follows. From the CLI, select Configuration ---> Security Services, Configuration ---> PKManager Certificate, Installation ---> install_certs (Install Certificates supplied by PKManager). Wait for the script to complete. Verify that no errors are displayed.

24

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

280

Preparation for disaster recovery

25

From the CLI, select Configuration ---> Security Services, Configuration ---> WebPKProxy/PKClient, Configuration---> register_pkclient. Enter the host name of the Certificate Manager. The Certificate Manager box must be fully functional for this step to succeed. Wait for the script to complete. Verify that no errors are displayed.

26 27

28

Start the server applications that run on the server. If one or more applications do not start, contact Nortel for assistance. If the files /etc/passwd, /etc/group were backed-up previously, the process prompts the following warning message:
WARNING !! Restoring /etc/passwd, /etc/shadow or /etc/group files could cause problems with respect to passwords, files ownership. Do you want to restore these files (y/n):[y] Select: n - No, if you have changed some passwords, added/changed new users/groups since the last backup. y - Yes, otherwise.

29

For Communication server 2000 Management Tools server applications see Starting the SESM server application, Starting the SAM21 Manager server application, Starting the APS server application, Nortel ATM/IP Solution-level Administration and Security ( (NN10402-600)) For MG 9000 Manager and mid-tier server applications see Nortel MG 9000 Fault Management ( (NN10074-911)) , Nortel MG 9000 Administration and Security ( (NN10162-611)) . For IEMS server application see Starting the NPM server application (page 297) Nortel IEMS Administration and Security ( (NN10336-611)) : "Starting the IEMS server or changing the IEMS server mode to standby."
--End--

30

31

Variable denitions
Variable <filena me> Value The name of the backup file created by the backup restore manager.

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

Procedure steps

281

Clearing the MATE alarm


Use this procedure to clear the MATE alarm.

Procedure steps
At the workstation UNIX prompt or VT-100 terminal prompt:

Step 1 2

Action Log in to the CBM using the root or emsadmin user ID and password. Start the cbmmtc tool by typing:
# cbmmtc

Check the MATE column on the banner. If the state is not "." (dot), this indicates the presence of an alarm. 3 4 5 6 Access the MATE by performing the procedure Accessing the MATE (page 222). Clear the alarms by performing the procedure Clearing the MATE alarm (page 281). Check to see if all fault conditions have now cleared.
querycbm flt

Log out of the MATE.


--End--

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

282

Preparation for disaster recovery

Stopping the SAM21 Manager server application


Stop the SAM21 Manager server application to end the SAM21 Manager server application on the SPFS-based server that is hosting the Communication Server 2000 Management Tools.

Procedure steps
Step 1 Action At your workstation, establish a logon session to the server, using one of the following methods In a two-server configuration, log in to the active server using the physical IP address of the active server, and ensure you are on the active server using the ubmstat command. 2 Verify the status of the SAM21 Manager server application by typing:
# servman query -status -group SAM21EM

and pressing the Enter key. 3 If SAM21EM is running continue with the next step and if SAM21EM is not running then you have completed this procedure. Stop the SAM21EM server application by typing:
# servstop SAM21EM

and pressing the Enter key. 5 Verify the SAM21EM server application stopped by typing:
# servman query -status -group SAM21EM

and pressing the Enter key. Example response:


# servman query -status -group SAM21EM Executing: /opt/servman/bin/servquery -status -group SAM21EM SAM21EM =============================== Executing : /opt/nortel/sam21em/bin/health Succession SAM21 Element Manager version: SAM21EM_10_0 39_0 Current status of the Succession SAM21 Element Manager: SNMP Access Gateway is not running. SAM21 Element Manager Server is not running.

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

Procedure steps

283

If applicable, return to the high level task or procedure that directed you to this procedure.
--End--

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

284

Preparation for disaster recovery

Stopping the SESM server application


Stop the Succession Element and Subelement Manager (SESM) server application to end the SESM server application on the CS 2000 Management Tools server.

Prerequisites

You must be logged in to the active server as the root user using telnet (unsecure) or ssh (secure).

Procedure steps
ATTENTION
In a two-server configuration, you must perform this procedure on the active server.

Step 1

Action Ensure you are logged in to the active server.


ubmstat

If the server response is ClusterIndicatorACTthen you are on the active server. If the server response is ClusterIndicatorSTBY then you are on the inactive server. You must be logged in to the active server. 2 3 Verify the status of the SESM server application by typing:
# servman query -status -group SESMService

If SESM service is running then continue with step 4 and if SESM service is not running then you have completed this procedure. Stop the SESM server application by typing:
# servstop SESMService

and pressing the Enter key. 5 6 Wait approximately 3 to 5 minutes before you proceed to the next step to allow the SESM server application to stop. To verify the SESM services stopped by typing:
# servman query -status -group SESMService

and pressing the Enter key. Example response:

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

Procedure steps CS2K Management Tools are not running

285

Verify the SESM applications stopped by typing:


# ptmctl status

and pressing the Enter key Example response:


SESM STATUS ------------------------COMPONENT STATUS --------- -----Proxy Agent NOT RUNNING RMI Regisry NOT RUNNING Snmpfactory NOT RUNNING MI2 Server NOT RUNNING Current number of SESM processes running: 0 (of 4) SESM APPLICATION STATUS: No Applications are ready

If applicable, return to the high level task or procedure that directed you to this procedure.
--End--

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

286

Preparation for disaster recovery

Stopping the NPM server application


Use this procedure to stop the Network Patch Manager (NPM) server application on a Server Platform Foundation Software (SPFS)-based server.

Prerequisites

You need root user privileges to perform this procedure. It is recommended that all users exit the NPM CLUI and GUI before stopping the NPM server application.

Procedure steps
ATTENTION
In a two-server configuration, perform the steps that follow on the Active server.

Step 1

Action At your workstation, to log in to the server type > telnet <server> and press the Enter key. In a two-server configuration, enter the physical IP address of the Active server (unit 0 or unit 1).

2 3

When prompted, enter your user ID and password. Change to the root user by typing:
$ su - root

and pressing the Enter key. 4 When prompted, enter the root password. In a two-server configuration, ensure you are on the Active server by typing ubmstat. If ClusterIndicatorSTBY is displayed in the response, which indicates you are on the Inactive server, log out of that server and log in to the other server through telnet using the physical IP address of the other unit. The response must display ClusterIndicatorACT, which indicates you are on the Active server. 5 Verify the status of the NPM server application by typing:
# servman query -status -group NPM

and pressing the Enter key.

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

Variable definitions

287

If the NPM server application is running then continue step 7 and if the NPM server application is not running then you have completed this procedure. Stop the NPM server application by typing:
# servstop NPM

and pressing the Enter key. 8 Verify the NPM server application is no longer running by typing:
# servman query -status -group NPM

and pressing the Enter key. Example response


NPM =========================== Executing: /op/NTnpm/.HA_NPM_status NPMServer is not running

--End--

Variable denitions
Variable <server> Value IP address or host name of the SPFS-based server where the NPM server application resides.

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

288

Preparation for disaster recovery

Stopping the APS server application


Stop the APS server application to end the APS server application on the Server Platform Foundation Software (SPFS)-based server. The APS resides on the same server as the Communication S 2000 Management Tools software (CS2M).

Prerequisites

Before performing this procedure, verify that the APS server application is running. Enter:
# servman query -status -group APS

You must have root user privileges to perform this procedure.

Procedure steps
ATTENTION
In a two-server configuration, perform the steps that follow on the Active server.

Step 1

Action At your workstation to log in to the server type > telnet <server> and press the Enter key. In a two-server configuration, enter the physical IP address of the Active server (unit 0 or unit 1).

2 3

When prompted, enter your user ID and password. Change to the root user by typing:
$ su - root

and pressing the Enter key. 4 When prompted, enter the root password. In a two-server configuration, ensure you are on the Active server by typing ubmstat. If ClusterIndicatorSTBY is displayed in the response, which indicates you are on the Inactive server, log out of that server and log in to the other server through telnet using the physical IP address of the other unit. The response must display ClusterIndicatorACT, which indicates you are on the Active server. 5 Stop the APS server application by typing:
# servstop APS

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

Variable definitions

289

and pressing the Enter key. 6 Verify the APS server application stopped by typing:
# servman query -status -group APS

and pressing the Enter key. Example response


Executing : /opt/servman/bin/servquery -status -group APS APS ====================================== Executing : /usr/ntdb/uas/scripts/apsHealth.ksh Oracle Instance is DOWN Oracle Listener is DOWN. PROBLEM: Oracle is not running. The APS requires ORACLE to be up.

--End--

Variable denitions
Variable <server> Value IP address or host name of the SPFS-based server where the APS resides.

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

290

Preparation for disaster recovery

Starting the APS server application


Starting the APS server application to begin the APS server application on the Server Platform Foundation Software (SPFS)-based server where the APS resides. The APS resides on the same server as the Communication Server 2000 Management Tools software (CS2M).

Procedure steps
ATTENTION
In a two-server configuration, perform the steps that follow on the Active server.

Step 1

Action At your workstation, log on to the server by typing: > telnet <server> and pressing the Enter key. In a two-server configuration, enter the physical IP address of the Active server (unit 0 or unit 1).

2 3

When prompted, enter your user ID and password. Change to the root user by typing:
$ su - root

and pressing the Enter key. 4 When prompted, enter the root password. In a two-server configuration, ensure you are on the Active server by typing ubmstat. If ClusterIndicatorSTBY is displayed in the response, which indicates you are on the Inactive server, log off of that server and log on to the other server through telnet using the physical IP address of the other unit. The response must display ClusterIndicatorACT, which indicates you are on the Active server. 5 Verify the status of the APS server application by typing:
# servman query -status -group APS

and pressing the Enter key. 6 7 If APS server is not running then continue the next step and if APS server is running then you have completed this procedure. Start the APS server application by typing:
# servstart APS

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

Procedure steps

291

and pressing the Enter key. 8 Verify the APS server application started by typing
# servman query -status -group APS

and pressing the Enter key. Example response:


APS =============================== Executing : /usr/ntdb/uas/scripts/apsHealth.ksh Oracle Listener and DB appear are up. APS server processes: APS PID files are on the filesystem. -rw-rw-rw- 1 root other 6 Jan 25 06:00 /opt/uas/aps/scrip ts/runAdmanager.pid -rw-rw-rw- 1 root other 6 Jan 27 03:06 /opt/uas/aps/scr ipts/runAlertHandler.pid -rw-rw-rw- 1 root other 6 Jan 24 22:56 /opt/uas/aps/sc ripts/runApsDbServer.pid -rw-rw-rw- 1 root other 6 Jan 24 22:56 /opt/uas/aps/scri pts/runDbServer.pid -rw-rw-rw- 1 root other 6 Jan 24 22:56 /opt/uas/aps/scrip ts/runFileUpload.pid -rw-rw-rw- 1 root other 6 Jan 24 22:56 /opt/uas/aps/scrip ts/runHeartbeat.pid -rw-rw-rw- 1 root other 6 Jan 24 22:56 /opt/uas/aps/sc ripts/SnmpAgent.pid Checking process: /opt/uas/aps/scripts/runApsDbServe r.pid The runApsDBServer process, PID: 15012 is ACTIVE and RUNNING. Checking process: /opt/uas/aps/scripts/runDbServer. pid The runDBServer process, PID: 14456 is ACTIVE and RUNNING. Checking process: /opt/uas/aps/scripts/runFileUploa d.pid The runFileUpload process, PID: 14826 is ACTIVE and RUNNING. Checking process: /opt/uas/aps/scripts/runAdmanager .pid The ADmanager process, PID: 23018 is ACTIVE and RUNNING. Checking process: /opt/uas/aps/scripts/runHeartbeat.p id The Heartbeatmon process, PID: 15403 is ACTIVE and RUNNING. Checking process: /opt/uas/aps/scripts/SnmpAgent.pid The SnmpAgent process, PID: 14271 is ACTIVE and RUNNING.

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

292

Preparation for disaster recovery APS is up and running.

--End--

Variable denitions
Variable <server> Value IP address or host name of the SPFS-based server where the APS resides.

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

Procedure steps

293

Starting the SESM server application


Start the Succession Element and Subelement Manager (SESM) server application to begin the SESM server application on the SPFS-based server.

Procedure steps
Step 1 Action At your workstation, establish a connection to the server that is hosting the CS 2000 Management Tools through telnet or SSH, and log on using the root user ID and password. In a two-server configuration, log on to the active server using the physical IP address of the active server, and ensure you are on the active server using the ubmstat command. 2 Verify the status of the SESM server application by typing:
# servman query -status -group SESMService

and press the Enter key. 3 4 If the response indicates not running then go to step 4 and if the response indicates running then go to step 7. Start the SESM server application by typing:
# servstart SESMService

and press the Enter key. 5 6 Wait approximately 3 to 5 minutes before you proceed to the next step to allow the SESM server application to start. To Verify the SESM server application started verify the SESM services started by typing:
# servman query -status -group SESMService

and press the Enter key. Example response:


CS2K Management Tools are running

Verify the SESM applications started by typing:


# ptmctl status

and press the Enter key. Example response:


SESM STATUS -------------------------

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

294

Preparation for disaster recovery COMPONENT STATUS -------------Proxy Agent RUNNING RMI Regisry RUNNING Snmpfactory RUNNING MI2 Server RUNNING Current number of SESM processes running: 4 (of 4) SESM APPLICATION STATUS: All Applications Ready

--End--

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

Procedure steps

295

Starting the SAM21 Manager server application


Start the SAM21 Manager server application to begin the SAM21 Manager server application on the SPFS-based server that is hosting the Communication Server Management Tools.

Procedure steps
Step 1 Action At your workstation, establish a connection to the server that is hosting the Communication Server Management Tools through Telnet or SSH, and log on using the root user ID and password. In a two-server configuration, log on to the active server using the physical IP address of the active server, and ensure you are on the active server using the ubmstat command. 2 Verify the status of the SAM21 Manager server application by typing:
# servman query -status -group SAM21EM

and pressing the Enter key. 3 4 If SAM21EM is not running then continue the next step and if the SAM21EM is running then you have completed this procedure. Start the SAM21EM server application by typing:
# servstart SAM21EM

and pressing the Enter key. 5 Verify the SAM21EM server application started by typing:
# servman query -status -group SAM21EM

and pressing the Enter key. Example response:


Executing: /opt/servman/bin/servquery -status -group SAM21EM SAM21EM =============================== Executing : /opt/nortel/sam21em/bin/health Succession SAM21 Element Manager version: SAM21EM_9_ 020_0 Current status of the Succession SAM21 Element Manager: SNMP Access Gateway is running. SAM21 Element Manager Server is running.

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

296

Preparation for disaster recovery

If applicable, return to the high level task or procedure that directed you to this procedure.
--End--

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

Procedure steps

297

Starting the NPM server application


Start the Network Patch Manager (NPM) server application to begin the NPM server application on a Server Platform Foundation Software (SPFS)-based server.

Prerequisites

You need root user privileges to perform this procedure, and CORBA must be running in order for the NPM to come up.

Procedure steps
ATTENTION
In a two-server configuration, perform the steps that follow on the active server.

Step 1

Action At your workstation, log on to the server by typing: > telnet <server> and pressing the Enter key. In a two-server configuration, enter the physical IP address of the Active server (unit 0 or unit 1).

2 3

When prompted, enter your user ID and password. Change to the root user by typing:
$ su - root

and pressing the Enter key. 4 When prompted, enter the root password. In a two-server configuration, ensure you are on the Active server by typing ubmstat. If ClusterIndicatorSTBY is displayed in the response, which indicates you are on the Inactive server, log out of that server and log on to the other server through telnet using the physical IP address of the other unit. The response must display ClusterIndicatorACT, which indicates you are on the active server. 5 Verify the status of the NPM server application by typing:
# servman query -status -group NPM

and pressing the Enter key. 6 If the NPM server application is not running then continue the next step and if the NPM server application is running then you have completed this procedure.

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

298

Preparation for disaster recovery

Start the NPM server application by typing:


# servstart NPM

and pressing the Enter key. 8 Verify the NPM server application is running by typing:
# servman query -status -group NPM

and pressing the Enter key. Example response:


Executing: /opt/servman/bin/servquery -status -group NPM NPM =============================== Executing : /opt/NTnpm/.HA_NPM_status The NpmServer is running

--End--

Variable denitions
Variable <server> Value IP address or host name of the SPFS-based server where the NPM server application resides.

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

Variable definitions

299

Performing a full system restore on an SPFS-based server


Use this procedure to perform a full system restore from backup media on a Server Platform Foundation Software (SPFS)-based server (Sun Netra 240). A full system restore consists of reverting to the previous release of SPFS, restoring the file systems, and restoring the system data.

ATTENTION
After performing a full system restore on IEMS you must also perform a subsequent central server restore. See "Restoring the Central Security Server" in Nortel IEMS Administration and Security ( (NN10336-611)) .

The server can be hosting one or more of the following components:

Integrated Element Management System (IEMS) Audio Provisioning Server (APS) Media Gateway (MG) 9000 Manager Network Patch Manager Core Billing Manager (CBM) Multiservice Data Manager (MDM)

Restoring the oracle data is not applicable to the CBM as it does not use an oracle database.

ATTENTION
System logs indicating application and database errors will be generated until the file systems and oracle data are restored on the system using this procedure and procedure Restoring the Oracle/MySQL data on an SPFS-based server. No database errors will be generated on the CBM as it does not use an oracle database.

ATTENTION
System logs indicating application and database errors will be generated until the file systems and oracle data are restored on the system using this procedure and procedure Restoring the Oracle/MySQL data on an SPFS-based server. No database errors will be generated on the CBM as it does not use an oracle database.

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

300

Preparation for disaster recovery

Prerequisites

you need SPFS CD disk #1 or SPFS DVD for the release you are reverting to you need the DVD on which you backed up the file systems you need the DVD on which you backed up the system data

ATTENTION
The recovery must be performed on the system where the backup was created. Recovery cannot be performed on a different system.

Procedure steps
ATTENTION
For MDM and other components that use a simplex configuration with no oracle database, complete ONLY steps 1-8. If the additional steps are executed, errors will occur. Simplex conguration

Step 1 2 3 4 5

Action At the server console, log in to the server through the console (port A) using the root user ID and password. Bring the system to the OK prompt by typing:
# init 0

Insert SPFS CD disk#1 or SPFS DVD into the drive. At the OK prompt, restore the system by typing: OK boot cdrom - restore When prompted, accept the software license restrictions by typing:
ok

and pressing the Enter key. The system reboots. If restoring from DVD, you will be prompted to insert Volume 1 of the backup DVD into the drive. Insert the DVD on which you backed up the file systems. During the restore process, the system will prompt you for additional Volumes if more than one DVD was used during the backup of file systems. The restore process can take several hours to complete depending on the number and size of the files that are being restored.

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

Procedure steps

301

Although it can appear as if the system is hanging at times, do not interrupt the restore process. If you suspect an issue with the restore process, please contact your next level of support. 6 7 8 Eject the backup DVD from the drive. Ensure you are at the root directory level by typing:
# cd /

Eject the DVD by typing:


# eject cdrom

If the DVD drive tray will not open after you have determined that the DVD drive is not busy and is not being read from or written to, enter the following commands:
# /etc/init.d/volmgt stop # /etc/init.d/volmgt start

Then press the eject button located on the front of the DVD drive. 9 10 11 12 13 14 15 Remove the backup CD or DVD from the drive. If the SPFS-based server is hosting the CBM then do step 12. If the server is hosting anything else then you have completed this procedure. List the oracle groups by typing:
# groups oracle

If the output is install data dba or install dba data then do step 16. If the output is otherwise then do step 15. Correct the oracle groups by typing:
# usermod -g oinstall -G data,dba oracle

and pressing the Enter key. 16 17 18 Restore the oracle data using procedure Restoring the Oracle/MySQL data on an SPFS-based server. Reboot the server by typing:
# init 6

Restore the central security server. For more information, see "Restoring the Central Security Server" in Nortel IEMS Administration and Security ( (NN10336-611)) .
--End--

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

302

Preparation for disaster recovery

High-availability conguration (two servers)


Use this procedure for high-availability configuration (two servers).

Procedure steps
ATTENTION
The recovery must be performed on the same unit where the backup was created.

Step 1

Action At the console connected to the inactive node, log in to the inactive node through the console (port A) using the root user ID and password. Bring the system to the OK prompt by typing:
# init 0

2 3

At the console connected to the active node, log in to the active node through the console (port A) using the root user ID and password. Bring the system to the OK prompt by typing:
# init 0

4 5 6 7

Insert SPFS CD disk#1 or SPFS DVD into the drive. At the OK prompt, restore the system by typing: OK boot cdrom - restore When prompted, accept the software license restrictions by typing:
ok

The system reboots. 8 When prompted, insert Volume 1 of the DVD on which you backed up the file systems into the drive. During the restore process, the system will prompt you for additional Volumes if more than one DVD was used during the backup of file systems. The restore process can take several hours to complete depending on the number and size of the files that are being restored. Although it can appear as if the system is hanging at times, please do not interrupt the restore process. If you suspect an issue with the restore process, contact your next level of support. 9 Ensure you are at the root directory level by typing:

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

Procedure steps # cd /

303

10

Eject the DVD by typing:


# eject cdrom

If the DVD drive tray will not open after you have determined that the DVD drive is not busy and is not being read from or written to, enter the following commands:
# /etc/init.d/volmgt stop # /etc/init.d/volmgt start

Then press the eject button located on the front of the DVD drive. 11 12 Remove the backup DVD from the drive. Remove the backup CD or DVD from the drive. Ensure the replicated file systems on the newly restored unit are active.
# cd /opt/nortel/sspfs/ha ./ActivateNode

The output from execution of the ActivateNode script is as follows, and the user must enter Y to the prompt:
This procedure may cause data loss. Are you sure you want to proceed? [Y/N]

13

Confirm the prompt.


Y Executing irrevocable procedure udactive.....

14

Execute the following command to confirm that all replicated file systems are active.
# udstat /audio_files: ACTIVE repair DOWN dirty /data: ACTIVE repair DOWN dirty /data_ordata (/data/ordata): ACTIVE repair DOWN dirty /user_audio_files: ACTIVE repair DOWN dirty

Contact your next level of support if any of the file systems show STANDBY as the state. 15 16 17 Use the following steps to determine your next step. If the SPFS-based server is hosting the CBM then do step 21. List the oracle groups by typing:

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

304

Preparation for disaster recovery # groups oracle

18 19 20

If the output is install data dba or nstall dba data then go to step 20. Correct the oracle groups by typing:
# usermod -g oinstall -G data,dba oracle

Restore the data using procedure Restoring the Oracle/MySQL data on an SPFS-based server. After the data restore is complete, execute step 17 of this procedure.

21 22

Reboot the server by typing:


# init 6

Restore the central security server. See "Restoring the Central Security Server" in Nortel IEMS Administration and Security ( (NN10336-611)) .

23

Ensure the node is active by typing:


# cd /opt/nortel/sspfs/ha # ./ActivateNode

The output from execution of the ActivateNode script is as follows, and the user must enter Y to the prompt:
This procedure may cause data loss. Are you sure you want to proceed? [Y/N]

24

Confirm the prompt.


Y Executing irrevocable procedure udactive.....

25

Re-image the inactive node using the active nodes image. If required, see the procedure Cloning the image of the active node to the inactive node of a Core and Billing Manager cluster.

--End--

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

Procedure steps

305

Locking a GWC card


Use this procedure to lock a single GWC card, stopping the services, applications, and platform software running on the GWC card. Use this procedure:

Prerequisites

when you are removing the card from service along with procedure Unlocking a GWC card (page 457) to reboot a GWC and force a software download as part of fault clearing activity to determine if a problem is temporary or persistent when you apply or remove a patch to the GWC software using the Network Patch Manager (NPM) and create a new GWC software image on the CS 2000 Core Manager when you are removing a GWC node from the CS 2000 GWC Manager database when you are replacing or upgrading hardware

If the card that you want to lock is active, switch call processing to its mate card in the node. The switch places the card in standby mode. If required, follow procedure "Invoking a manual protection switch (warm SWACT)" in Nortel Gateway Controller Administration and Security ( (NN10213-611)) . When the card is in standby mode, disable (busy) services on the card. Follow procedure "Disabling (Busy) GWC card services" in Nortel Gateway Controller Administration and Security ( (NN10213-611)) .

Procedure steps
At the CS 2000 GWC Manager client

Step 1

Action At the CS 2000 Management Tools window, click the Gateway Controller folder from the Device Types directory tree in the far left frame. From the Contents of: GatewayController frame, select the GWC node that contains the card you want to lock.

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

306

Preparation for disaster recovery

Select the Maintenance tab to display maintenance information about the node.

4 5

Click the Card View button for the card you want to lock. At the CS 2000 SAM21 Manager client in the card view, select the States tab, then click the Lock button to lock the card.

Observe the system response in the History window.

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

Procedure steps

307

The card is locked when you see the text "Application locked successfully" in the History display. The lock icon (circled in the following figure) should also be present on the card graphic at the left of the screen:

If necessary, repeat this procedure for the next GWC card in the node.
--End--

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

308

Preparation for disaster recovery

Locking the Call Agent


Use this procedure to Lock the Call Agent.

CAUTION
Possible service interruption
Do not lock the active Call Agent.

Procedure steps
At the CS 2000 SAM21 Manager client

Step 1

Action From the Shelf View, right click on the card and select Lock from the context menu.

Lock is also available from the States tab of the Card View window. 2 Do not confirm a lock warning. The warning is only available for the active Call Agent. Wait for the lock icon to appear on the selected card.
--End--

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

Job aid

309

Job aid
The CS 2000 SAM21 Manager client responds to an active Call Agent lock with the prompt show in Figure 20 "Call Agent lock warning" (page 309). Do not click Yes. The inactive Call Agent is located in the other CS 2000 SAM21 Manager shelf and a lock request does not provide a prompt when the Call Agent is inactive.
Figure 20 Call Agent lock warning

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

310

Preparation for disaster recovery

Installing the remote backup server


Use this procedure to install the remote backup server for Geographic Survivability. Backing up the remote server provides a standby backup system which is ready to provide service if the primary system is unavailable for an extended period of time. The remote server can assume the identity of the primary server with data and files accurate to the last synchronization. This procedure applies only to DVD installations.

Prerequisites

Collect the following information beforehand to ensure that you have the information ready for input for this procedure. Also acquire the root user ID and password.

ATTENTION
Also gather this information prior to an upgrade. System variable Hostname IP address Netmask Router Unit 0 IP address Daily backup (up to four) Default gateway IP IP of primary cluster unit 0 Format = where MM minutes (00 - 59) DNS domain IP address DNS search domain(s) Timezone SPFS serial number Timezone = Timezone (# echo $TZ) Located on label on back of SPFS DNS server(s) Remote backup server Actual value Description

DNS variables apply only when a DNS server has been configured.

Procedure steps
Installing the remote backup server on a Geographic Survivability standby server

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

Procedure steps At your workstation or the remote server console

311

Step 1

Action To log in using ssh, type:


ssh -l root <server>

press the Enter key. Go to step 5. 2 To log in using telnet, type:


telnet

and press the Enter key. Go to step 5. 3 Log in to the remote server through the console (port A) and when prompted, enter the root or emsadm user ID and password. Bring the system to the {0} OK prompt by typing:
# init 0

4 5

Enter the following command to verify that the auto-boot option is set to true.
{0} ok printenv auto-boot?

If it is set to false, enter the following command.


{0} ok setenv auto-boot?=true

Insert the SPFS0** DVD into the DVD drive on the remote backup unit. where ** refers to the release (for example, 091 for SN09.1)

Install the remote backup server for Geographic Survivability.


{0} OK boot cdrom - rbackup

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

312

Preparation for disaster recovery

Example responses
{{0} ok boot cdrom - rbackup Boot device: /pci@1e,600000/ide@d/cdrom@0,0:f File and args: -r SunOS Release 5.8 Version Generic_108528-18 64-bit Copyright 1983-2001 Sun Microsystems, Inc. All rights reserved. Hardware watchdog enabled Configuring /dev and /devices Using RPC Bootparams for network configuration information. Skipping interface bge3 Skipping interface bge2 Skipping interface bge1 Skipping interface bge0 / This CD contains software that under the terms of your license agreement with Nortel Networks is licensed for use only on Nortel Networks OAM&P servers. Type "ok" to acknowledge this restriction on your use of the software. Any other input will halt the installation of this software. Type "ok" to acknowledge this restriction on your use of the software. Any other input will halt the installation of this software.

Type OK and press Enter to acknowledge restriction on your use of the software.
Searching for configuration file(s)...

Enter the serial number located in the rear view of the SPFS and press Enter.
*** REMOTE Backup INSTALL *** Begin entering installation information Please select a server profile for this system rbackup: Remote Automatic Backup standby system Enter the server profile for this system: ( rbackup )

10

Select the rbackup server profile for the system.


Installing with server profile "Remote Automatic Backup standby system" Install can proceed with factory default values for network configuration or you may specify the network configuration now.

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

Procedure steps Use factory default network configuration: [yes] no

313

11

Enter No to not select the default settings. The N240 server must be connected to the network and must have access to the default gateway. This allows you to enter the servers settings for the installation. Enter site-specific information in response to the following prompts. (See the information entered in the table at the beginning of this procedure.)
Enter Enter Enter Enter Enter the hostname for this system. the IP address for <hostname>. the subnet mask for this network. the IP address for this networks router. the timezone for this system [US/Eastern].

12

The default timezone is US/Eastern. Enter ? for a list of supported time zones.
Will this system use DNS? [yes]

Enter yes or no. If you answer yes, you are prompted for the DNS domain name, name server IP addresses, and the search domains. You can enter several name servers and search domains. Enter a blank line to stop the entry. 13 Click OK to accept current settings. The installation of the DVD takes approximately 60 minutes. If necessary, press Enter to redisplay the login prompt on screen after the following message displays:
NOTE: NOTE: SSPFS Installation is now completed. NOTE: HostName> console login:

14 15 16 17 18 19 20

Log in to the server using the root user ID and password. (At this point, the root password has been reset to "root.") Remove the SPFS DVD from the server. If necessary, enter
eject cdrom

Enter the command line interface (CLI) tool.


cli

Enter the number next to the Configuration option in the menu. Enter the number next to the Succession Element Configuration option in the menu. Enter the number next to the PSE Application Configuration option in the menu. Enter the number next to the Configure PSE option in the menu.

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

314

Preparation for disaster recovery

21 22

Enter the primary/cluster IP address of CS 2000 Management Tools server. This is the address of the NPM server. Enter Y to confirm the IP address. Ignore the following error message if it displays.
Cant configure PSE on remote backup unit to enable NPM

23 24 25

Enter X to exit each level until you have exited from the cli tool. Start the PSE server.
# pse start

Verify that the server has started. If the server does not start, contact your next level of support.
# pse status

26 27 28

If an SPFS MNCL CD is to be installed, see the documentation included with the CD for complete installation instructions. Enter NPM on the CS 2000 Management Tools server and follow patching procedures to apply all relevant patches. To forward SNMP traps on the server to the IEMS, perform the procedure "Configuring the destination for SNMP traps" in Nortel ATM/IP Solution-level Configuration ( (NN10409-500)) .
--End--

Variable denitions
Variable <server> Value Name of the N240 server.

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

Procedure steps

315

Scheduling automatic backups on the remote server


Use this procedure to schedule automatic backups on the remote sever. Backing up the primary server provides a standby backup system which is ready to provide service if the primary system is unavailable for an extended period of time. The remote server can assume the identity of the primary server with data and files accurate to the last synchronization.

ATTENTION
This procedure is for use with Geographic Survivability only.

Procedure steps
Step 1 2 Action Determine how to log in to the Sun Netra 240 (N240) server that is installed as the remote backup server. To log in using SSH, type:
ssh -l root (server>

and press the Enter key. Go to step 4. 3 To log in using Telnet, type:
telnet <server>

and press the Enter key. 4 5 6 7 When prompted, enter the root password. Start the command line interface tool by entering:
cli

Enter the number next to the Configuration option in the menu. The system displays the Configuration menu. Enter the number next to the Remote Backup option in the menu. Response:
Remote Backup Configuration 1 - rbackup_display (Display Remote Backup Configurati on) 2-rbackup_config (Remote Backup Configuration) 3-rbackup_exec (Execute Remote Backup Now) 4 - rbackup_cancel (Cancel Running Remote Backup) X-exit

Select

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

316

Preparation for disaster recovery 2-rbackup_config (Remote Backup Configuration)

The system responds with the IP address of the primary server that is currently configured as the remote server, and the times that are currently configured for automatic backups. 9 Enter the unit 0 IP address of the primary server to be backed up.
<nnn.nnn.nnn.nnn> is alive

10 11 12

If the system prompts you to accept the ssh key Enter yes. Go to step 13. If the system does not prompt you to accept the ssh key then go to step 13. Enter the first time for a daily backup to occur. You can configure up to four times for daily backup to occur. Response:
Enter a time for a daily backup to occur (HH:MM):

Response:
Enter a second time for a daily backup to occur (HH:MM) or enter "x" to stop provisioning backup times:

13 14 15 16 17 18 19 20 21 22 23

If you want to enter another time for a remote backup to occur enter a second time for a daily backup to occur. Go to step 16. If you do not want to enter another time for a remote backup to occur enter x. Go to step 20. If you want to enter another time for a remote backup to occur, enter a third time for a daily backup to occur. Go to step 18. If you do not want to enter another time for a remote backup to occur enter x. If you want to enter another time for remote backup to occur enter a fourth time for a daily backup to occur. Go to step 20. If you do not want to enter another time for a remote backup to occur enter x. Enter a bandwidth limit in KB/s ( 0 means no limit ) [0]: To commit changers go to step 24. To exit enter quit. To re-enter setting enter anything other than ok or quit. Enter
ok === "rbackup_config"completed successfully

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

Variable definitions

317

24

Exit the Remote Backup Configuration level.


x
--End--

Variable denitions
Variable <nnn.nnn.nnn.nnn> <server> HH MM Value IP address that you entered. Name of the N240 server. HH is hours. Valid values are 00 to 23. MM is minutes. Valid values are 00 to 59.

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

318

Preparation for disaster recovery

Viewing conguration information for remote server backups


Use this procedure to view configuration information for remote server backups. The system displays the IP address of the target system and the times in which automatic backups of the target system will occur.

ATTENTION
This procedure is for use with Geographic Survivability only.

Procedure steps
Step 1 2 Action Log in to the Sun Netra 240 (N240) server that is installed as the remote backup server. To log in using ssh, type:
ssh -l root <server>

and press the Enter key. Go to step 4. 3 To log in using telnet, type:
telnet <server>

and press the Enter key. 4 5 When prompted, enter the root password. Start the command line interface tool by entering:
cli

The system responds by displaying a menu. 6 7 Enter the number next to the Configuration option in the menu. The system displays the Configuration menu. Enter the number next to the Remote Backup option in the menu. Response:
Remote Backup Configuration 1-rbackup_display (Display Remote Backup Configuration) 2-rbackup_config (Remote Backup Configuration) 3-rbackup_exec (Execute Remote Backup Now) 4 - rbackup_cancel (Cancel Running Remote Backup) X-exit

Select

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

Variable definitions

319

1-rbackup_display (Display Remote Backup Configuration)

Response:
Current settings: Target system is: <nnn.nnn.nnn.nnn> Back up times are: <Time 1>...<Time n>

Exit the Remote Backup Configuration level by typing:


x
--End--

Variable denitions
Variable server <nnn.nnn.nnn.nnn> <Time 1>...<Time n> Value Name of the N240 server. IP address of the remote server. Set of times at which automated backups occur.

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

320

Preparation for disaster recovery

Performing a manual backup of the target server


Performing a manual backup of the target server. Backing up the primary server provides a standby backup system which is ready to provide service if the primary system is unavailable for an extended period of time. The remote server can assume the identity of the primary server with system configuration data and files accurate to the last synchronization.

ATTENTION
This procedure is for use with Geographic Survivability only.

Procedure steps
Step 1 2 Action Log in to the Sun Netra 240 (N240) server that is installed as the remote backup server. To log in using ssh, type:
ssh -l root <server>

and press the Enter key. Go to step 4. 3 If you want to log in using telnet, enter:
telnet <server>

and press the Enter key. 4 5 When prompted, enter the root password. Start the command line interface tool.
cli

The system responds by displaying a menu. 6 7 Enter the number next to the Configuration option in the menu. The system displays the Configuration menu. Enter the number next to the Remote Backup option in the menu.
Remote Backup Configuration 1-rbackup_display (Display Remote Backup Configuration) 2-rbackup_config (Remote Backup Configuration) 3-rbackup_exec (Execute Remote Backup Now) 4 - rbackup_cancel (Cancel Running Remote Backup) X-exit

Select
3-rbackup_exec (Execute Remote Backup Now)

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

Variable definitions

321

An automatic backup is made. Pressing 4 during execution of the backup halts the process and performs necessary clean-up operations.

10

Exit the Remote Backup Configuration level.


x
--End--

Variable denitions
Variable <server> Value Name of the N240 server.

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

322

Preparation for disaster recovery

Viewing logs from a remote backup


View logs from a remote backup created during automatic and manual backups of the remote server.

Procedure steps
Step 1 2 Action Log in to the Sun Netra 240 (N240) server that is installed as the remote backup server. To log in using SSH, enter:
ssh -l root <server>

and press the Enter key. Go to step 5. 3 To log in using Telnet, enter:
telnet <server>

and press the Enter key. Go to step 5. 4 5 To log in to the remote server console then go to step 5. Enter:
less /var/adm/messages

The system responds by displaying the contents of the log file.


--End--

Variable denitions
Variable <server> Value Name of the N240 server.

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

Procedure steps

323

Initiating a recovery back to the cluster


Use this procedure to initiate a recovery back to the cluster.

Prerequisites

If the server was previously a CBM and contains billing files not already sent to a downstream billing server, using the cluster server, copy these files to a downstream server before performing this procedure. Otherwise these files and the billing records will be lost. Contact next level of support for assistance. Both cluster units 0 and 1 must be in the firmware mode. (Enter command init 0 to perform this action) In this procedure, unit0 server of the cluster is used as a remote backup server. Use the same hostname and IP address that was used to configure the remote backup server.

WARNING
If both the remote backup unit and the high-availability (HA) pair are in service after a switch command, both units vie for dominance. This conflict results in a loss of communication to the core.

Procedure steps
Step 1 Action Follow procedure Installing the remote backup server (page 310). Use only one automated schedule and make sure to select a time that will not be invoked shortly. 2 3 Follow procedure Performing a manual backup of the target server (page 320). Bring down the machine currently active by following procedure Two-server (cluster) configuration in Shutting down an SPFS-based server in . Follow procedure Initiating a switch over to the remote backup server (page 325) to bring the services back to unit0 of the cluster. Follow procedure Cloning the image of one server in a cluster to the other server in the document . If the server was previously a CBM and contains billing files not already sent to a down-stream billing server, use the remote backup server to copy these to a downstream server before

5 6

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

324

Preparation for disaster recovery

performing Installing the remote backup server (page 310). Otherwise these files and the billing records are lost. Contact next level of support for assistance. 7 8 Reinstall the backup server following procedure Installing the remote backup server (page 310). Reconfigure the backup server following procedure Scheduling automatic backups on the remote server (page 315).
--End--

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

Procedure steps

325

Initiating a switch over to the remote backup server


Use this procedure to initiate a switch over to the remote backup server.

ATTENTION
When completed, this procedure reboots the remote backup server as the unit0 of the cluster.

Prerequisites

Prior to starting this procedure, shut down the Cluster machine. The user must be logged in as the root or emsadm user to initiate the switch command. For more information, see the section Two-server (cluster) configuration in Shutting down an SPFS-based server in document Nortel ATM/IP Solution-level Fault Management ( (NN10408-900)) , , and .

CAUTION
If configuration, provisioning, patching or other write-type operations occurred since the last remote backup, the remote backup system can be out of sync compared to the data in network elements and/or the primary OAM system. Take actions before initiating the switchover to a remote backup OAM server (that is, response to a geographic or other prolonged outage of the primary OAM system) to halt or prevent write-type operations by OSSs and operations personnel until an in-sync status is achieved. When initiating a switchover to a remote backup OAM server, do not execute configuration, provisioning, patching or other write-type operations through the remote backup OAM system until out-of-sync conditions are cleared.

Procedure steps
Step 1 Action Log in to the server by typing
> telnet <server>

and pressing the Enter key. 2 3 When prompted, enter your user ID and password. Change to the root user.
su - root

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

326

Preparation for disaster recovery

4 5 6

When prompted, enter the root password. Invoke the switch by typing:
$ /opt/sspfs/rbks/switch

When ready, indicate you want to proceed by typing:


OK
--End--

Variable denitions
Variable <server> Value IP address or host name of the SPFS-based remote backup server.

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

Procedure steps

327

Disabling Ethernet Routing Switch 8600 WAN Edge routing

ATTENTION
Perform this procedure immediately after a site failure occurs to limit effects to Call Processing.

This procedure is required to workaround a deficiency in the architecture when the active in-service and SYSB (out of service) GWC send messages to a gateway provisioned off the GWC causing mismatches in the gateway resulting in loss of call processing. This procedure will impact every element sitting off the ERS 8600. There are two options for this procedure. The first option will disable all dynamic routing on the chassis and no traffic will be able to leave or enter the VLANs configured on that chassis when using Open Shortest Path First (OSPF). Static routes and Local communication is still possible. The second option will disable the WAN Edge Interface. All traffic, whether through dynamic or static routes will not be able to leave or enter the chassis.

Procedure steps
At the Ethernet Routing Switch 8600 Workstation

Step 1 2 3

Action To use option 1, go to step 3 To use option 2, go to step 5 Disable OSPF by typing
#config ip ospf disable

and pressing the Enter key. 4 5 This procedure is complete. Identify the WAN Edge VLAN ID by typing
#show vlan info basic

and pressing the Enter key. Example response

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

328

Preparation for disaster recovery

Identify the ports used for WAN Edge connectivity by typing


#show vlan info ports <vlan_id>

and pressing the Enter key. Example response

Disable the ports used for WAN Edge connecitvity by typing


#config ethernet <port_range_1> state disable #config ethernet <port_range_2> state disable ... #config ethernet <port_range_n> state disable

and pressing the Enter key.


--End--

Variable denitions
Variable <vlan_id> <port_range_1> Value is the VLAN ID obtained in step 4 is a range of ports used for WAN Edge

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

Procedure steps

329

Disabling and re-enabling the router ports on the Ethernet Routing Switch 8600
Use this procedure when a failure occurs, such that Sites A and B are isolated from each other, to disable and re-enable the Ethernet Routing Switch 8600 router ports that are connected to the GWC in the inactive site.

Prerequisites

You must know the list of router ports connected to the GWCs.

Procedure steps
Step 1 2 3 Action At the Ethernet Routing Switch 8600 Workstation, confirm which site has the active Call Agent. Confirm that the GWCs in the active site are active. Disable the ports to the GWC units on the inactive site by typing:
#config ethernet <port_range_1> state disable #config ethernet <port_range_2> state disable ... #config ethernet <port_range_n> state disable

and pressing the Enter key. 4 If you disabled OSPF in the procedure Disabling Ethernet Routing Switch 8600 WAN Edge routing on page 205, re-enable it by typing:
#config ip ospf enable

and pressing the Enter key. 5 If you disabled the ports used for WAN Edge connectivity, re-enable those ports by typing:
#config ethernet <port_range_1> state enable #config ethernet <port_range_2> state enable ... #config ethernet <port_range_n> state enable

and pressing the Enter key. 6 Check the time by typing:

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

330

Preparation for disaster recovery #date

and pressing the Enter key. Example response


local time: THU MAR 31 17:32:57 2005 EST hardware time: THU MAR 31 22:32:57 2005 UTC

7 8

Allow at least one minute to pass to ensure that internal GWC audits set the units to inactive. Check the time by typing:
#date

and pressing the Enter key. Example response


local time: THU MAR 31 17:33:59 2005 EST hardware time: THU MAR 31 22:33:59 2005 UTC

9 10 11 12 13

If the inactive building is accessible, perform steps 10 through 12 to verify the inactive GWC units have gone inactive. Connect the terminal to the console port on the GWC. Enter PMDEBUG and perform the T, N, and A commands. Confirm that the state is inactive. if not, repeat steps 3 through 8 on those ports. Re-enable the ports to the GWC units in the inactive site by typing
#config ethernet <port_range_1> state enable #config ethernet <port_range_2> state enable ... #config ethernet <port_range_n> state enable

and pressing the Enter key.


--End--

Variable denitions
Variable <port_range> Value is a range of ports connected to the GWC (example: 3/1-3/48)

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

331

Network power down


This chapter highlights the sequence of steps necessary to power down all of the network elements in the office. Not all network elements covered in this section apply to your office. Prerequisites

The network elements should be backed up before powering down if time permits.

Network power down task flow This task flow shows you the sequence of procedures you perform to power down the network.

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

332

Network power down

Figure 21 Powering down the network procedures - Part 1

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

Variable definitions

333

Figure 22 Powering down the network procedures - Part 2

Network power down navigation:

Powering down an MG 9000 device (page 335) Powering down the IPmedia 2000 shelves (page 337) Powering down the SAM16 shelves (page 338) Powering down a Border Control Point (page 339)

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

334

Network power down

Powering down the MCS servers (page 340) Powering down the MDM workstation (page 341) Powering down an SPM device (page 342) Powering down a Media Gateway/Multiservice Switch 7400 (page 345) Powering down a Media Gateway/Multiservice Switch 15000 (page 346) Powering down a USP (page 347) Powering down the SAMF frame (page 348) Powering down the Call Control Frame (page 350) SAM21 Shelf Controller security and administration fundamentals (page 352) Powering down the Border Control Point Manager (page 355) Powering down the Border Control Point Manager (page 355) Powering down a Netra 240 server (page 356) Powering down the SDM (page 357) Powering down a Ethernet Routing Switch 8600 (page 358) Powering down the XA-Core (page 360) Shutting down the IEMS server (page 359) Performing a partial power down of the MG 9000 (page 363) Performing a partial power down of the Media Gateway/Multiservice Switch 15000 (page 364) Performing a partial power down of the USP (page 368) Performing a partial power down of the Ethernet Routing Switch 8600 (page 369) Performing a partial power down of the Netra 240 server (page 370) Performing a partial power down of the XA-Core (page 371)

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

Job aid

335

Powering down an MG 9000 device


This procedure details the sequence of steps necessary to power down Media Gateway 9000 (MG 9000).

Prerequisites

The optimal power threshold should be 43.7 volts. The minimum is 42.5 volts.

Procedure steps
Step 1 Action

ATTENTION
It is recommended to only unseat the fuses to allow for rapid restoration.

At the MG 9000 frame, remove the CE (SB) fuses from the frame. See Job aid (page 335) to identify the fuse locations. 2 3 Remove the TB fuses from the frame. Remove the CU fuses from the frame.
--End--

Job aid
The following figure identifies the fuse locations.

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

336

Network power down Figure 23 MG 9000 frame

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

Procedure steps

337

Powering down the IPmedia 2000 shelves


Power down the IPmedia 2000 shelves that house the MS 2000 series devices and the Packet Media Anchor. The IPmedia 2000 shelves reside in either a SAMF frame or CCF, depending on whether or not your solution uses the CS 2000 or CS 2000-Compact.

Procedure steps
Step 1 2 Action At the CCF or SAMF frame, open the front door of the CCF/SAMF frame. At the EBIP at the top of the frame, turn the breaker associated the Media Server to the OFF position.
--End--

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

338

Network power down

Powering down the SAM16 shelves


The SAM16 shelves reside in either a SAMF frame or CCF, depending on whether or not your solution uses the CS 2000 or CS 2000-Compact.

Procedure steps
Step 1 2 Action At the CCF or SAMF frame, open the front door of the CCF/SAMF frame. At the EBIP at the top of the frame, turn the breakers associated the SAM16 to the OFF position.
--End--

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

Procedure steps

339

Powering down a Border Control Point


Use the following procedure to power down a Border Control Point. The Border Control Point shelves are powered from either a PDU or EBIP located at the top of the frame depending on if the frame is AC or DC powered.

Procedure steps
Step 1 Action At the frame housing the Border Control Point, power down the shelf by switching the breaker on the back of the chassis to the OFF position. For AC powered shelves, push the rocker located on the top of the AC outlet to apply power to the chassis. It should be switched to the "0" position. For DC powered shelves, switch the circuit breakers at the EBIP that provide power to the shelf being powered to the OFF position. The breaker will have the 0 side (bottom) of the breaker depressed.
--End--

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

340

Network power down

Powering down the MCS servers


Use the following procedure to power down the MCS servers.

Procedure steps
Step 1 Action At the frame housing the MCS servers, power down the MCS servers by turning off the circuit breakers that provide power to the servers.
--End--

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

Procedure steps

341

Powering down the MDM workstation


Use the following procedure to power down the MDM Sun FireTM V480 workstation.

Prerequisites

It is recommended that you shut down the MDM workstation while on battery power.

Procedure steps
Step 1 Action At the workstation terminal window, enter the following commands to shut down it:
# /bin/sync # /bin/sync # /sbin/init 0
--End--

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

342

Network power down

Powering down an SPM device


Use this procedure to power down an Spectrum Peripheral Module (SPM) device, Media Gateway 4000 (MG 4000), Interworking Spectrum Peripheral Module (IW-SPM), or Dynamic Packet Trunking Spectrum Peripheral Module (DPT-SPM).

Prerequisites

MG 4000s may or may not be located at the same location as the Call Server and therefore power conservation may or may not need to be considered separately from the core location.

Procedure steps
Step 1 Action

ATTENTION
This procedure is not necessary for the IW-SPM or DPT-SPM.

At the MAP terminal, post and busy the trunks for the SPM or MG 4000 to be powered down at the TTP level.
>MAPCI;MTC;TRKS;TTP;POST D SPM <spm_no>;BSY ALL

ATTENTION
This procedure is not necessary for the IW-SPM or DPT-SPM.

Busy all the trunks in the PRADCH level.


>PRADCH;POST AD SPM <spm_no>; BSY all

3 4 5

Access the carrier level.


>QUIT ALL;MAPCI;MTC;TRKS;CARRIER

Post and busy the DS1P carriers.


POST SPM <spm_no> DS1P; BSY ALL

ATTENTION
The VT15P and DS3P carriers may not both exist.

Post and busy the VT15P carriers.


POST SPM <spm_no> VT15P; BSY ALL

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

Procedure steps

343

ATTENTION
The VT15P and DS3P carriers may not both exist.

Post and busy the DS3P carriers.


POST SPM <spm_no> DS3P; BSY ALL

7 8

Post and busy the STS1P carriers.


POST SPM <spm_no> STS1P; BSY ALL

ATTENTION
This step only applies to an MG 4000 or IW-SPM.

Post and busy the STS3cp carriers.


POST SPM <spm_no> STS3cp; BSY ALL

9 10 11 12 13 14 15 16 17

Post and busy the STS3L carriers.


POST SPM <spm_no> STS3L; BSY ALL

Post and busy the OC3s carriers.


POST SPM <spm_no> OC3s; BSY ALL

Access the PM level.


>QUIT ALL;MAPCI;MTC;PM;POST SPM <spm_no>

Select and BSY the inactive RMs.


>SELECT <rm_type> <inactive_rm_number>; BSY

Select and BSY FORCE the active RMs.


>SELECT <rm_type> <active_rm_number>; BSY FORCE

Select and BSY the inactive CEM.


>SELECT CEM <inactive_cem_number>; BSY

Select and BSY FORCE the active CEM.


>SELECT CEM <active_cem_number>;BSY FORCE

Power down the SIM cards using the switch on the front. Then pull the fuses from the PCIU. Unseat hardware as required to avoid water damage and shorts.
--End--

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

344

Network power down

Variable denitions
Variable <spm_no> <rm_type> <inactive_rm_number> <inactive_cem_number> Value is the number of the SPM/MG 4000/IW-SPM/D PT-SPM is the type of RM to select (OC3, DSP, and VSP) is the number of the inactive RM is the number of the inactive CEM

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

Procedure steps

345

Powering down a Media Gateway/Multiservice Switch 7400


Use the following procedure to power down the Media Gateway/Multiservic e Switch 7400.

Prerequisites

Ensure recent provisioning changes are committed.

Procedure steps
Step 1 Action

WARNING
If primary power to the shelf assembly is on, never set the individual power supply switches to the standby position. This can cause an overload to the on-line power supply.

ATTENTION
The front panel LED of each power supply remains lit for approximately 2 minutes after the power supply loses power.

At the chassis, turn off the circuit breakers for the outlets that supply power to your switch. 2 After a couple of minutes, verify the LEDs for the power supplies, cooling unit, and each processor card faceplate are off.
--End--

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

346

Network power down

Powering down a Media Gateway/Multiservice Switch 15000


Use the following procedure to power down the Media Gateway/Multise rvice Switch 15000.

Prerequisites

Recent provisioning changes need to be committed.

Procedure steps
Step 1 Action At the chassis, breaker interface panels (BIPs) are located at the top of the chassis. Power down all five breakers simultaneously on each BIP, starting at the left. Shut off the BIPs in the following order:

ATTENTION
This action shuts down the upper shelf.

BIP B2 BIP A2

ATTENTION
This action shuts down the lower shelf.

BIP B1 BIP A1

--End--

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

Procedure steps

347

Powering down a USP


Use the following procedure to power down a Universal Signaling Processor (USP).

Prerequisites

The USP backup data should be up-to- date. No datafill changes should be made between the final data snapshot and the powering down of the USP. All users need to be logged out of the USP GUI and CLI.

Procedure steps
Step 1 Action

CAUTION
The following steps will take the USP out of service. In a redundant network configuration, the USPs mated STP will handle traffic after the USP is powered down. End office isolation will occur if the USP is configured as a Signaling Gateway.

ATTENTION
The USP bulk-input feature and the CLI can be used to automate these tasks in the event there are a large number of links in question (see the "Bulk Data Input of System-db Tables" section of the USP CLI Interface Specification Manual)

Inhibit and deactivate all links. 2 At the USP Manager, ensure that the LEDs on slot 13 and slot 16 front panel cards are not lit, and immediately turn off the A/B Power buttons on the control shelf. Turn off the A/B Power buttons located on the backside of each extension shelf. In the event that access to the USP is not available, it will be necessary to disconnect the power source that feeds the USP.
--End--

3 4

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

348

Network power down

Powering down the SAMF frame


Use the following procedure to power down the SAMF frame. This involves locking all cards and the SAM21 shelf and powering down the Session Server unit and Policy Controller unit.

Procedure steps
Step 1 2 3 Action At the Session Server unit, perform procedure Lock the SIP gateway application on the active Session Server unit. Perform procedure Suspend the SIP gateway application on the active Session Server unit.

ATTENTION
The state of the SIP Gateway application is saved when the unit is powered off. When the unit is powered up, the SIP Gateway application initializes in the same state in which it was powered down.

Perform procedure Halt (shutdown) a Session Server unit on the inactive Session Server unit. 4 5 Perform procedure Power-Off a Session Server unit on the inactive Session Server unit.

ATTENTION
The state of the SIP Gateway application is saved when the unit is powered off. When the unit is powered up, the SIP Gateway application initializes in the same state in which it was powered down.

Perform procedure Halt (shutdown) a Session Server unit on the active Session Server unit. 6 7 8 9 10 11 Perform procedure Power-Off a Session Server unit on the active Session Server unit. At the Policy Controller Unit, perform procedure Halt (shutdown) a Policy Controller unit on the inactive Policy Controller unit. Perform procedure Power-Off a Policy Controller unit on the inactive Policy Controller unit. Perform procedure Halt (shutdown) a Policy Controller unit on the active Policy Controller unit. Perform procedure Power-Off a Policy Controller unit on the active Policy Controller unit. Lock all active GWC cards.

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

Procedure steps

349

12 13 14 15 16 17

At the CS 2000 GWC Manager, lock all inactive GWC cards. At the CS 2000 SAM21 Manager, lock any inactive cards other than the SC cards. Lock any active cards other than the SC cards. Lock the inactive SC card. Lock the active SC card. At the SAMF frame, power down the SAMF frame BIP breakers. At the top of the cabinet, turn off the breakers that supply power to the two SAM21 shelves and the Session Server (if equipped).
--End--

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

350

Network power down

Powering down the Call Control Frame


Use the following procedure to power down the Call Control Frame (CCF).

Procedure steps
Step 1 Action At the Session Server unit, perform procedure Lock the SIP gateway application on page 66 on the active Session Server unit. Perform procedure Suspend the SIP gateway application on page 68 on the active Session Server unit.

2 3

ATTENTION
The state of the SIP Gateway application is saved when the unit is powered off. When the unit is powered up, the SIP Gateway application initializes in the same state in which it was powered down.

Perform procedure Halt (shutdown) a Session Server unit on page 58 on the inactive Session Server unit. 4 5 Perform procedure Power-Off a Session Server unit on page 71 on the inactive Session Server unit.

ATTENTION
The state of the SIP Gateway application is saved when the unit is powered off. When the unit is powered up, the SIP Gateway application initializes in the same state in which it was powered down.

Perform procedure Halt (shutdown) a Session Server unit on page 58 on the active Session Server unit. 6 7 8 9 10 11 12 13 Perform procedure Power-Off a Session Server unit on page 71 on the active Session Server unit. At the CS 2000 GWC Manager, lock all inactive GWC cards. Lock all active GWC cards. At the CS 2000 SAM21 Manager, lock the inactive USP-lite card. Lock the active USP-lite card. Lock the inactive Compact Call Agent (CCA) card. Lock the active CCA card. At the front panel of the STORM SAM-XTS units, if your CCF is equipped with SAM-XTS units, disconnect the power to each

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

Job aid

351

unit using the main power switch located on the front panel. For figure, see Job aid (page 351). 14 At the CCF, power down the CCF BIP breakers. At the top of the cabinet, turn off the breakers that supply power to the Session Server, the two SAM21 shelves, and the Storm SAM-XTS units or Storm Disk Array.
--End--

Job aid
The following figure helps you to power down the Call Control Frame.

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

352

Network power down

SAM21 Shelf Controller security and administration fundamentals


This section contains conceptual information related to the SAM21 Shelf Controller (SC) administration and security.

Navigation
Security and administration strategy (page 352) Tools and utilities (page 352) User administration (page 352) Device administration (page 353)

Security and administration strategy


Security for the SAM21 SCs is implemented according to the items in the following list:

physical security of the client workstation physical security of the SAM21 shelf security mechanisms available through the client workstation operating system SAM21 SCs only allow login connections from the CS 2000 Core Manager, the host that provides the CS 2000 Management Tools, and the mate SAM21 SC.

Tools and utilities


There are no tools or utilities for security. The CS 2000 SAM21 Manager client is the interface for administration of the SAM21 SCs. If a Card View window is opened and a task or maintenance is completed, close the window rather than minimize the window. Memory consumption is kept to a minimum, but several unused and open Card View windows can consume memory on the CS 2000 SAM21 Manager client workstation.

User administration
The SAM21 SCs do not implement any custom security features. The CS 2000 SAM21 Manager client authenticates users when the application is started.

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

Device administration

353

Adding a user
The SAM21 SCs do not provide user administration. The CS 2000 SAM21 Manager provides authentication, but not user administration. For procedures on how to add a user to the CS 2000 SAM21 Manager client workstation, contact your system administrator or refer to the operating system reference documentation.

Setting privileges
The SAM21 SCs and the CS 2000 SAM21 Manager client do not implement user privilege administration.

Removing a user
To prevent user access to the CS 2000 SAM21 Manager client, contact your system administrator to disable the user account on the workstation.

Device administration
Administration of the SAM21 CSs is done through the CS 2000 SAM21 Manager client. This section lists the administration tasks that you can perform.

Unlocking a SAM21 Shelf Controller


Unlock a SAM21 SC from the Shelf View. Right-click on the SAM21 Shelf Controller and from the menu, select Unlock.

Locking a SAM21 Shelf Controller


Lock a SAM21 SC from the Shelf View. Right-click on the SAM21 Shelf Controller and from the menu, select Lock.

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

354

Network power down

Only lock a circuit pack that is in service. If a SAM21 SC is unlocked and then re-locked before the SAM21 SC boots, flash memory can be corrupted and require a replacement SAM21 SC. Check that a SAM21 SC is Unlocked and Enabled from the Status tab of the Card View window.

Invoking manual protection switch


Complete this task from the Shelf View. Right-click on the SAM21 Shelf Controller and from the menu, select Swact. Swact is available on the active SAM21 SC only.

Replacing air lters


Replace the three air filters from the front of the fan sleds on the SAM21 shelf with part number A0828397. This part number includes three air filters. Replace these air filters every 10 000 hours. When performing the replacement, replace one air filter at a time and do not leave a fan not inserted for more than one minute.

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

Procedure steps

355

Powering down the Border Control Point Manager


Use the following procedure to power down the Border Control Point Manager. The Border Control Point Manager is installed on a Sun Netra 240 (N240).

Prerequisites

You must have the root user password for N240. You must have the IP addresses for N240.

Procedure steps
Step 1 2 Action At the N240 server, Telnet or SSH into the server and logon as the root user. Enter
#init 0

to shutdown the server and to close the Telnet or SSH connection. 3 At this point, the server can be turned off at the circuit breakers.
--End--

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

356

Network power down

Powering down a Netra 240 server


Use the following procedure to power down a Netra 240 server.

Prerequisites

You must have the root user password for the N240 server. If you are shutting down the N240s, you need IP addresses for both the active and inactive units.

Procedure steps
Step 1 2 Action At the N240 server, Telnet or SSH into the inactive server and logon as the root user. Type
#init 0

The server shuts down gracefully and the Telnet or SSH connection is closed. 3 4 Telnet or SSH into the active server and log in as the root user. Type
#init 0

The server shuts down gracefully and the Telnet or SSH connection is closed. 5 Turn off both servers at the circuit breakers.
--End--

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

Procedure steps

357

Powering down the SDM


Use the following procedure to power down the SuperNode Data Manager (SDM) that houses the CS 2000 Core Manager.

Procedure steps
Step 1 2 Action At the VT100 terminal, login as the root user. Shutdown the SDM by typing
# Shutdown -f

. 3 Wait for the system to display the following:


Halt Completed

At the SDM frame, turn off the two breakers at the top of the SDM frame.
--End--

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

358

Network power down

Powering down a Ethernet Routing Switch 8600


Use the following procedure to power down the Ethernet Routing Switch 8600.

Procedure steps
Step 1 Action At the Ethernet Routing Switch 8600 frame, If you have an 8010co Chassis unit and an optional breaker interface panel (BIP) installed, set the BIP circuit breakers to the off position. See Installing the Breaker Interface Panel for the 8010co Chassis, 312755-E. 2 3 Turn the power supply switch to the off position. See Job aid (page 358) for figure. Disable the incoming power from the DC inpt power source. You may need to switch a circuit breaker or turn off the DC input power source.
--End--

Job aid
Figure 24 Turning the power supply switch to the off position

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

Procedure steps

359

Shutting down the IEMS server


The IEMS server shutdown process shuts down all the subprocesses and releases all the system resources. You must perform this procedure to correctly shutdown all the processes to ensure that the system does not leave any operation incomplete, and that the database information is in a consistent state.

Prerequisites

The shutdown process checks for the authenticity and the permissions of the user who invokes operation. You must be authorized to perform this procedure.

Procedure steps
Step 1 2 3 4 5 6 Action Stop all the schedulers. Notify the registered shutdown observers. Shut down all the subprocesses (sub-modules), which execute specific tasks. Disconnect all database connections. If it was started by IEMS, shut down the web server . Exit (the main process). To shut down the IEMS server, kill the related Java application shell, press Ctrl+C. The shutdown process checks for the authenticity and the permissions of the user invoking the shutdown operation, and is allowed only if the user has the proper permissions.
Shutting down the IEMS server from the command line

To shutdown the server using the servman command, start the command line interface and type /opt/servman/bin/servstop IEMS System response Stopping group using servstop The I-EMS Server on host "192.168.4.176" was successfully shutdown IEMS Stopped
--End--

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

360

Network power down

Powering down the XA-Core


Use this procedure to power down a DMS switch as follows:

in the event of an emergency, for example, flooding or fire to protect equipment if the available functioning voltage at the power distribution center (PDC) falls below -43.75 V dc when instructed by the next level of support

Procedure steps
CAUTION
This procedure results in a complete loss of subscriber service. Nortel recommends that before you perform this procedure, you should contact emergency Technical Assistance Services (ETAS) of Nortel or contact your next level of support.

Step 1

Action During emergency shutdown of the switch, at your current location, notify network management personnel of the impending service interruption. Notify emergency services (police, fire, ambulance) of the impending service interruption. If the switch must be shut down immediately due to dangerous environmental conditions then go to step 5. If there is time (one half hour or more) to shut down the switch in an orderly fashion then go to step 7.

2 3 4 5

ATTENTION
Turning off the switch in this manner should be done only if absolutely necessary, as current arcing may occur.

Turn off the power to the power distribution centers (PDCs) by disconnecting the power feeds at the power room. 6 7 You have completed the procedure. Using office records, identify the peripheral modules that host emergency services (such as fire, police, and ambulance), so that these peripherals can be shut off last.

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

Procedure steps

361

ATTENTION
Older peripheral modules, such as line modules and digital carrier modules, use universal tone receivers resident in maintenance trunk modules.

Turn off the power converter for each maintenance trunk module shelf, except those identified in step 7 as essential for emergency service lines. 9 10 Turn off the inverters for all MAPs and printers, except the operator MAP and one printer. Turn off the power converters on all digital trunk controllers and trunk modules, except those identified in step 7 as essential for emergency service communications. Turn off the power converters on each line module shelf, except those identified in step 7 as hosting emergency service lines. Turn off the power converters on all line concentrating modules, except those identified in step 7 as hosting emergency service lines. Turn off the power converters on all line group controllers and line trunk controllers, except those identified in step 7 as essential for emergency service communications. Turn off the power on all remaining peripheral modules, leaving essential service peripheral modules until last.

11 12

13

14 15

ATTENTION
To power down the ENET and the LPP, unseat and then reseat the power converters.

Turn off the power for the network modules, link peripheral processor, and input/output controllers (IOCs). 16 Turn off the power for all remaining devices, including the inverter that supplies the operator MAP, and any external printers, tape drives, or disk drives. Turn off the power for the XA-Core. Power down both SIM cards in the XA-Core. In each SIM card you must turn off three breakers. Turn off the power for the message switch. Power down one plane by turning off the power converters. Power down the second plane by unseating and then reseating the power converters.

17

18

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

362

Network power down

19

Turn off the power to the power distribution centers (PDCs) by discontinuing the A and B feeds at the power room.
--End--

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

Procedure steps

363

Performing a partial power down of the MG 9000


This procedure is used to perform a partial power down of an MG 9000.

Procedure steps
Step 1 2 3 4 Action At the MG 9000 Manager, determine the inactive Supercore, ITP, and ITX cards. Lock each inactive card using the MG 9000 Manager. Take the cards offline using the MG 9000 Manager. After the cards are offline, they can be manually unseated.
--End--

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

364

Network power down

Performing a partial power down of the Media Gateway/Multiservice Switch 15000


Use this procedure to perform a partial power down of a Media Gateway/Multiservice Switch 15000 and to move the services from a select set of cards to another set of cards. Then it powers down the select set of cards as well as any cards that host the standby ports. The following slots on each of the two shelves will remain powered up:

0 3 4 7 8 11 12 15

The high level steps of this procedure include the following activities: ensuring that the LAPS ports are active on the powered-up cards and inactive on powered-down cards ensuring the active PBG port is on a powered-up card ensuring the PBG port that connects to the SAM21 shelf controller is active ensuring the correct DS3 card remains active switching off selected power breakers

Procedure steps
Step 1 Action Ensure that the active lines are on the cards to remain powered up by typing
> d laps/* nearEndRxActiveLine, workingLine, protectionLine

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

Procedure steps

365

If there are any LAPS ports on a card that will be powered down, a protection switch needs to be performed. To switch from the protection line to the working line, type
> switch -protectionworking laps/x

To switch from the working line to the protection line, typ


> switch -workingtoprotectioon laps/y

One of the HIOPs on the XA-Core may be powered down for power conservation reasons. Determine which PBG port the HIOP corresponds to by typing
> d pbg/* working line

For example, see Job aid (page 366). 4 5 6 Determine which PBGs are connected to the XA-Core by typing
>d atmif/* remoteatmif

Match the PBG port number with the AtmIf number. Display the Sonet ports to determine whether the remote-end HIOP cards are still in service by typing
>d lp/10 sonet/3 operational

followed by typing
>d lp/11 sonet/0 operational

Determine the states of the losAlarm and rxAisAlarm. If the losAlarm is On, then the corresponding HIOP card is likely powered down. If the rxAisAlarm is On, the HIOP card is probably locked.

Verify that this HIOP card is selected to be powered down for power conservation. The other Sonet port should have no alarms raised under the operational group. Select AtmIFs with matching instance numbers. For example, if pbg/1100 and 1003 are provisioned, type
>d atmif/1100 vcc/* vcd tm atmService Category >d atmif/1003 vcc/* vcd tm atmService Category

10

Determine which PBG port is active by displaying the corresponding AtmIf vcc traffic counts. Type
>d atmif/X vcc/Y txcell, rxcell

11

The active PBG port will have incrementing txcell and rxcell counts. Ensure that the active PGB ports workingLine is on one of the cards that will remain in-service (0, 3, 4, 7, 8, 11, 12, or 15).

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

366

Network power down

12

If the PVG ports workingLine is not on one of the cards that will remain in-service, perform a Swact of the SAM21 Shelf Controller. Verify that the other PBG port to this SAM21 shelf now has incrementing txcell and rxcell counts. For a 4pDS3Ch or 12pDS3 card pair in slots 14 and 15, type
>d -p lp/14 activeCard, mainCard, spareCard Lp/14

13 14 15

Ensure that the active card is in the set of cards that will remain in-service. If not, switch Lp activity by typing
>switch lp/14

. 16 Ensure that the activeCard from step 14 matches the MG 9000 equipment that will remain in-service during the power conservation activity. If not, the MG 9000 equipment that is powered down will need to be switched. Manually switch off the following breakers on the upper shelf:

17


18

B2.2 B2.5 A2.2 A2.5

Manually switch off the following breakers on the lower shelf:

B1.2 B1.5 A1.2 A1.5

--End--

Variable denitions
Variable activeCard mainCard spareCard Value Shelf Card/14 Shelf Card/14 Shelf Card/15

Job aid
Example response
Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

Job aid Pbg/* +=====+--------------------+--------------------------------------| Pbg | working |Response +=====+--------------------+--------------------------------------| 1003|Lp/10 Sonet/3 | | 1100|Lp/11 Sonet/0 |

367

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

368

Network power down

Performing a partial power down of the USP


Use this procedure to perform a partial power down of a USP.

Procedure steps
Step 1 Action At the USP frame, unseat the front panel and back panel CC cards in Slot 18, beginning with the CC Slot 18 card on each of the extension shelves. Unseat the front panel and back panel CC card in Slot 18 on the Control shelf. Open the RTC system node provisioning and maintenance window. Click on the System Mgmt item on the main menu to open the System Configuration window. Click on the icon for the control CAM shelf to open the shelf_name window. Click on the icon for an RTC system node. An operational-state of enabled indicates that the card is in-service. An activity state of inactive indicates that the associated card is the inactive RTC. 7 8 At the USP frame, ensure the LEDs on the slot 13 and 16 front panel cards are not lit. Unseat the inactive RTC card identified in step 6, its associated SCSI (front panel), and the associated cards on the back panel.
--End--

2 3 4 5 6

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

Procedure steps

369

Performing a partial power down of the Ethernet Routing Switch 8600


Use this procedure to perform a partial power down of an Ethernet Routing Switch 8600.

Procedure steps
Step 1 Action At the chassis, each chassis has its own BIP circuit breaker. Set the BIP circuit breaker to the off position to work in simplex mode. Turn one of the three DC power supplies off. This will leave 2 DC power supplies to support the remaining chassis.
--End--

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

370

Network power down

Performing a partial power down of the Netra 240 server


Use this procedure to perform a partial power down of the N240 servers. The N240 servers are commissioned with one active and one inactive unit. This procedure shuts down the inactive unit.

Prerequisites

You must have the root user password for the N240. You must have the IP address for the inactive unit.

Procedure steps
Step 1 2 Action At the N240 server, Telnet or SSH into the inactive server and logon as the root user. Type
#init 0

to shutdown the server and to close the Telnet or SSH connection. 3 At this point, the inactive server can be turned off at the circuit breaker.
--End--

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

Prerequisites

371

Performing a partial power down of the XA-Core


Use this procedure to maintain emergency backup power for a DMS SuperNode or a DMS SuperNode SE switch. Use this procedure when the switch has an eXtended Architecture Core (XA-Core) configuration. This procedure reduces the loss from emergency batteries to a minimum by closure of power to equipment in stages. The equipment shutdown is not necessary to maintain subscriber service. Equipment shutdown is in moving up order based on its effect on switch reliability. The equipment shutdown begins with less required equipment, such as spare printers, and ends with more required equipment.

Prerequisites

Take into consideration the configuration and condition of your switch. Also take into consideration the expected period of the power outage, and the quantity of reserve power available. Continue as follows:

ATTENTION
If you receive a warning message for a loss of service if you busy a plane or unit, do not continue. Clear the problem that can cause a loss of service before you busy the plane or unit. Also you can leave both planes or units of that subsystem in service.

Complete the number of steps of this procedure as your set of conditions needs. For example, if you expect power to restore, you can decide to leave important systems to operate in a duplex mode. The message switch (MS) is an example of a system that can have a duplex mode. Equally for reliability, you can decide to leave both units in service on peripheral modules required for emergency service lines. This procedure requires a condition when this procedure instructs you to busy down one plane or unit of a system. The condition is that the mate plane or unit you leave in service, is fault free and can operate normally. When possible, take the same plane or unit out of service on each subsystem (for example, ENET plane 0, LIM unit 0, MS 0). This action decreases the possible result of error and reduces recovery time.

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

372

Network power down

Procedure steps
CAUTION
Potential service interruption or extended outage
Nortel recommends that you call Emergency Technical Assistance Services (ETAS) of Nortel. Also call your next level of support before you perform this procedure.

CAUTION
Potential loss of service or extended outage
This procedure is only for conservation of emergency backup power. Do not use this procedure or sections for equipment maintenance purposes.

Step 1

Action At your current location, use office records to identify and record the power converters that supply the MAPs and printers for the switch. Turn the power off on all power converters identified in step 1. Do not turn the power off for the power converters which supplies the operators MAP and one printer connected to IOC 0. At the MAP terminal, to confirm that an office image is available to reload the switch if a total turn off of power becomes necessary, type: >AUTODUMP STATUS . For example, see Job aid (page 374).

4 5 6

If an office image is available then go to step 7. If an office image is not available then go to step 6. Record the office image. Perform the procedure, "How to record an XA-Core office image on a disk" in this document. Return to this step when you complete the procedure to record the office image.

ATTENTION
Cards that have an effect on service include digitone receiver cards (NT2X48), centralized automatic message accounting (CAMA) cards (NT2X66), and digital recorded announcement machine (DRAM) cards.

Turn off power for all maintenance trunk modules (MTM) in the office. Do not turn power off if the MTM contains cards that have an effect on service. To turn off power perform the procedure,

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

Procedure steps

373

"Emergency shutdown of maintenance trunk modules" in Recovery Procedures () , 297-8021-545. 8 Turn off power for one of the line module controllers (LMC) (NT2X14 shelf) in each double-bay line module (LM) pair in the office. To turn off the power, perform the procedure, "Emergency shutdown of one half of a line module pair" in Recovery Procedures () , 297-8021-545. Turn off power for one unit of all line concentrating modules (LCM) in the office. To turn off the power, perform the procedure, "Emergency shutdown of one unit of LCMs" in Recovery Procedures () , 297-8021-545. Turn off power for one unit of all line group controllers (LGC), line trunk controllers (LTC), and digital trunk controllers (DTC) in the office. To turn off the power, perform the procedure, "Emergency shutdown of one LGC, LTC, and DTC unit" in Recovery Procedures () , 297-8021-545. Turn off power for one plane of all network shelves in the office. For ENET, perform the procedure, "Emergency shutdown of one enhanced network plane" in Recovery Procedures () , 297-8021-545. For JNET, perform the procedure, "Emergency shutdown of one junctored network plane" in Recovery Procedures () , 297-8021-545. If you removed power from a complete network frame in steps 12 or 13, turn off power for the cooling fans for the frame. To turn off power, remove the correct power fuses from the PDC. Busy and power down one local message switch in the link peripheral processor (LPP). Perform the procedure, "Emergency shutdown of one LIM unit on each LPP" in Recovery Procedures () , 297-8021-545. Turn off power for one unit of all CCS7 message switch and buffers (MSB7) in the office. To turn off power, perform the procedure, "Emergency shutdown of one unit of MSB7s" in Recovery Procedures () , 297-8021-545. Turn off power for one message switch (MS) plane. To turn off power, perform the procedure, "Emergency shutdown of one DMS SuperNode MS plane" in Recovery Procedures () , 297-8021-545. If your office has a remote oscillator shelf (NT3X9507), busy the clock related to the MS number. The MS number is for the MS that you turned off power in step 17. To busy the clock, perform

10

11 12

13

14

15

16

17

18

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

374

Network power down

the procedure, "Emergency shutdown of one remote ascillator shelf plane" in Recovery Procedures () , 297-8021-545.
--End--

Job aid
Example of a MAP response:
Successful Image: 990215_CM Taken: 1999/03/17 21:47:32:04.138 WED. On Volume: F17LIMAGE Successful Image: 990215_MS Taken: 1999/03/17 21:47:32:04:138 WED. On Volume: F17LIMAGE SCHEDULED-Image Dump is ON. Next scheduled dump is MONDAY at 22:30 hours. Next image to be dumped on F02LIMAGE.

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

375

Network power up
This chapter describes restarting the network for the Nortel Carrier VoIP solutions. Network power up task flow This task flow shows you the sequence of procedures you perform to power up the network.

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

376

Network power up

Figure 25 Powering up the network procedures - Part 1

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

Job aid

377

Figure 26 Powering up the network procedures - Part 2

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

378

Network power up

Figure 27 Powering up the network procedures - Part 3

Network power up procedure navigation

Powering up the XA-Core (page 380) Powering up the Message Switch (page 396) Powering up the Enhanced Network (page 399) Powering up an Ethernet Routing Switch 8600 (page 401) Powering up the Link Peripheral Processors (page 402) Powering up the SDM (page 405) Powering up a Netra 240 server (page 406) Powering up the Border Control Point Manager (page 407) Powering up the SAMF frame (page 408) Powering up the Call Control Frame (page 410) Powering up a USP (page 412) Powering up a Media Gateway/Multiservice Switch 15000 (page 414)

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

Job aid

379

Powering up a Media Gateway/Multiservice Switch 7400 (page 415) Powering up the MDM workstation (page 417) Powering up an SPM device (page 418) Powering up the MCS servers (page 420) Powering up a Border Control Point (page 424) Powering up the IPmedia 2000 shelves (page 425) Powering up the SAM16 shelves (page 426) Powering up an MG 9000 device (page 427) Restoring certificates and Pre-shared Keys for IPSec (page 432) Restoring the MG 9000 manager system (page 436) Restoring files from a DVD-RW (page 437) Reloading or restarting a SAM21 Shelf Controller (page 440) Restarting or rebooting a GWC card (page 441) Performing a partial power up of the Netra 240 server (page 443)

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

380

Network power up

Powering up the XA-Core


Use this procedure to power up a dead switch that has an eXtended Architecture Core (XA-Core). The switch can be a DMS SuperNode or SuperNode SE switch. The switch is dead if the complete switch is without power. The power loss results from a loss or interruption of A and B dc power feeds to the power distribution center (PDC).

Procedure steps
CAUTION
Call ETAS or your next level of support
In the event of a dead system, call the Emergency Technical Assistance Services (ETAS) of Nortel. Also call your next level of support before you perform this procedure.

DANGER
Risk of electrocution
Do not touch the cabinet wiring. Connections with unshielded cabinet wiring can result in electric shock. Only qualified power maintenance personnel can perform the voltage measurements in step 3.

Step 1

Action At the PDC, when possible after detection of the power outage, remove all the fuse holders for the line concentrating equipment (LCE) talk batteries, trunk module equipment (TME) talk batteries and PDC filter fuses from the correct PDCs

ATTENTION
The location of the fuse holders in the fuse panel can vary, depending on your office configuration. For help in locating the fuse holders, see the fuse assignment diagram for your office. Also call your next level of support to help you locate the fuses.

2 3 4

If the switch power is -48 V dc then go to step 4. If the switch power is -60 V dc then go to step 8. Continue when you know of restored power at the power plant for your office. Power maintenance personnel must check for restored power at each PDC. At the rear of each PDC, measure the dc voltage across the A feed bus and the battery return plate. Repeat the dc voltage measurement for the B feed bus. Power is correct when the voltage on each feed is -48 V dc.

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

Procedure steps

381

ATTENTION
Power can be at a nominal potential of -48 V dc. Under conditions that are not normal, the operating voltage can range from -43.75 V dc to -55.8 V dc. A not normal condition is a commercial power failure

5 6 7

If the switch has power retrieval then go to step 11. If the switch has no power retrieval then go to step 10. Continue when you know of restored power at the power plant for your office. Request power maintenance personnel to check for restored power at each PDC. At the rear of each PDC, measure the dc voltage across the A feed bus and the battery return plate. Repeat the dc voltage measurement for the B feed bus. Power is correct when the voltage on each feed is -60 V dc.

ATTENTION
Power can be at a nominal potential of -60 V dc. Under conditions that are not normal, the operating voltage can range from -57.4 V dc to -67.7 V dc. A not normal condition is a commercial power failure.

8 9 10

If power has power retrieval then go to step 11. If power has no power retrieval then go to step 10. At the power room, for help in restoring power to the PDC, call the personnel responsible for maintenance of power at your site. When power restores to the PDC, return to this point. At the PDC, inspect the alarm indication fuses for the XA-Core and the network cabinets. If there are blown fuses, go to step 14. If there are no blown fuses, go to step 22. Replace the blown cartridge fuse in the rear of the fuse holder. Make sure that the amperage of the replacement cartridge fuse matches the amperage marked on the PDC. Remove the blown alarm-indication fuse from the front of the fuse holder. Re-insert the fuse holder, with the alarm-indication fuse removed, into the PDC. Get an alarm-indication fuse for replacement. Insert the alarm-indication fuse that is for replacement into the fuse holder. If the replacement fuse is not successful and blows repeatedly, go to step 21. If the replacement fuse is successful, go to step 22. Call your next level of support for help.

11 12 13 14

15 16 17 18 19 20 21

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

382

Network power up

Continue when you complete replacement of all blown fuses and restored power to the XA-Core and network cabinets. 22 If a second person is available to help in the recovery, continue this procedure with two sets of tasks. Request the second person to restore power from the pdc to the peripheral module frames. The second person restores power by use of step 82 through step 97 of this procedure. While the second person restores power, you recover the core and network by completion of step 23 through step 65. If one person is available, recover the core and network first. If the switch has a remote oscillator shelf, go to step 25. If the switch does not have a remote oscillator shelf, go to step 26. At the remote oscillator shelf, turn on the power converters for the shelf. At the XA-Core cabinet, determine if the switch is a SuperNode switch or a SuperNode SE switch. If the switch is a SuperNode switch then go to step 29. If the switch is a SuperNode SE switch then go to step 32. Turn on circuit breakers for shelf interface module (SIM) A. Turn on the three switches for circuit breakers A1, A2, and A3. The SIM A card is in slot 3R of the XA-Core shelf. Turn on circuit breakers for shelf interface module (SIM) B. Turn on the three switches for circuit breakers B1, B2, and B3. The SIM B card is in slot 16R of the XA-Core shelf. Go to step 33. Turn on circuit breakers for shelf interface module (SIM) A. Turn on the two switches for circuit breakers A1 and A2. The SIM A card is in slot 3R of the XA-Core shelf. Turn on circuit breakers for shelf interface module (SIM) B. Turn on the two switches for circuit breakers B1 and B2. The SIM B card is in slot 16R of the XA-Core shelf. If all the power converters do not have power, go to step 35. If all the power converters have power, go to step 37. To power up the frame perform the procedure, "Clearing an Ext FSP DPCC cabinet major alarm" in Alarm Clearing and Performance Monitoring Procedures, 297-8001-543 (North American market) or 297-9051-543 (International market). When you completed the procedure, return to this point. 36 37 Go to step 27. At the XA-Core reset terminal, monitor the XA-Core reset terminal to determine if the switch has booted.

23 24 25 26 27 28 29

30

31

32

33 34 35

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

Procedure steps

383

When the switch boots, the XA-Core reset terminal displays a response to indicate a boot in progress. The response also displays different diagnostic messages and alphanumeric addresses. When the switch has completely booted, an A1 appears on the RTIF display. 38 39 40 If the response has an A1 then go to step 40. If the response has no A1 after approximately 15 min then go to step 180. At the MAP terminal, see the Caution statement.

CAUTION
Extended service interruption
The exact log on procedure can vary, depending on your office configuration. If you need additional help, call the personnel responsible for the next level of support.

Determine if you have to log on.

ATTENTION
The log on message indicates that you have to manually log on. An automatic log on can occur if the office parameters have automatic log on.

Example of a MAP response


Please Login.

41 42 43

If the log on is not automatic, then go to step 43. If the log on is automatic, then go to step 47. Press the break key. Example of a MAP response
?

44

Log on to the MAP terminal.


>LOGIN

Example of a MAP response


Enter User Name

45

Enter the user name.


>user_name

Example of a MAP response


Enter Password

46

Enter the password.

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

384

Network power up >password

Example of a MAP response


SuperNode1 Logged in on 1997/01/15 at 20:37:17

CAUTION
All customers must follow the sequence of steps set out in this procedure
Do not interrupt this procedure at this point to clear an alarm. If a TOD critical alarm appears under the APPL level in the alarm banner, and if the system uses Network Time Protocol (NTP), you must complete all steps in the sequence shown. You will clear the TOD critical alarm by completing step 58. (For information on NTP, see step 58.)

47

To turn on priority, type:


>PRIORITY ON

and press the Enter key. Example of a MAP response


>Pref

48

Determine if the system time is correct.


>TIME

Example of a MAP response


Time is 14:55:50

49 50 51

If the system time is not correct then go to step 51. If the system time is correct, then go to step 53. Enter the correct time (by use of the 24 hour clock).
>SETTIME hh mm

Example of a MAP response


Warning: There is an automated TOD clock change request scheduled on: 1997/10/15 at 1:00 (see table DSTTABLE). Do you want to proceed with this request? Please confirm ("YES", "Y", "NO", or "N")

52

Confirm the command.


>YES

Example of a MAP response


Time is 20:40:00 on WED 1997/10/15.

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

Procedure steps

385

53 54 55 56

Determine if the system date is correct. If the system date is not correct, then go to step 56. If the system date is correct, then go to step 58. Enter the correct date.
>SETDATE dd mm yyyy

Example of a MAP response


Warning: There is an automated TOD clock changerequest scheduled on: 1997/10/15 at 1:00 (see table DSTTABLE). Do you want to proceed with this request? Please confirm ("YES", "Y", "NO", or "N")

57

Confirm the command.


>YES

Example of a MAP response


Date is WED. 15/OCT/1997 00:00:00

58

If the system uses Network Time Protocol (NTP) as the timing reference, and if a TOD critical alarm appears under the APPL level in the alarm banner, perform the procedure titled "How to check and adjust the XA-Core TOD" in the Nortel XA-Core Fault and Performance Management ( (297-8991-510)) . By performing that procedure, you will clear the TOD critical alarm. Return to this point when finished.

ATTENTION
In the German market only, switches can use Network Time Protocol (NTP) as the timing reference for the time-of-day clock. The system uses Network Time Protocol if the value of the SNTP_CLIENT office parameter in table OFCENG has been set to Y. For information about the office parameter, see the section titled "XA-Core data schema overview" in the Nortel XA-Core Reference Manual ( (297-8991-810)) .

59

At the MAP terminal, access the NET level of the MAP display.
>NET

Example of a MAP response


NET 11111 11111 22222 22222 33 Plane 01234 56789 01234 56789 01234 56789 01 0 0000 1 0000 JNET
Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

386

Network power up

60 61 62 63

Manually busy the network module for return to service.


>BSY plane_no pair_no

Return the network module to service.


>RTS plane_no pair_no

Repeat step 60 and step 61 for each JNET shelf. When all JNET shelves recover, continue this procedure. Determine if there are additional input output controller (IOC) and maintenance and administration position (MAP) terminals to recover. If recovery of additional IOCs and MAP terminals is not complete, then go to step 66. If recovery of additional IOCs and MAP terminals is complete, then go to step 79. Restore power to all remaining power inverters in the office. At the IOC, locate the IOC for recovery. If recovery is for an IOC, then go to step 70. If recovery is for an IOM, then go to step 74. Turn on the power converters on the IOC.

64 65 66 67 68 69 70

ATTENTION
The version of IOC determines if the IOC has one or two power converters.

71 72 73 74 75

While you press the reset button on one of the IOC power converters, lift the related circuit breaker to turn on the FSP. Release the reset button. Repeat step 67 through step 72 for each IOC for recovery. Access the input output device (IOD) level of the MAP display.
>IOD

Access the IOC level of the MAP display for the IOC for recovery.
>BSY ioc_no

76 77 78

Return the IOC or IOM to service.


>RTS ioc_no

Repeat step 75 and step 76 for each IOC or IOM for recovery. Log on to additional MAP terminals as required.

ATTENTION
Step 44 through step 46 describe how to log on to the MAP terminal.

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

Procedure steps

387

79

At the PDC, step 80 through step 104 describe how to restore power from the PDC to Series I and Series II peripheral module frames. If the PDC power to the PM frames does not restore correctly, then go to step 82. If the PDC power to the PM frames restores correctly, then go to step 105. Get a 100-W, 120-V light bulb installed into a socket that has pigtail leads or tool number T000655 (CPC number NTA0600512), that has a fuse holder-style connector instead of pigtail leads for easier insertion

80 81 82

83

WARNING
Possible equipment damage or extended service interruption
Use correct fuses. When you return fuses to the PDC, make sure that the amperage of the fuses is correct. The fuse amperage must match the amperage marked on the PDC.

At the first empty fuse slot in the PDC, connect the leads of the capacitor charging tool. Connect the leads across the contacts for the fuse holder until the lamp decreases brightness. If you use a charging tool with a fuse holder-style connector, insert the connector into the slot. Insert the connector until the lamp decreases brightness. 84 85 86 Remove the capacitor charging tool and immediately insert again the correct fuse holder into the slot. Repeat steps 83 and step 84 for the LCE talk battery, TME talk battery, and PDC filter fuse holders you removed. Determine if any alarm-indicating fuses blew.

ATTENTION
The fuse alarm-indicator lamp lights when an alarm-indicating fuse blows.

87 88 89

If no alarm-indicating fuses have blown, then go to step 89. If any alarm-indicating fuses have blown, then go to step 180. Locate a fuse holder with a blown alarm-indicating fuse.

ATTENTION
You can replace blown fuses in any order.

90

The cartridge fuse in the fuse holder has blown. Remove the fuse holder from the PDC.

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

388

Network power up

91

Replace the blown cartridge fuse in the rear of the fuse holder. Make sure that the amperage of the replacement cartridge fuse matches the amperage marked on the PDC. Remove the blown alarm-indicating fuse from the front of the fuse holder. Insert again the fuse holder, with the alarm-indicating fuse removed, into the PDC. Get a replacement alarm-indicating fuse. Insert the replacement alarm-indicating fuse into the fuse holder. Determine if the alarm-indicating fuse blows.

92 93 94 95 96

ATTENTION
The fuse alarm indicator lamp lights when an alarm-indicating fuse blows.

97 98 99 100 101 102 103 104 105

If the alarm-indicating fuse is not blown, then go to step 99. If the alarm-indicating fuse is blown, then go to step 89. Determine if you replaced all the blown alarm-indicating fuses. If you have replaced all the blown alarm-indicating fuses then go to step 102. If you have not replaced all the blown alarm-indicating fuses then go to step 89. Determine if the fuse alarm indicator lamp lit. If the fuse alarm indicator lamp is not lit, then go to step 105. If the fuse alarm indicator lamp is lit , then go to step 180. At the PM frames, select a peripheral module (PM) frame to power up.

ATTENTION
The PM frames can power up in any order.

106 107 108 109 110 111 112

Locate the frame supervisory panel (FSP) and the power converters on the frame. Determine if the FSP for the frame has fuses or circuit breakers. If the FSP has fuses, then go to step 110. If the FSP has circuit breakers, then go to step 124. Determine if the power converters have Power Reset buttons or Power Reset switches. If the power converters have power Reset buttons, then go to step 113. If the power converters have power Reset switches, then go to step 117.

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

Procedure steps

389

113

To turn on each power converter press and hold its Power Reset button for 2 s.

ATTENTION
The Converter Fail light goes off when the power converter turns on.

114 115 116 117

Determine if all the power converters turn on correctly, indicated by all the Converter Fail lights going off. If all the Converter Fail lights are off, then go to step 127. If all the Converter Fail lights are not off, then go to step 130. To turn on each power converter pull out the power switch and toggle it to the Power Reset position.

ATTENTION
The Converter Fail light goes off when the power converter turns on.

118 119 120 121 122 123

Determine if all the power converters turns on correctly, indicated by all the Converter Fail lights are off. If all the Converter Fail lights are off, then go to step 127. If all the Converter Fail lights are not off, then go to step 130. Turn on each power converter by toggle the circuit breaker to the ON position. Press and hold the Power Reset button for 2 s. Release the circuit breaker and the Power Reset button.

ATTENTION
The Converter Fail light goes off when the power converter turns on.

124 125 126 127 128 129 130 131 132 133 134 135

Determine if all the power converters turn on correctly, indicated by all the CONVERTER FAIL lights are off. If all the Converter Fail lights are off, then go to step 127. If all the Converter Fail lights are not off, then go to step 130. Determine if all PM frames turn on. If all the PM frames turn on yes, then go to step 136. If all the PM frames turn on no, then go to step 130. Determine if an attempt was made to power up the remaining PM frames. If power up has not been attempted, then go to step 133. If power up has been tried and failed, then go to step 135. Power up the next PM frame. Go to step 106. To power up the remaining PM frames perform the correct procedures in Alarm Clearing and Performance Monitoring

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

390

Network power up

Procedures () , 297-8001-543 (North American market) or 297-9051-543 (International market). 136 At the MAP terminal, access the SRSTATUS level of the MAP display.
>MAPCI;MTC;SRSTATUS

Example of a MAP response


SRSTATUS 0 Quit OVERALL STATUS Pend: 0% Inprg: 0% Comp: 100% Fail: 0% 2 View_ View: SYSTEM 14:08:30 3 List_ Pend InPrg Comp Fail Pend InPrg Comp Fail 4 MS 0 0 2 0 IOD 5 5 30 2 5 NET 0 0 6 0 Other 21 3 13 3 6 SER1 0 41 0 0 7 SER2 0 39 0 0 8 SER3 0 0 37 0 9 10 MTC: 11 STATUS:

137

From the MAP display, determine the recovery status of the Series I and II PMs.

ATTENTION
Series I PM recovery status displays to the right of the word "SER1" in the MAP display. Series II PM recovery status displays to the right of the word "SER2" in the MAP display. Recovery status can be one of pending, in progress, complete, or failed.

138 139 140 141

If the recovery status is not zero, then go to step 140. If the recovery status is zero, then go to step 142. Determine from office records or other office personnel which PMs to manually recover first. To manually recover the PMs in the required order, perform the correct alarm clearing procedures in Alarm Clearing and Performance Monitoring Procedures () , 297-8001-543 (North American market) or 297-9051-543 (International market).

142

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

Procedure steps

391

CAUTION
Loss of billing data
Different billing systems than automatic message accounting (AMA) or additional billing system, can be in your office configuration. Call your next level of support to determine if other billing systems are in your office, and if you require recovery action.

Access the device independent recording package (DIRP) level of the MAP.
>IOD;DIRP

143

Determine the state of the recording volumes for the billing system.
>QUERY subsystem ALL

Example of a MAP response


SSNAME SSNO SEQNO ROTATES POOLNO PARLPOOL EMERGENCY AMA 0 1 6 9 62 ***YES*** REGULAR FILE(S) STATE VOLUME RECCOUNT BLOCK E V V_B VLID FNUM FRN# ACTIVE NONEg STANDBY1 NONE PARALLEL FILE STATE VOLUME BLOCK E V V_B VLID NUM FRN# NONE REGULAR VOLUME(S) VOL# VOLNAME STATE IOC CARD VOL FSEG ROOM VLID FILES REGULAR SPACE

144

If the state of the recording volumes for the billing system has no volumes allocated, as indicated by the word NONE under the state header on the MAP display, then go to step 150. If the state of the recording volumes for the billing system has any volume is IN ERROR, as indicated under the REGULAR VOLUME(S) header on the MAP display, then go to step 147. If the state of the recording volumes for the billing system has all volumes are READY, as indicated under the REGULAR VOLUME(S) header on the MAP display then go to step 151.

145

146

ATTENTION
Different billing systems than automatic message accounting (AMA) or additional billing system, can be in your office configuration. Call your next level of support to determine if other billing systems are in your office, and if you require recovery action.

147

Reset any volumes that are IN ERROR.

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

392

Network power up >RSETVOL vol_name

148 149 150

If the volumes reset passed, then go to step 151. If the volumes reset failed, then go to step 180. Perform the procedure, "Allocating recording volumes in the DIRP utility" in Routine Maintenance Procedures () , 297-8001-546 (North American market) or 297-9051-546 (International market). When you complete the procedure, return to this point. Determine the state of the DLOG recording volumes.
>QUERY DLOG ALL

151

Example of a MAP response


SSNAME SSNO SEQNO ROTATES POOLNO PARLPOOL EMERGENCY DLOG 2 1 102 10 NONE ****YES*** . REGULAR FILE(S) STATE VOLUME RECCOUNT BLOCK E V V_B VLID FNUM FRN# ACTIVE AVAIL S01DDLOG 6 6 0 22 NO 8447 0013 204D STANDBY1 AVAIL S00DDLOG 0 0 0 23 NO 8408 0014 309B . REGULAR VOLUME(S) VOL# VOLNAME STATE IOC CARD VOL FSEG ROOM VLID FILES 22 S01DDLOG READY N/A N/A 7 5 18 8447 A 23 S00DDLOG READY N/A N/A 8 4 18 8408 S1 REGULAR SPACE VOL# VOLNAME STATE SEGS EXP UNEXP TOTAL 22 S01DDLOG READY 5 13 0 18 23 S00DDLOG READY 4 14 0 18

152

If the state of DLOG recording volumes is no allocated volumes, as indicated by the word NONE under the state header on the MAP display, then go to step 158. If the state of DLOG recording volumes is any volume is IN ERROR, as indicated under the REGULAR VOLUME(S) header on the MAP display then go to step 155. If the state of DLOG recording volumes is all volumes are READY, as indicated under the REGULAR VOLUME(S) header on the MAP display then go to step 159.

153

154

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

Procedure steps

393

ATTENTION
Different billing systems than automatic message accounting (AMA) or additional billing system, can be in your office configuration. Call your next level of support to determine if other billing systems are in your office, and if you require recovery action.

155 156 157 158

Reset any volumes that are IN ERROR.


>RSETVOL vol_name

If the volumes reset passed, then go to If the volumes reset failed, then go to step 180. Perform the procedure, "Allocating recording volumes in the DIRP utility" in Routine Maintenance Procedures () , 297-8001-546 (North American market) or 297-9051-546 (International market). When you complete the procedure, return to this point. Determine from your next level of support if you need to recover other DIRP systems (for example, JF, OM). If you need to recover other DIRP systems, then go to step 162. If you do not need to recover other DIRP systems, then go to step 168. Perform the correct procedures in Alarm Clearing and Performance Monitoring Procedures () , 297-8001-543 (North American market) or 297-9051-543 (International market). When you complete the procedures, return to this point. Determine if DIRP logs generated.
>LOGUTIL;OPEN DIRP

159 160 161 162

163 164 165 166

If DIRP log generated a yes, then go to step 166. If DIRP log generated a no, then go to step 168. See the Log Report Reference Manual () , 297-8001-840 (North American market) or 297-9051-840 (International market), and take the correct action. When you complete the log report activities, return to this point. Turn off priority.
>PRIORITY OFF

167 168

Access the SRSTATUS level of the MAP display.


>MAPCI;MTC;SRSTATUS

Example of a MAP response

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

394

Network power up SRSTATUS 0 Quit OVERALL STATUS Pend: 0% Inprg: 0% Comp: 100% Fail: 0% 2 View_ View: SYSTEM 14:08:30 3 List_ Pend InPrg Comp Fail Pend InPrg Comp Fail 4 MS 0 0 2 0 IOD 5 5 30 2 5 NET 0 0 6 0 Other 21 3 13 3 6 SER1 0 0 41 0 7 SER2 0 0 39 0 8 SER3 0 0 37 0 9 10 MTC: 11 STATUS:

169 170 171 172 173 174

Determine the status of the switch recovery. If the status of the switch recovery is any Series III PMs that failed recovery, then go to step 174. If the status of the switch recovery is any Series I or II PMs that failed recovery, then go to step 176. If the status of the switch recovery is any IODs or other devices and services that failed recovery, then go to step 179. If the status of the switch recovery is that the system has completely recovered, then you have completed the procedure. Manually recover the PMs. For more information, see Recovery Procedures, 297-8001-545 (North American market) or 297-9051-545 (International market). When you complete the procedure, return to this point. Go to step 168. Determine from office records or other office personnel which PMs you can recover first. To manually recover the PMs in the required order perform the correct alarm clearing procedures in Alarm Clearing and Performance Monitoring Procedures () , 297-8001-543 (North American market) or 297-9051-543 (International market). When you complete the procedure, return to this point. Go to step 168. To manually recover IODs and other devices and services, perform the correct procedure in this document. Also you can see your site-related operating procedures.

175 176 177

178 179

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

Procedure steps

395

180

For additional help, call the personnel responsible for the next level of support.
--End--

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

396

Network power up

Powering up the Message Switch


Use this procedure to power up the Message Switch (MS) when recovering from a power outage.

Procedure steps
Step 1 Action At the XA-Core frame, locate the NT9X31 and NT9X30 power converters for message switch 0 (MS 0) in slots 33F and 36F on the MS 0 shelf. Turn on the NT9X31 and NT9X30 power converters in slots 33F and 36F of the MS 0 shelf at the same time. Lift and release the power switches located on the faceplates of the converters. Locate the NT9X31 and NT9X30 power converters for MS 0 in slots 1F and 4F on the MS 0 shelf. Turn on the NT9X31 and NT9X30 power converters in slots 1F and 4F of the MS 0 shelf at the same time. Lift and release the power switches located on the faceplates of the converters. Locate the NT9X31 and NT9X30 power converters for message switch 1 (MS 1) in slots 33F and 36F on the MS 1 shelf. Turn on the NT9X31 and NT9X30 power converters in slots 33F and 36F of the MS 1 shelf at the same time. Lift and release the power switches located on the faceplates of the converters. Locate the NT9X31 and NT9X30 power converters for MS 1 in slots 1F and 4F on the MS 01shelf. Turn on the NT9X31 and NT9X30 power converters in slots 1F and 4F of the MS 1 shelf at the same time. Lift and release the power switches located on the faceplates of the converters. At the MAP terminal, access the SRSTATUS level of the MAP.
>MAPCI;MTC;SRSTATUS

3 4

5 6

7 8

Example of a MAP response

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

Procedure steps SRSTATUS 0 Quit OVERALL STATUS Pend: 0% Inprg: 0% Comp: 100% Fail: 0% 2 View_ View: SYSTEM 14:08:30 3 List_ Pend InPrg Comp Fail Pend InPrg Comp Fail 4 MS 0 0 2 0 IOD 5 5 30 2 5 NET 0 0 6 0 Other 21 3 13 3 6 SER1 0 41 0 0 7 SER2 0 39 0 0 8 SER3 0 37 0 0 9 10 MTC: 11 STATUS:

397

10

Determine the recovery status of the MSs.

ATTENTION
MS recovery status displays to the right of the word "MS" in the MAP display. Recovery status for each MS can be one of pending, in progress, complete, or failed.

11 12 13 14

If the recovery status is MS failed recovery then go to step 20. If the recovery status is MS continues to have pending recovery then go to step 14 If the recovery status is a status other than failed or pending recovery then go to step 15 Wait until both MSs are either in recovery development or have completed recovery. If neither MS indicates pending recovery, go to step 10. Access the MS Clock level of the MAP display.
>MAPCI;MTC;MS;CLOCK

15 16 17 18 19

Synchronize the clocks.


>SYNC

If the SYNC command is successful then go this procedure is complete If the SYNC command is failed then continue. Record the reason for synchronization failure, as shown in the MAP response. Repeat the try to synchronize the MS clocks later, after networks and PMs are in service.

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

398

Network power up

20

For additional help, call the personnel responsible for the next level of support.
--End--

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

Procedure steps

399

Powering up the Enhanced Network


Use this procedure to manually power up the enhanced network (ENET) in the event of a power failure.

Procedure steps
Step 1 2 3 4 5 6 7 8 Action At the ENET frames, locate the NT9X31 power converters in slots 1F and 33F on the ENET shelves. To turn on the NT9X31 power converters lift and release the power switches located on the faceplates of the converters. Locate the NT9X30 power converters in slots 4F and 36F on the ENET shelves. Turn on the NT9X30 power converters lift and release the power switches located on the faceplates of the converters. Determine if all the converters have power. Power is indicated by all the Converter Off lights going off. If all the power converters have not power then go to step 8. If all the power converters have power then go to step 9. Power up the ENET frame. Perform procedure, "Clearing an Ext FSP DPCC cabinet major alarm" in the document, Alarm and Performance Monitoring Procedures () , 297-8021-543. When you complete the procedure, return to this point. 9 At the MAP terminal, access the SRSTATUS level of the MAP display.:
>MAPCI;MTC;SRSTATUS

Example of a MAP response

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

400

Network power up SRSTATUS 0 Quit OVERALL STATUS Pend: 0% Inprg: 0% Comp: 100% Fail: 0% 2 View_ View: SYSTEM 14:08:30 3 List_ Pend InPrg Comp Fail Pend InPrg Comp Fail 4 MS 0 0 2 0 IOD 5 5 30 2 5 NET 0 0 6 0 Other 21 3 13 3 6 SER1 0 41 0 0 7 SER2 0 39 0 0 8 SER3 0 37 0 0 9 10 MTC: 11 STATUS:

10

From the MAP display, determine the recovery status of the network.

ATTENTION
Network recovery status displays to the right of the word "NET" in the MAP display. Recovery status can be one of pending, in progress, complete, or failed.

11 12 13 14 15 16 17

If the status of any network element is failed then go to step 16 If the status of any network element is pending then go to step 14 If the status of any network element is any other status then this procedure is complete. Continue when there are no network elements that continue to be pending recovery. Go to step 10. Manually recover the ENET. Perform procedure "Recovering the Enhanced Network" in Recovery Procedures () , 297-8021-545. Go to step 9.
--End--

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

Procedure steps

401

Powering up an Ethernet Routing Switch 8600


Use this procedure to power up a Ethernet Routing Switch 8600.

Procedure steps
Step 1 Action At the Ethernet Routing Switch 8600 frame, turn the DC input power source on or reset the power source circuit breaker to provide power to the power supply. Turn the power supply switch for all three DC power supplies to the on position.

ATTENTION
You must turn on two of the power supply units within 2 seconds of each other. If you wait longer to turn on the second power supply, one of the power supplies could shut down. To correct this condition, turn off both power supplies, wait at least 30 seconds, and then turn on both power supplies again within 2 seconds.

--End--

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

402

Network power up

Powering up the Link Peripheral Processors


Use this procedure to power up the Link Peripheral Processors (LPP) or fiberized LPPs (FLPPs) in the event of a power failure.

ATTENTION
Throughout this procedure, LPP is used to refer to both the LPP and FLPP.

Procedure steps
Step 1 2 3 4 5 Action If PDC has not power restored to the LPPs then go to step 3 If PDC has power restored to the LPPs then go to step 7. If there are blown PDC fuses supplying the LPP,then go to step 5. If there are novisible blown fuses then go to step 6. Replace the blown fuses.

ATTENTION
If fuses blow repeatedly, call your next level of support for help.

When PDC power restores to the LPPs, continue this procedure at step 7. 6 Call the personnel responsible for maintaining power at your site, or see your next level of support for help. When PDC power restores to the LPPs, continue this procedure. 7 8 9 10 11 Locate the LPPs for recovery. At the LPP cabinet, release the locking levers and gently unseat the NT9X74 cards on the link interface shelves(LIS). Refer to Toggle the switch on each NT9X30/NTDX16 power converter cards on the link interface module (LIM) unit shelf. Re-toggle the switch on each NT9X30/NTDX16 power converter cards on the link interface module (LIM) unit shelf. If all of the cards did not turn on the power correctly, go to step 13. All of the CONVERTER OFF lights go off when the power turns on correctly. 12 If all of the cards turned on the power correctly, go to step 15. All of the CONVERTER OFF lights go off when the power turns on correctly.

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

Procedure steps

403

13

To power up the frame, perform the procedure "Clearing and Ext FSP LPP cabinet major alarm" in Alarm Clearing and Performance Monitoring Procedures (297-8991-543) for North American markets or (297-9051-543) for International markets. Go to step 8. Slide each of the NT9X74 cards back into its slot. Gently push on the upper and lower edges of the faceplate of each card until the card is locked into the shelf. Close the locking levers on each card. Repeat steps 5 through 17 for each LPP in your office. Continue this procedure when power has been restored to all LPPs. Access the SRSTATUS of the MAP display>MAPCI;MTC;SRS TATUS
SRSTATUS 0 Quit OVERALL STATUS Pend: 0% Inprg: 0% Comp: 100% Fail: 0% 2 View_ View: SYSTEM 14:08:30 3 List_ Pend InPrg Comp Fail Pend InPrg Comp Fail 4 MS 0 0 2 0 IOD 5 5 30 2 5 NET 0 0 6 0 Other 21 3 13 3 6 SER1 0 41 0 0 7 SER2 0 39 0 0 8 SER3 0 37 0 0 9 10 MTC: 11 STATUS:

14 15 16 17 18 19

20

If the recovery status of the Series III PMs is not zero, go to step 22.

ATTENTION
Series III PM recovery status displays to the right of the word "SER3" (line 8) in the MAP display. Recovery status can be pending, in progress, complete or failed.

21 22

If the recovery status of the Series III PMs is zero, this procedure is complete. To manually recover the PMs, perform the procedure "Recovering Link Peripheral Processors" in Recovery Procedures, (297-8021-545).
--End--

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

404

Network power up

Job aid
Card NT9X74 NT9X30 NTDX16 Slot position 7F, 32F 4F, 36F 1F, 4F, 33F, 36F

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

Procedure steps

405

Powering up the SDM


Use this procedure to power up the SuperNode Data Manager (SDM) that houses the Communication Server 2000 Core Manager.

Procedure steps
Step 1 2 3 4 5 Action At the SDM frame, turn on the two top breakers at the top of the SDM frame. Wait a few moments until the green lights on each card are steady. At the VT100 terminal, log on as root and type
# sdmmtc

Verify that all hardware is in service. Verify that rootvg and datavg logical volumes are mirrored at the storage level.

ATTENTION
There will likely be alarms at the connection and application levels. The general status of the SDM will be in-service trouble (IsTb).

--End--

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

406

Network power up

Powering up a Netra 240 server


Use this procedure to power up a Netra 240 server. Powering up the N240 server supplies power for the following applications:

IEMS Core and Billing Manager Communication Server 2000 Management Tools applications MG 9000 Manager

Procedure steps
Step 1 2 Action At the SDM frame, turn on the circuit breakers to bring the duplex N240 servers into service. To help with recovery, turn the power on to what was the active server first. If there is no record of this, turn on the Unit 0 server before the Unit 1 server. After power is restored, the servers recover on their own.
--End--

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

Procedure steps

407

Powering up the Border Control Point Manager


Use this procedure to power up the Border Control Point Manager. The Border Control Point Manager is installed on a Sun Netra 240 (N240).

Procedure steps
Step 1 Action At the frame housing the Border Control Point Manager, for DC powered shelves, switch the circuit breakers at the EBIP that provide power to the servers being powered to the ON position. The breaker will have the | side (top) of the breaker depressed.

ATTENTION
The DC powered N240 does not have a Power ON/OFF switch and can only be powered ON/OFF from the breaker at the EBIP.

For AC powered shelves, push the rocker located on the top of the AC outlet to apply power to the servers. It should be switched to the "|" position. After the breakers are switched ON at the power source, power up the servers by switching the breakers on the back of the chassis to the ON position. Verify the system is powered up correctly by observing the LEDs on the front of the N240.
--End--

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

408

Network power up

Powering up the SAMF frame


Use this procedure to power up the SAMF frame. Powering up the SAMF frame allows the SAM21 hardware cards, the Policy Controller unit, and the Session Server unit to return to service.

Procedure steps
Step 1 Action At the SAMF frame, power up the SAMF BIP breakers. At the top of the cabinet, turn on the breakers that supply power to the two SAM21 shelves and the Session Server (if equipped). At the Communication Server 2000 SAM21 Manager, if necessary, unlock the previously active Shelf Controller (SC). See Unlocking a SAM21 Shelf Controller (page 353). 3 4 5 If necessary, unlock the previously inactive SC. See Unlocking a SAM21 Shelf Controller (page 353). If necessary, unlock any remaining locked cards on the SAM21 other than the GWC cards. At the Communication Server 2000 GWC Manager, if necessary, unlock all previously active GWC cards. See Unlocking a GWC card (page 457). 6 7 If necessary, unlock all previously inactive GWC cards. See Unlocking a GWC card (page 457). At the Policy Controller unit, determine which physical or logical unit you want to become active and power that unit up first by executing procedure Power-On and boot a Policy Controller unit.. After the active unit has begun call processing, power up the mate unit by executing procedure Power-On and boot a Policy Controller unit.. At the Session Server unit, determine which physical or logical unit you want to become active and power that unit up first by executing procedure Power-On and boot a Session Server unit.. Log onto the active unit to monitor the status of the SIP Gateway application using procedure View the operational status of the SIP Gateway application.. Verify that the Administrative state of the SIP Gateway application becomes Unlocked and the Operational state becomes Enabled. If the active unit comes up in any state other than Unlocked:Enabled, see Nortel Session Server Trunks Administration and Security ( (NN10346-611)) ,
Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

10

Procedure steps

409

11

Complete one or both of the following procedures:Unsuspend the SIP gateway application OR Unlock the SIP gateway application

12

After the active unit has begun call processing, power up the mate unit by executing procedure Power-On and boot a Session Server unit.. From the active unit, verify that the SIP Gateway application databases on the both units have synchronized using procedure Verify synchronization status of Session Server units. Monitor the system for an appropriate period according to your site guidelines. If you experience problems with call processing, see Nortel Session Server Trunks Administration and Security ( (NN10346-611)) ,

13

14

15

Perform the following procedures in order: Invoke a maintenance SwAct of the Session Server platform AND Inhibit a system SwAct (Jam)

16

Contact your next level of support or Nortel GNPS for further assistance.
--End--

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

410

Network power up

Powering up the Call Control Frame


Use this procedure to power up the Call Control Frame (CCF). Powering up the CCF allows the SAM21 hardware cards and the STORM disk array to return to service.

Procedure steps
Step 1 2 3 4 Action At the CCF, determine the configuration of your CCF. If the CCF configuration includes a STORM disk array and two STORM cards in the Call Agent shelf then go to step 4. If the CCF configuration includes two STORM SAM-XTS units then go to step 5. Power up the CCF BIP breakers. At the top of the cabinet, turn on the breakers that supply power to the STORM disk array and the two SAM21 shelves. Go to step 8. 5 Power up the CCF BIP breakers. At the top of the cabinet, turn on the breakers that supply power to the two SAM21 shelves and the STORM SAM-XTS units. At the front panel of the STORM SAM-XTS units, if necessary, power on the STORM SAM-XTS units using the main power switch located on the front panel.

7 8

At the console, monitor the boot progress of the unit until booting is complete. At the Communication Server 2000 SAM21 Manager, if necessary, unlock the previously active Shelf Controller (SC). See Unlocking a SAM21 Shelf Controller (page 353).

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

Procedure steps

411

9 10

If necessary, unlock the previously inactive SC. See Unlocking a SAM21 Shelf Controller (page 353). If necessary, unlock the previously active Compact Call Agent (CCA) card. See Unlocking the Call Agent (page 456). If necessary, unlock the previously inactive CCA card. See Unlocking the Call Agent (page 456). If necessary, unlock the previously active USP-lite card. If necessary, unlock the previously inactive USP-lite card. If necessary, unlock any remaining locked cards on the SAM21 other than the GWC cards. At the Communication Server 2000 GWC Manager, if necessary, unlock all previously active GWC cards. See Unlocking a GWC card (page 457). If necessary, unlock all previously inactive GWC cards. See Unlocking a GWC card (page 457).
--End--

11 12 13 14 15

16

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

412

Network power up

Powering up a USP
Use this procedure to power up a Universal Signaling Processor (USP). The shelf uses the USP RTC-12 to initially get back into service.

Prerequisites

The Communication Server 2000 Core and USP ABS server must be in-service.

Procedure steps
Step 1 Action At the USP shelf, ensure all cards are seated in the USP shelf with the exception of the RTC-15 (slot 15) front panel card on the Control shelf. Power up both A/B buttons on the control and extension shelves. RTC-12 should SCSI boot and it should come into service within 1-2 minutes of being powered up. At the USP Manager, after RTC-12 is up, use the USP Manager to uninhibit and activate all links, where applicable. See Configuring Links. Verify logs and alarms to ensure no unexpected problems and perform test calls. Verify that all cards on all shelves are up and running with the exception of RTC-15. Ensure that the System Date/Time is set to the current local time, if not using SNTP. See Administration: Date and Time. At the USP frame, after all cards are up and properly running, reseat RTC-15. RTC-15 should SCSI boot and perform a database synchronization with RTC-12. RTC-15 should come into service in approximately 5-7 minutes. At the USP Manager, verify that RTC-15 is in-service and inactive. Verify logs and alarms again to ensure no unexpected or undesired events and perform test calls. Verify that the USP can communicate with the ABS. From the USP Manager main menu, click on the Administration button and select ABS Settings. Select the Test button to initiate a series of tests to ensure the USP can properly communicate with the ABS.

2 3 4

5 6 7 8

9 10 11

12

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

Procedure steps

413

13

Check the USP logs for the test results. Troubleshoot ABS connectivity in the event that the tests fail.

ATTENTION
To assist with troubleshooting, the Logs message includes the nature of the test failure.

14 15

Take a snapshot of the USP, and copy the snapshot to the ABS using the File Manager. Using the ABS CM GUI, modify the site information to point to the new snapshot.
--End--

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

414

Network power up

Powering up a Media Gateway/Multiservice Switch 15000


Use this procedure to power up a Media Gateway/Multiservice Switch 15000 when it is necessary to power up a Media Gateway/Multiservice Switch 15000.

Procedure steps
Step 1 Action At the chassis, locate the breaker interface panels (BIP). All the BIPs are at the top of the chassis.

ATTENTION
If the CP card or its disk becomes unserviceable, the backup-up provisioning data and software are downloaded from the MDM backup copy and SDS site, respectively.

Switch the breakers on all the BIPs to the ON position. The system should come up on its own.
--End--

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

Procedure steps

415

Powering up a Media Gateway/Multiservice Switch 7400


Use this procedure to power up a Media Gateway/Multiservice Switch 7400.

Prerequisites

All power supply switches must be in the standby position.

Procedure steps
Step 1 2 3 Action At the chassis, disengage all processor cards from the backplane of the shelf. Engage the minimum number of processor cards required, based on the number of power supplies in the shelf. If you have one power supply, engage a CP in slot 0 and at least one FP. Do not engage more than seven FPs. Ensure that all other processors are disengaged. 4 5 6 7 If you have two power supplies, engage at least one CP and two FPs. Or, engage two CPs and at least one FP. Turn on the circuit breakers for the outlets that supply power to your switch. Verify that the LEDs on all power supplies are red. Apply power to one of the power supplies by setting its power switch to the on position. Verify that the power supply LED is green. Normal operation is not guaranteed if there are no processor cards in the shelf. If you have not installed any processor cards before you switch on a power supply, the system does not supply power to the shelf and the power supply LED remains red. 8 9 Verify that the cooling unit LED is green and is operational. You should be able to hear the fans start to rotate. If necessary, apply power to a second and third power supply by setting their power switches to the on position. Verify that the LED for each power supply is green.

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

416

Network power up

If you install eight or more FPs, you must use a second power supply. If your system contains fewer than eight FPs, the system uses the second power supply for redundancy. 10 11 12 If necessary, engage the remaining processor cards. Verify the LED color on each FP in appropriate. Verify that the appropriate LEDs are illuminated on termination panels.
--End--

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

Procedure steps

417

Powering up the MDM workstation


Use this procedure to power up the MDM Sun FireTM V480 workstation.

ATTENTION
If your MDM is an N240 server, see the procedure "Powering up a Netra 240 server."

Prerequisites

An MDM user account is required.

Procedure steps
Step 1 2 Action At the workstation, switch the workstation on. After the workstation has started, log on with an MDM user account and ensure that the MDM toolset is available.
--End--

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

418

Network power up

Powering up an SPM device


Use this procedure to power up a Spectrum Peripheral Module (SPM), Media Gateway 4000 (MG 4000), Interworking Spectrum Peripheral Module (IW-SPM), or Dynamic Packet Trunking Spectrum Peripheral Module (DPT-SPM).

Procedure steps
Step 1 2 3 4 5 6 Action At the MAP terminal, restore power to the frame and re-insert the fuses. Reseat the SIM cards if they were unseated and turn them on one at a time using the switches on the front. Reseat any unseated cards in the shelves. Post the SPM/MG 4000/IW-SPM/DPT-SPM.
>MAPCI;MTC;PM;POST SPM <spm_no>

Select and RTS the CEMs.


>SELECT CEM ALL;BSY ALL; LOADMOD ALL;RTS ALL

Select and RTS the SRM, OC3 RMs, ATM RMs, and/or GEM RMs.
>QUIT;SELECT <rm_type> ALL;BSY ALL;LOADMOD ALL;RTS ALL

ATTENTION
To RTS the ATM, use the BSY FORCE and RTS FORCE commands.

Bring the carriers into service.


>QUIT ALL;MAPCI;MTC;TRKS;CARRIER;POST SPM <spm_no>;BSY ALL;RTS ALL

Post and recover the trunks on the SPM at the TTP level. (Press the Enter key after each of the following commands.)
>QUIT ALL;MAPCI;MTC;TRKS;TTP;PRADCH;POST AD SPM <spm_no>;BSY ALL;RTS ALL >QUIT ALL;MAPCI;MTC;TRKS;TTP;POST D SPM <spm_no>;BSY ALL;RTS ALL

9 10

Return to the PM level and post the SPM.


>QUIT ALL;MAPCI;MTC;PM;POST SPM <spm_no>

Select and RTS the remaining RMs.

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

Variable definitions >SELECT <rm_type> ALL;BSY ALL;LOADMOD ALL;RTS ALL


--End--

419

Variable denitions
Variable <spm_no> <rm_type> Value The number of the SPM/MG 4000/IW-SPM/D PT-SPM. This is either SRM, OC3, ATM, or GEM.

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

420

Network power up

Powering up the MCS servers


Use this procedure to power up the eight MCS servers. The database server should be brought into service before any other servers. If other servers come up first, restart the management server.

Procedure steps
Step 1 Action

ATTENTION
The following shows the preferred order, which is not the way the system comes up if all servers are powered on at the same time.

At the frame housing the MCS servers, power on the database server by turning on the circuit breakers that provide power to the servers. It takes from 5 to 8 minutes for the server to boot and load Oracle. 2 3 After the DBSvr powers on, telnet to the box and run a spot check to make sure it is ready. Make sure Oracle is loaded. The best way is to telnet to the database. Type: sqlplus and fill in the username and password (in other words, log on as user, not root). If Oracle is running, you should receive the SQL> command prompt. Type quit to exit from sqlplus. Verify that the snmp service is running on the database box. Type: ps -ef | grep snmp. You should see three snmp processes:
/usr/local/sbin/snmpd -f udp:161 /bin/sh /opt/app/oracle/product/9.2.0/bin/dbsnmpwd /opt/app/oracle/product/9.2.0/bin/dbsnmp

The snmp processes are not critical to getting the system up. However, if any of the snmp processes are not running, you will not be able to see accurate reporting of this box on the System Management Console. 5 To stop or start oracle, telnet to the database as sysadmin and then su to root. Go to the following directory: /etc/init.d. The stop command is ./dbora stop. The start command is ./dbora start. These commands will severely affect service. Verify that the SysMgr processes on the MgmtSvr/AcctMgr box are running to control the MCS components from the System

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

Procedure steps

421

Management Console. Telnet to the MgmtSvr/AcctMgr box as nortel. 7 Make sure that the SysMgr processes are running by typingmeinit -p. You should see three processes running. If any of these processes are not running, you will not be able to launch the System Management Console:
Rel2.0 NTme_pids mgmtsvr mgmtsvr.3 Rel2.0 NTme_pids tsscma tsscma.5 Rel2.0 NTme_pids tssfpma tssfpma.6

To stop or start the SysMgr, go to the following directory: IMS/m gmtsvr/bin/mgmtsvr/ and type: ./MgmtSvrShutdown.pl. This will kill any (or all) of the SysMgr processes. To start the SysMgr, stay in the same directory and type: MgmtSvrConfigSetup.pl. This will try to start all three processes. You will see a messages on the telnet session that say something like "starting tsscma waiting 30 seconds." The tssfpma process will start next, followed by the mgmtsvr processes. If you want to watch the SysMgr startup logs, after performing step 3b above, go to the following directory: /var/Rel2.0/mgmtsvr. Then type: tail -f mgmtsvr.3.log. Verify that the snmp process is running on the MgmtSvr/AcctMgr box by typing ps -ef | grep snmp. You should see one process running: /usr/local/sbin/snmpd -f udp:161. If this process is not running or you get an alarm from the System Management Console that there is a problem with the MgmtSvr/AcctMgrs snmp, then kill the process. The process will automatically restart in about 30 seconds.

10

11

12

Restart the AcctMgr process. If any of the components send an alarm in the System Management Console indicating that the component cannot communicate with the primary or backup CAM, then again restart the AcctMgr process. Right click on the Accounting component in the System Management Console and select Restart. Confirm the request. A progress box will pop up and then disappear when the restart has begun. While the Accounting module is rebooting, the other MCS component will throw alarms to the System Management Console, but those will go away when the Acct component is fully operational. Now verify the state of the other machines and the MCS components on them. Launch the System Management Console as you normally would. Expand the navigation tree fully so that each component is fully exposed.

13

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

422

Network power up

14

Restart the AppSvr. If the clients are having trouble communicating with the AppSvr, then restart the AppSvr. Right click on the AppSvr component in the System Management Console and select Restart. Confirm the request. You should see a progress window and it disappears after the restart has begun. To view AppSvr startup logs, telnet to the AppSvr box (we recommend that you telnet as nortel) and go to the following directory: /var/Rel2.0/appsvr. Type: tail -f appsvr.0.log. Verify that the AppSvrs snmp process is running. This step is the same as step 3e above. Restart the IPCM. If the i2004 phones are not responding to client registers, the hollow blinking icons in the i2004 display, then restart the IPCM component. Right click on the IPCM component in the System Management Console and select Restart. Confirm the request. You see a progress window that will disappear after the restart has begun. To view IPCM startup logs, telnet to the IPCM box (recommend as nortel) and go to the following directory: /var/Rel2.0/esm. Type: tail -f esm.1.log. Verify that the IPCMs snmp process is running. This step is the same as step 3e above. Most likely, the Provisioning Module and the WebClient Module are deployed on the same box as the IPCM. Therefore, you only need to verify the snmp service on the box once. Restart the Provisioning component. The best way to see if the Provisioning Module is behaving properly is to log on to the ProvClient. Launch a web browser to the IP address of the box where the Provisioning Module is deployed, such as http:192.168.0.10/prov. Log on as admin/admin and attempt any of the List options in the navigation tree, for example, List Devices or List Users. If the browser doesnt respond properly, then restart the Provisioning Module by right clicking on the Provisioning component in the System Management Console and select Restart. Confirm the request. Restart the WebClientMgr component. Log on to the Personal Agent through your browser, such as http://192.168.0.10/pa. Log on as a subscriber, such as myusername@mydomain.com, and enter the subscribers password. The Personal Agent should appear for that subscriber. If it does not, or the interface does not seem to respond correctly, restart the Provisioning Module described in step 4e above. (If youve already restarted it once and the ProvClient interface seems ok, but the Personal Agent interface is still not working correctly, it is probably a configuration issue.

15 16

17

18

19

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

Procedure steps

423

20

Launch the WebClient GUI from inside a subscribers Personal Agent page. Attempt to log on to the WebClient when the System Management Console completes loading.
--End--

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

424

Network power up

Powering up a Border Control Point


Use this procedure to power up the Border Control Point. The Border Control Point shelves are powered from either a PDU or EBIP located at the top of the frame depending on if the frame is AC or DC powered.

Procedure steps
At the Ethernet Routing Switch 8600 Workstation

Step 1

Action At the frame housing the Border Control Point, for DC powered shelves, switch the circuit breakers at the EBIP that provide power to the shelf being powered to the ON position. The breaker will have the 1 side (top) of the breaker depressed. For AC powered shelves, push the rocker located on the top of the AC outlet to apply power to the chassis. It should be switched to the "|" position. After the breaker is switched ON at the power source, power up the shelf by switching the breaker on the back of the chassis to the ON position. Observe the System, Power/Fan, Slot (filled) and Host Card LEDs as the unit is being powered. After power on is complete, they should all have a steady green color.
--End--

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

Procedure steps

425

Powering up the IPmedia 2000 shelves


Use this procedure to power up the IPmedia 2000 shelves that house the MS 2000 series devices and the Packet Media Anchor. The IPmedia 2000 shelves reside in either a SAMF frame or CCF, depending on whether or not your solution uses the Communication Server 2000 or Communication Server 2000-Compact.

Procedure steps
Step 1 2 3 Action At the CCF or SAMF frame, open the front door of the CCF/SAMF frame. At the EBIP at the top of the frame, turn the breaker associated the Media Server to the ON position. Ensure that the Green Power LED on the front of the Nortel Media Server 2000 is on (right hand side of the card). After 1 to 2 minutes, the Green LED on the left hand side of the card lights. 4 Verify the status of the LEDs on the front of the Nortel Media Server 2000 . 5 6 Red Fail LED is off (left hand side of the board). ACT LED is on (left hand side of the board).

ATTENTION
It may take up to 60 seconds for the ACT LED to light up.

7 8 9

Blue LED does not remain on (right hand side of the board). PWR LED is on (right hand side of the board). Both Ethernet Link LEDs are on (middle of the board).

ATTENTION
The Ethernet LEDs are Green when indicating a connection, and Orange when indicating activity.

ATTENTION
Check the Media Server 2000 to make sure it is set to an unlocked state.

--End--

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

426

Network power up

Powering up the SAM16 shelves


Use this procedure to power up the SAM16 shelves that house the UAS devices. The SAM16 shelves reside in either a SAMF frame or CCF, depending on whether or not your solution uses the Communication Server 2000 or Communication Server 2000-Compact.

Procedure steps
Step 1 2 Action At the CCF or SAMF frame, open the front door of the CCF/SAMF frame. At the EBIP at the top of the frame, turn the breakers associated the SAM16 to the ON position.
--End--

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

Procedure steps

427

Powering up an MG 9000 device


Use this procedure to power up the MG 9000 device.

Procedure steps
Step 1 Action At the Subnet View of the MG 9000 Manager, select the MG 9000 to be re-commissioned. A warning window indicating that your connection to the gateway is down may appear. This will not block the saving of services information. Save a copy of the MG 9000 Manager SLoA provisioning data as follows (youll be re-entering this data later): To Save the SLoA services provisioning data, at the Frame (Element) view, select the "Save SLOA services" option from the Actions menu.

2 3

An acknowledgment window appears in response to this request. The acknowledgment window contains the name of the two files (with .html and .text suffixes) where your data had been stored on the MG 9000 Manager. 4 Record the file name provided in the Save acknowledgement window for later use.

Save a copy of the MG 9000 Manager PLoA provisioning data as follows (youll be re-entering this data later):

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

428

Network power up

To save the PLoA services provisioning data, at the Frame (Element) view, select the "Save PLOA services" option from the Actions menu. An acknowledgment window appears in response to this request. The acknowledgment window contains the name of the files where your data had been stored on the MG 9000 Manager.

Record the file name provided in the Save acknowledgement window for later use.

Obtain the IP address of the MG 9000 Manager server (not the mid-tier or client). At the Subnet View, select Configuration->View/Modify NE Properties from the menu bar.

Record the IP address in the MG 9000 Manager IP Address field.

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

Procedure steps

429

10

FTP to the address recorded in step 5 and copy the files in the /tmp directory to a secure location, such as your desktop or permanent server before proceeding.

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

430

Network power up

11

At the MG 9000 frame, power up the shelves in the frame by reinstalling the fuse modules on the IBIP. Restore all fuses left to right in the frame. Connect a laptop PC to the active DCC card and launch the local craft interface (LCI). Wait for the restart to complete and for all cards initialize, typically takes 5 minutes. The Administrative State of all common cards will be Locked. From the LCI, Unlock the active DCC OC-3/DS1 IMA card by selecting the Admin. State Unlock radio button. Wait for the status at the LCI to change to Enabled, Online, and Unlocked. From the LCI continue to unlock the common cards in the following order until all are unlocked and online:

12 13

14

15

active ITX active ITP OC-3 carriers on the active OC-3 DCC card inactive DCC inactive ITX inactive ITP OC-3 carriers on the inactive OC-3 DCC card

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

Procedure steps

431


16 17 18

DS1 cards (if provisioned) ABI cards (if provisioned)

Initiate rediscovery by sending a cold start trap from the LCI. At the MG 9000 Manager, monitor the MG 9000 network element and wait until the discovery process is complete. Wait for the network element to show it is operational, then check for dial tone.
--End--

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

432

Network power up

Restoring certicates and Pre-shared Keys for IPSec


Use this procedure to ensure that the current version of the certificates and/or Pre-shared Keys is restored using the backup images of the CVoIP devices. Apache/Tomcat web services and IPSec use these certificates to secure and authenticate call processing traffic. Pre-shared Keys are used to secure and authenticate call processing traffic.

Prerequisites

Perform regular full system backups and critical data backups according to corporate policy. This measure ensures that the current backup version is restored, and that a disaster recovery process is executed quickly without additional outages. Ensure that the next level of support is available for assistance during the recovery process. Have immediate access to the Certificate Manager. Restore the Oracle data on the Communication Server 2000 Management Tools. Also perform this restore procedure for all other SPFS-based servers, including the MDM and MG 9000 Manager. For more information, see Nortel Carrier Voice over IP Disaster Recovery Procedures ((NN10450-900)) .

Procedure steps
Step 1 2 3 4 Action Reinstall the WEBPKPROXY certificate for the Certificate Manager. Launch the Certificate Manager GUI. Unmanage and delete all WEBPKPROXY certificates. Clean up directory /data/NTPkmgr/cert/export on the device on which the Certificate Manager resides by deleting all files in this directory.
>rm -f /data/NTPkmgr/cert/export/*

Do not delete the directory. Delete only the files within it. 5 Recreate and re-install all WEBPKPROXY certificates. This step ensures that directory /data/pkclient/certificates is cleaned up on each management system. In addition, this step re-enables

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

Procedure steps

433

communication between the Certificate Manager and following components:

the Communication Server 2000 Management Tools server the MG 9000 Manager system the MDM

Check that call processing traffic between each Gateway Controller (GWC) and the MG 9000 is not affected. Ensure that the connection between the GWC and the media gateways is still enabled.

CAUTION
Check the list of all certificates in the Certificate Manager. Ensure that each MG 9000 displays the same certificate serial number as the ones listed in the Certificate Manager. Do not rely on the displayed information in the VMG, as communication between the manager and the media gateways can be compromised. An outage can occur if the Certificate Authority serial numbers between the GWC and the media gateways do not match. Contact your next level of support to resolve this conflict.

See procedure Viewing certificate using Certificate Manager in NTP NN10453-100, Nortel Carrier VoIP Security Service Implementation Guide, for details. 8 Contact your next level of support within the next off-peak maintenance window to assist you with performing the following tasks:

to download the certificates to the network element to busy and return to service the inactive GWC node to perform a warm SWACT to revoke the certificate for the MG 9000 VMG or the MG 15000 that this GWC control(s) to busy and return to service the newly active side to clear any GWC307 alarms manually download all certificates on the MG 9000 VMG

During the next available maintenance window, ensure that all Gateway Controller alarms and all network gateway alarms are cleared. Launch the MG 9000 Manager web-based GUI.

10

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

434

Network power up

11

From the Subnet View, select Configuration > View/Modify NE Properties The NE Properties View displays.

12 13

Display the Properties menu of each MG 9000 node to verify that the Push and Pull Certificates match. Click on the NE Security tab. The NE Security View displays.

14

Compare the serial numbers in fields Last Pulled Certificate and Last Pushed Certificates against their Certificate Manager counterparts. If the serial numbers differ, click the Download Certificates button to match the pushed and pulled certificates. Then click the Refresh button.

15

From the menu bar, select Services > Switched Lines Services Manager. The Switched Lines Services Manager displays.

16 17

Select the VMG for which security is to be provisioned. Click the GW Security Config tab for each GWC card with a certificate. The GW Security Config screen displays.

18

Compare the serial numbers in fields Last Pulled Certificate and Last Pushed Certificates against their Certificate Manager counterparts. If the serial numbers differ, click the Download Certificates button to match the pushed and pulled certificates. Then click the Refresh button.

19 20

Verify that the call processing traffic between each GWC and media gateway has not been affected. Perform a full system backup on all management systems and network gateways.
--End--

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

Job aid

435

Job aid
This procedure applies to IEMS, Gateway Controller (GWC), Communication Server 2000 Management Tools, GWC Manager, MDM, MG 15000 and MG 9000 Manager in CVoIP IP Disaster Recovery and IP Security applications. Therefore, see the appropriate NTPs when directed as follows:

Nortel CVoIP IPSec Service Configuration ( (NN10453-100)) for use with WEBPKPROXY certificates for the Certificate Manager Nortel Carrier VoIP Disaster Recovery Procedures ((NN10450-900)) for details and procedures for scheduled system backups. (See Prerequistes.) If Pre-shared Keys are used for authenticating call processing traffic using IPSec, verify that the following conditions exists after a system recovery:

All Gateway Controller alarms and network gateway alarms are


cleared. Connectivity exists between all Gateway Controllers and network gateways.

All pre-shared key data is correct according to current provisioning.

Nortel Carrier Voice over IP Fault Management Logs Reference Volume 2 ( (NN10275-909v2)) for details on alarm CMT301 Nortel Gateway Controller Fault Management ( (NN10202-911)) for details on alarm GWC307

When backing up devices that use certificates provided by the IEMS/Certificate Manager and/or Pre-shared Keys for IPSec, always note the versions of the data in the back-up image of the device. Due to the timing sensitivity of the certificates, a potential conflict can occur during restore between the version of the certificate in the back-up image versus the version of the certificate provided by the Certificate Manager. If the restored data has expired or is not current, call processing traffic that is secured and/or authenticated using IPSec can be affected. This condition can result in additional outages.

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

436

Network power up

Restoring the MG 9000 manager system


Use this procedure to restore the MG 9000 manager system when certificates provided by the Certificate Manager are used in an office.

Procedure steps
Step 1 Action Unmanage and delete the WEBPKPROXY certificates for the MG 9000 manager system. Performing this step disables all communication between the Certificate Manager and the MG 9000 manager system. See Nortel CVoIP IPSec Service Configuration ( (NN10453-100)) for details. 2 Recreate and re-install the WEBPKPROXY certificates for the MG 9000 manager system only, generate the WEBPKPROXY certificate using the IP address and FQDN of the MG 9000 manager system. This step is required for cleaning up the /data/pkclient/certificates directory on each management system. See Nortel CVoIP IPSec Service Configuration ( (NN10453-100)) for details. 3 Install the WEBPKPROXY certificate for the MG 9000 manager system only. See Nortel CVoIP IPSec Service Configuration ( (NN10453-100)) for details. 4 Register the WEBPKPROXY certificate for the MG 9000 manager system only. This step is required to re-enable communication between the Certificate Manager and the MG 9000 Manager System. See Nortel CVoIP IPSec Service Configuration ( (NN10453-100)) for details. 5 Revoke all MG 9000 NE certificates and verify delivery of all MG 9000 certificates. See Nortel CVoIP IPSec Service Configuration ( (NN10453-100)) for details. 6 Perform a full system backup on all management systems and network gateways
--End--

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

Procedure steps

437

Restoring les from a DVD-RW


Use this procedure to restore files from a DVD-RW to restore office images from a digital video disk read write optical disk (DVD-RW). Do not restore an image and overwrite one with the same name. To restore a volume from DVD-RW, contact Nortel support personnel. Perform this procedure when required by your office. Understanding of the IMPORT command in DISKUT, SCANF for listing volumes, and the CBF, LBF, and SBF commands in ITOCCI is required. The IP addresses of the STORM units are needed. Use the mount command from a Call Agent card to determine the addresses.

Procedure steps
Step 1 Action At the Communication server Management Tools server insert the DVD-RW into the DVD tray. If the Communication server Management Tools server is a pair of Sun Microsystems Netra 240s, put the DVD in the unit with the USER LED lit.
Volume management software on the Communication server Management Tools server operating system mounts the DVD-RW.

List the contents:


$ ls -as /cdrom/cdrom0 The contents of the DVD are printed.

3 4

Change directory to the DVD-RW:


$ cd /cdrom/cdrom0

At the MAP enter the DISKADM level and busy the volumes to copy:
> DISKADM <sd0x>; BSY <volname> > QUIT

The BSY command fails if applications have open files on any volume for the device. A busied disk may affect data recording or retrieval for applications. Consider to RTS the volume as soon as possible or convenient after copying the files. If applications are active and writing to the volume, determine if the application can ROTATE the disk writing activity to a backup volume. If unsure, contact your next level of support. 5 At a Communication server Management Tools server terminal secure copy the files from the DVD-RW on the Communication

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

438

Network power up

server Management Tools server to one STORage Management (STORM) unit:


$ scp <image_files> root@<stormip>:</path_to_files>

Example Copy an office image named S040210135002_CM to SD00IMAGE1:


$ scp S040210135002_CM "root@<stormip>:/nfsserv/3pc/cs/sd00/image1"

ATTENTION
The first time this command is issued, the secure copy program provides a prompt to exchange keys. Confirm the exchange with a "y." The secure copy program provides a progress indicator during the copy. Wait for the copy to complete and the $ prompt to return.

At the MAP enter the DISKADM level and RTS the volume:
> DISKADM <sd0x>; RTS <volname> > QUIT

Enter the DISKUT level and IMPORT the image files:


> DISKUT; IMPORT SD00IMAGE1 > QUIT

The status of the IMPORT command is printed.

List the contents of the volume so the file can be added to the Image Table of Contents (ITOC) in the next step:
> SCANF SD00IMAGE1 The contents of the volume are printed.

Enter ITOCCI and register the image in the ITOC:


> ITOCCI > SBF CM S040210135002_CM <itoc_pos> <alr_flag>

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

Variable definitions

439

ATTENTION
To determine if a free position is available, use the LBF CM command. If all positions are used, clear the file in position 15. Determine which volume the position 15 is on, use SCANF to list that volume, and then use the CBF CM FILE <image_name> command.

10

At a Communication server Management Tools server terminal unmount the DVD-RW, eject it, and exit from root privilege:
$ eject cdrom

11 12

At the Communication server Management Tools server remove the DVD-RW from the tray and close the tray. Store the DVD-RW according to office procedure.
--End--

Variable denitions
Variable <alr_flag> Value If this image should be booted for the next restart, specify ALR. Otherwise, leave the field blank. The file name for a single image file, or a wildcard expression such as "*_CM" An integer between 0 and 15, and is a free position in the ITOC. The absolute path to the files on the STORM unit to place the files such as /nfsserv/3pc/cs/sd00/image1 SD00 or SD01 The IP Address of the STORM unit such as 172.18.96.6 The name of the volume such as TEMP

<image_files> <itoc_pos> </path_to_files>

<sd0x> <stormip> <volname>

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

440

Network power up

Reloading or restarting a SAM21 Shelf Controller


Use this procedure to reload or restart a SAM21 Shelf Controller (SC).

Procedure steps
Step 1 Action From the Shelf View, right-click on the card and select Lock.

Wait for the lock icon to appear on the selected card and the other SAM21 SC to indicate that it is in simplex (alarm 2C on the other SAM21 SC). Right-click on the card again and select Unlock. The card resets, downloads software, and reboots.
--End--

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

Procedure steps

441

Restarting or rebooting a GWC card


Restart a GWC card to stop all software processes on the GWC card, perform a hardware reset, and reload the GWC card software from the Core and Billing Manager (CBM). Reboot a GWC card and force a GWC to download and execute a software load from the Communication server 2000 Core Manager or CBM.

Prerequisites

To reduce the risk of service interruption, first busy the GWC applications on specific GWC nodes using the Communication server 2000 GWC Manager. For more information, see Nortel Gateway Controller Administration and Security ( (NN10213-611)) .

Procedure steps
Step 1 Action At the Communication server 2000 SAM21 Manager client from the Shelf View, right-click the GWC card you want to reboot and select Card View.

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

442

Network power up

2 3

At the Card View, select the States tab. Click the Lock button to lock the card.

4 5 6

Wait until the Card Status is Locked and the History window indicates that: Application locked successfully. Click the Unlock button. Monitor the reboot process. Wait until the Card Status changes to Unlocked and the History window indicates Bootloaded successfully.
--End--

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

Procedure steps

443

Performing a partial power up of the Netra 240 server


Use this procedure to perform a partial power down recovery of the N240 servers. The N240 servers are commissioned with one active and one inactive unit. In a partial power down, the inactive unit is powered down, while the active unit remains powered up. This procedure powers up the inactive unit.

Prerequisites

You must have the root user password for the N240. You need to IP address for the inactive unit.

Procedure steps
Step 1 2 Action At the N240 server, the N240 server is inactive. Turn on the circuit breakers. The server should recover on its own and start syncing with the active server.
--End--

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

444

Network power up

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

445

Disaster recovery maintenance


This chapter describes how to maintain the Disaster recovery procedures for the Nortel Carrier VoIP solutions. The following network elements are covered by this section:

XA-Core 3rd Party Core Message Switch ENET Ethernet Routing Switch 8600 LPP SDM, Communication server 2000 Core Manager, or Core and Billing Manager IEMS Communication server 2000 Management Tools MG 9000 Manager Border Control Point Manager SAM21 shelf and cards CICM USP Media Gateway 15000/7400 or Multiservice Switch 15000/7400 MDM workstation SPM, MG 4000, IW-SPM, DPT-SPM MCS servers Border Control Point MS 2000 Series or UAS

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

446

Disaster recovery maintenance

Packet Media Anchor MG 9000

Navigation Performing a partial power down recovery of the USP (page 447) Performing a partial power down recovery of the XA-Core (page 448) Performing a partial power down recovery of the Ethernet Routing Switch 8600 (page 451) Performing a partial power down recovery of the Media Gateway/Multiservice Switch 15000 (page 452) Performing a partial power down recovery of the SPM (page 453)

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

Procedure steps

447

Performing a partial power down recovery of the USP


This procedure is used to perform a partial power down recovery of a USP.

Procedure steps
Step 1 2 Action At the USP frame reseat the front panel and back panel CC card in Slot 18 on the Control shelf. Reseat the front panel and back panel CC cards in Slot 18, beginning with the CC Slot 18 card on each of the extension shelves. Reseat the inactive RTC. The inactive RTC should SCSI boot and perform a database synchronization with the active RTC. It should come into service in approximately 5-7 minutes. 4 5 6 At the USP Manager verify the reseated RTC is in-service and inactive. Verify logs and alarms to ensure there were no unexpected or undesired events. Perform test calls.
--End--

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

448

Disaster recovery maintenance

Performing a partial power down recovery of the XA-Core


Use this procedure to return to normal operation an extended Architecture Core (XA-Core) configuration of a switch. The switch is a DMS SuperNode or a DMS SuperNode SE switch. Use this procedure after having performed all parts of the procedure "Emergency power conservation shutdown" to maintain emergency backup power. The procedure "Emergency power conservation shutdown" follows an extended commercial power outage. This procedure describes equipment recovery in decreasing order based on its effect on system reliability. The equipment restoration begins with more required equipment and ends with less required equipment such as maintenance trunk modules. The configuration of your office and the requirements of your operating company, can change the order of equipment restoration. The return of elements of the switch to service can be in an different order.

Procedure steps
WARNING
Potential extended equipment outage
Nortel recommends that you perform this procedure under the supervision of Emergency Technical Assistance Services (ETAS) of Nortel. Also call your next level of support before you perform this procedure.

CAUTION
Potential loss of service or extended outage
This procedure is only to restore normal operation after the performance of measures for emergency power conservation. Do not use this procedure or parts of this procedure for equipment maintenance purposes.

Step 1

Action At your current location use office records to identify and record the power converters which supply the MAPs and printers of the switch. Restore power to all the power converters identified in step 1that supply power to the MAPs and printers of the switch. At the MAP restore power to one side of the remote oscillator shelf (NT3X9507), If you turned off power to save emergency backup power. To restore power, perform the procedure

2 3

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

Procedure steps

449

"Restoring the remote oscillator shelf to duplex operation" in Recovery Procedures () , 297-8021-545. 4 Restore power to one message switch (MS) shelf if you turned off power to save emergency backup power. To restore power, perform the procedure "Restoring the MS duplex operation" in Recovery Procedures () , 297-8021-545. Restore power to one unit of the CCS7 message switch and buffer (MSB7) if you turned off power to save emergency backup power. To restore power, perform the procedure "Restoring the MSB7 to duplex operation" in Recovery Procedures () , 297-8021-545. Restore power to one link interface module (LIM) unit on one or more link peripheral processors (LPP) if you turned off power to save emergency backup power. To restore power, perform the procedure "Restoring LPP LIMs to duplex operation" in Recovery Procedures () , 297-8021-545. If you removed power from one or more network frames, restore power to the changed frames at the power distribution center (PDC). Restore power to one or more junctored network (JNET) shelves if you turned off power to save emergency backup power. To restore power, perform the procedure "Restoring the junctored network to duplex operation" in Recovery Procedures, 297-8021-545. Restore power to one or more enhanced network (ENET) shelves if you turned off power to save emergency backup power. To restore power, perform the procedure "Restoring the enhanced network to duplex operation" in Recovery Procedures () , 297-8021-545. Restore power to one or more units of a line group controller (LGC), line trunk controller (LTC), or digital trunk controller (DTC). Restore power if you turned off power to save emergency backup power. To restore power, perform the procedure "Restoring LGCs, LTCs, and DTCs to duplex operation" in Recovery Procedures () , 297-8021-545. Restore power to one or more units of a line concentrating module (LCM) if you turned off power to save emergency backup power. To restore power, perform the procedure "Restoring LCMs to duplex operation" in Recovery Procedures () , 297-8021-545. Restore power to one or more line modules (LM) if you turned off power to save emergency backup power. To restore power, perform the procedure "Restoring line modules to duplex operation" in Recovery Procedures () , 297-8021-545.

10

11

12

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

450

Disaster recovery maintenance

13

Restore power to one or more maintenance trunk modules (MTM) if you turned off power to save emergency backup power. To restore power, perform the procedure "Returning maintenance trunk modules to service" in Recovery Procedures () , 297-8021-545. To clear any alarms on the MAP display, perform the correct alarm clearing procedure. The correct alarm clearing procedure is in this document or in the Alarm and Performance Monitoring Procedures () , 297-8021-543.
--End--

14

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

Procedure steps

451

Performing a partial power down recovery of the Ethernet Routing Switch 8600
This procedure is used to perform a partial power down recovery of an Ethernet Routing Switch 8600.

Procedure steps
Step 1 Action At the chassis turn the DC input power source on, or reset the power source circuit breaker to provide power to the power supply. Turn the power supply switch to the on position.
--End--

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

452

Disaster recovery maintenance

Performing a partial power down recovery of the Media Gateway/Multiservice Switch 15000
This procedure is used to perform a partial power down recovery of a Media Gateway/Multiservice Switch 15000.

Procedure steps
Step 1 Action At the frame manually switch on the following breakers on the lower shelf:

A1.5 A1.2 B1.5 B1.2

Manually switch on the following breakers on the upper shelf:

A2.5 A2.2 B2.5 B2.2

The system should recover and come up on its own.


--End--

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

Variable definitions

453

Performing a partial power down recovery of the SPM


This procedure is used to perform a recovery of a partial power down of a DMS-SPM, MG 4000, IW-SPM, or DPT-SPM. In this procedure, SPM refers to the completed family of SPM devices unless otherwise noted.

Procedure steps
Step 1 Action At the MAP post the SPM to recover by typing:
>POST SPM <spm_no>

and pressing the Enter key. 2 3 Manually reseat the inactive CEM. Post, busy, and return the inactive CEM to service by typing:
>POST CEM <inactive_cem>; BSY; RTS

and pressing the Enter key. 4 5 Manually reseat each RM. Select, busy, and return each inactive resource module to service by typing:
>POST <rm_type> ALL; BSY ALL; RTS ALL

and pressing the Enter key. 6 Perform step 5 for each RM type.
--End--

Variable denitions
Variable rm_type spm_no Value is SRM, OC3, ATM, GEM, DSP, VSP, ALM, or DLC is the number of the SPM to be powered down

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

454

Disaster recovery maintenance

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

455

Disaster recovery power up


This chapter describes how to power up the disaster recovery procedures for the Nortel Carrier VoIP solutions. Navigation

Unlocking the Call Agent (page 456) Unlocking a GWC card (page 457)

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

456

Disaster recovery power up

Unlocking the Call Agent


Use this procedure to unlock the call agent.

Procedure steps
Step 1 Action At the Communication server 2000 SAM21 Manager client from the Shelf View, right click on the card and select Unlock from the context menu.

Unlock is also available from the States tab of the Card View window. The card resets, downloads software, and reboots. 2 Wait for the lock icon to disappear. Do not perform any patching activities on the Call Agent until ten minutes have passed.
--End--

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

Procedure steps

457

Unlocking a GWC card


Use this procedure to initiate a reboot of the GWC card causing the card to download its software from the Communication server 2000 Core Manager and to restart its call processing services and applications software. Use this procedure:

after replacing a GWC card as part of a fault clearing activity when a new software load is available when you complete reprovisioning a GWC card or GWC node (a node is made up of unit 0 and unit 1 GWC cards) and you want the card or node to use the new provisioning values when you applied or removed a patch to the GWC software using the Network Patch Manager (NPM) and saved a new GWC software image to the Communication server 2000 Core Manager

Prerequisites

The card must be locked. If IP addresses for the card that you want to unlock and its mate are not contiguous, you cannot unlock the card. You must correct these addresses using procedure "Manually re-provisioning GWC cards" in Nortel Gateway Controller Configuration ( (NN10205-511)) before attempting to unlock the card. If the GWC hardware does not support the service profile assigned to the GWC node, the unlock process fails and the system generates the CMT305 critical alarm. For the description of each profile and its hardware requirement, see Nortel Gateway Controller Configuration ( (NN10205-511)) .

Procedure steps
Step 1 Action At the Communication server 2000 GWC Manager client at the Communication server 2000 Management Tools window, click the Gateway Controller folder from the Device Types directory tree in the far left frame. From the Contents of: GatewayController frame, select the GWC node that contains the card you want to unlock.

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

458

Disaster recovery power up

Select the Maintenance tab to display maintenance information about the node.

Click the Card View button for the card you want to unlock. The Communication server 2000 SAM21 Manager opens. If a card is locked, all fields display the value <unknown>. At the Communication server 2000 SAM21 Manager in the card view, select the States tab, then click the Unlock button to unlock the card. If you want to display the status of all cards in the shelf, select Shelf View from the View menu.

If the IP addresses for the selected card and its mate are not contiguous, the system displays the following error message:

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

Procedure steps

459

Follow procedure "Manually re-provisioning GWC cards" in Nortel Gateway Controller Configuration ( (NN10205-511)) to correct these addresses, then repeat this procedure. If the GWC service profile and the GWC hardware are not compatible, the unlock operation fails and the system generates the CMT305 critical alarm. For the description of the alarm and the required action, see Nortel Carrier Voice over IP Fault Management Logs Reference Volume 2 ( (NN10275-909v2)) . 6 Observe the system response in the History window. The card is unlocked when you see the text Application unlocked successfully . 7 If required, repeat this procedure for the next GWC card until all the GWC cards have been unlocked and brought into service. Each GWC node has two cards.
--End--

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

460

Disaster recovery power up

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

461

Reference
The reference chapter describes information about the disaster recovery solutions.

Disaster recovery preparation


The section gives you referential information on how and when to have the data backup of the components.

Backup operations
There are two types of backup operations for components of CVoIP solutions: service data backup and fileset level backup. Perform service data backups once on each day during a period of low activity. Table 27 "Service data backup components" (page 461) lists components that are typically backed up using a service data backup.

ATTENTION

Point 1Service data from the SAM21, GWC, and MG 9000 is not backed up locally. This data is backed up at the manager. The manager is backed up to DVD using CS 2000 Core Manager or Core and Billing Manager (CBM) or CS 2000 Management Tools procedures. Point 2Service data from the IW SPM and MG 4000 is automatically backed up when the CS 2000 is backed up. Point 3Perform MDM backups by copying the files to tape or an off-product storage system through UNIX commands or CRON jobs. The script purgeTempData.sh in the /opt/nortel/iems/current/bin directory purges all event, alarm, and performance data from the IEMS database. After you purge the data you cannot retrieve it. The purge deletes all events, alarms, and performance data. To reduce the time taken to backup or restore, you must stop the IEMS server and execute the purgeTempData.sh script to purge the events, alarms, and performance data from the database.

Table 27 Service data backup components Component Procedure (s)

NETWORK INTELLIGENCE

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

462

Reference

Table 27 Service data backup components (contd.) Component CS 2000 Call Agent SAM21 GWC Session Server Policy Controller Ethernet Routing Switch 8600 UAS MS 2000 Series Procedure (s) Backing up an XA-Core office image from disk to tape (page 119) Backing up Call Agent (page 126) none (See "Point 1" (page 461) ) none (See "Point 1" (page 461) ) Backing up the SST (page 129) Performing a manual backup of the Policy Controller database (page 162) Saving the Ethernet Routing Switch 8600 boot configuration file (page 136) Backing up UAS configuration files (page 160) For the following MS 2000 Series procedures, see Nortel Media Server 2000 Series Configuration ( (NN10340-511)) Displaying the MS 2000 Series node configuration: [***Configuring Automated INI file backup on a Media Server 2000 node] [***Changing the SNMP community string passwords] [***Changing the SNMP community string password for a Media Server 2020 node] Backing up all MS 2000 Series node INI files: [***Backing up INI files for all nodes] Configuring the MS 2000 Series CLUI tool: [***Displaying Media Server 2010 node current configuration] [***Displaying Media Server 2020 node current configuration] Backing up the APS-specific Oracle database and application files (page 150)

APS USP Real-time Transport Protocol (RTP) Media Portal CICM CORE NETWORK Multiservice Switch or Media Gateway devices GATEWAYS MG 15000 MG 9000 MG 4000 IW SPM

RTP Media Portal Basics () (NN10367-111)

CICM Security and Administration () (NN10252-611)

Restoring Multiservice Switch or Media Gateway service data (page 23)

none (See "Point 1" (page 461) ) none (See "Point 2" (page 461) ) none (See "Point 2" (page 461) )

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

Disaster recovery preparation

463

Table 27 Service data backup components (contd.) Component Procedure (s)

NETWORK MANAGEMENT CS 2000 Core Manager Core Element Manager CS 2000 Management Tools Starting or stopping the automated synchronous backup restore manager service in Nortel ATM/IP Solution-level Administration and Security ( (NN10402-600)) Starting or stopping the automated synchronous backup restore manager service (page 107)Performing a backup of Oracle/MySQL data on an SPFS-based server (page 156) Performing a backup of Oracle/MySQL data on an SPFS-based server (page 156) Performing a backup of file systems on an SPFS-based server (page 164) Starting or stopping the automated synchronous backup restore manager service in Nortel ATM/IP Solution-level Administration and Security ( (NN10402-600)) Starting or stopping the automated synchronous backup restore manager service (page 107) Performing a backup of Oracle/MySQL data on an SPFS-based server (page 156) IEMS centralized security server MG 9000 Manager Taking a full IEMS backup (page 169) Performing a backup of file systems on an SPFS-based server (page 164) Starting or stopping the automated synchronous backup restore manager service (page 107) MDM Backing up an SPFS-based security client Performing a backup of oracle data on an SPFS-based server Performing a backup of file systems on an SPFS-based server Starting or stopping the automated synchronous backup restore manager service

Integrated Elem ent Management System

Fileset level backups back up the software and configuration data. Perform this type of backup after hardware changes, software updates or patches, or major configurations. You must perform a fileset backup before a major upgrade. Table 28 "Fileset level backup components" (page 463) provides a list of components with fileset level backup procedures.
Table 28 Fileset level backup components Component Network intelligence Procedures

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

464

Reference

Table 28 Fileset level backup components (contd.) Component GWC USP Gateways Multiservice Switch or Media Gateway devices Network management CS 2000 Core Manager orCore and Billing Manager (CBM) Creating system image backup tapes (S-tapes) manually Configuring SBA backup volumes on the core Starting or stopping the automated synchronous backup restore manager service. Performing a backup of file systems on an SPFS-based server. Configuring SBA backup volumes on a billing stream. Performing a backup of file systems on an SPFS-based server (page 164) Configuring SBA backup volumes on the core Restoring Multiservice Switch or Media Gateway service data (page 23) Procedures Create a backup of the GWC load file in Nortel Carrier Voice over IP Upgrade and Patches ( (NN10440-450)) . Administration: USP Manual Backup

Frequency of backup operations


Table 29 "Component backup frequency" (page 464) lists each component along with how often Nortel recommends that you perform backup operations.

ATTENTION

Point 1Service data from the SAM21, GWC, and MG 9000 is not backed up locally. The data is backed up at the manager. The manager is backed up to DVD using CS 2000 Core Manager or Core and Billing Manager (CBM) or CS 2000 Management Tools procedures. Point 2Service data from the IW SPM and MG 4000 is automatically backed up when the CS 2000 is backed up.

Table 29 Component backup frequency Component Network Intelligence CS 2000MSC Server 1000 XA-Core Call Agent Weekly Backup Frequency

Weekly

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

Disaster recovery preparation

465

Table 29 Component backup frequency (contd.) Component SAM21 GWC Session Server CS 2000 CS LAN UAS MS 2000 Series APS RTP Media Portal CICM USP Core network Multiservice switches Gateways MG 9000 MG 4000 IW SPM See "Point 1" (page 464) See "Point 2" (page 464) See "Point 2" (page 464) Full backup - monthly, and prior to all migrations and patch operations Backup Frequency See "Point 1" (page 464) See "Point 1" (page 464) Weekly Full backup - monthly, and prior to all migrations and patch operations and configuration changes Daily Daily Daily Daily Daily Full system backup - once a week. Modified (differential) backup - once a day

Network management CS 2000 Management Tools MG 9000 Manager IEMS Nortel MDM Daily

Daily Daily Daily

Storage and maintenance of CDs and DVDs


Follow these guidelines for the proper storage and maintenance of CDs and DVDs:

Store CDs and DVDs at a temperature between 5 and 25 C and a humidity level between 10% and 80%. Do not touch the surface of a CD or DVD with your hands and do not mark the surface with a pencil, pen, or marker. Use a cotton cloth or disk cleaning kit to clean CDs and DVDs.

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

466

Reference

ATTENTION
If checksum errors occur, perform a visible inspection of the disk to check for scratches or damage. If no scratches are visible, clean the disk using a cotton cloth or disk cleaning kit and retry.

Operational status of the NCGL platform


Table 30 "System information page fields" (page 466) lists information about System information page fields.
Table 30 System information page fields Field Unit Activity Jam Description Unit number in the node that you are logged on to. Activity of the unit (either active or standby). Indicates if the inactive unit is Jammed. The value is yes only if you are logged on to the inactive unit. From the active unit, the status is no, but a JInact alarm indicates the inactive is Jammed. Indicates if the node is operating in duplex (fault-tolerant) mode or simplex mode (the standby unit is off-line). State of the network links on the node. Name of the unit (not node). System date as maintained by the network time protocol (NTP) server. Indicates the amount of time that elapsed since the unit was last rebooted. Recorded system time that the unit has been powered up. Indicates whether the unit is currently booted from the hard drive or DVD-ROM drive. Indicates any event that forced a platform reboot (manual or system generated). Indicates any event that affected the power supply subsystem of the unit chassis. Installed version of the NCGL platform software. Unit IP address. IP address of the client web browser. When a web proxy is used, the IP address of the machine performing the proxy is displayed.

State

Connectivity Host Name Date Time since last reboot System Power-On Time System booted from Last restart cause Last power event cause Current version Platform IP Address Platform EM Client IP Address

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

Operational status of the NCGL platform

467

Table 30 System information page fields (contd.) Field Server Location Host Name Description Physical location of the unit. Name of the unit.

Table 31 "Node maintenance page fields" (page 467) lists information about Node maintenance page fields.
Table 31 Node maintenance page fields Field Operation State Activity Jam State Maintenance Actions (active unit only) Description Indicates the operational state of the NCGL software. Indicates the activity state of the platform software. Indicates if the inactive unit is Jammed. Maintenance page for performing SWACT and to Jam. See the maintenance and manual cold SWACT procedures in this document.

Table 32 "Chassis information page fields" (page 467) lists information about Chassis information page fields.
Table 32 Chassis information page fields Field Chassis information: Self Test Chassis information: Chassis Subsystems Description Status of the self test performed on the platform at bootup. Status of the platform hardware subsystems including the memory, CPU, all drives, network connection, power supplies, cooling and other I/O connections. Indicates the 1, 5 and 15 minute load averages for the CPU utilization. Indicates the 1 minute CPU load average utilization threshold value. When the set threshold value is exceeded, the appropriate minor, major or critical alarm is raised. Indicates the 5, 20, and 30 minute CPU utilization average. When the threshold value is exceeded, an alarm is raised.

CPU Information: load average CPU Information: load average threshold values

Chassis Utilization: Utilization average

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

468

Reference

Table 32 Chassis information page fields (contd.) Field Chassis Utilization: alarm threshold values Description Indicates the 5, 20, and 30 minute CPU utilization average threshold value. When the set threshold value is exceeded, an alarm is raised. Total number of processes (non-threaded) that are running on the SST Platform. Number of defunct or terminated NCGL zombie processes.

Process Information: Number of Processes Process Information: Number of zombie processes

A zombie process is a process that has terminated either because it has been killed by a signal or because it has called an exit() and whose parent process has not yet received notification of its termination. A zombie process exists solely as a process table entry and consumes no other resources. Process Information-zombie: minor alarm threshold value Maximum number of zombie processes allowed to be run by the CPU before a minor alarm is raised indicating that the set threshold has been exceeded. Maximum number of zombie processes allowed to be run by the CPU before a major alarm is raised indicating that the set threshold has been exceeded. Maximum number of zombie processes allowed to be run by the CPU before a critical alarm is raised indicating that the set threshold has been exceeded. Total amount of RAM installed on the mother board of each SST unit. Both units must have the same amount. Amount of memory available unallocated for use. Amount of memory available for programs. Indicates the threshold amount of available memory (in Mbytes) that the system must drop below before a minor alarm is raised. Indicates the threshold amount of available memory (in Mbytes) that the system must drop below before a major alarm is raised. Indicates the threshold amount of available memory (in Mbytes) that the system must drop below before a critical alarm is raised.

Process Information-zombie: major alarm threshold value

Process Information-zombie: critical alarm threshold value

Memory Information: Total Memory (MB) Memory Information: Free Memory (MB) Memory Information: Available memory (MB) Memory Information: minor alarm threshold value Memory Information: major alarm threshold value Memory Information: critical alarm threshold value

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

Operational status of the NCGL platform

469

Table 33 "Network connectivity page fields" (page 469) lists information about Network connectivity page fields.
Table 33 Network connectivity page fields Field Unit 0,1 Links Unit 0,1 Status Unit 0,1 Activity Unit 0,1 Maintenance Description Indicates which Ethernet IP links are installed on the units (each unit has two links). Status of the Ethernet links. Activity status of the Ethernet links, either active or inactive. Indicates the maintenance actions that can be performed on the Ethernet links, either Lock, Unlock or Swlink. Status of the PTP links between both units in the node. Network IP address of the SST unit. IP address of the local (active) SST unit. IP address of the mate (inactive) SST unit. IP address of the active or inactive Ethernet port 0. IP address of the active or inactive Ethernet port 1. IP address of the active or inactive PTP link.

Unit 0,1 PTP Links status Unit IP Active IP Inactive IP Port 0 IP Port 1 IP PTP IP

Table 34 "Disk Services page fields" (page 469) lists information about Disk Services page fields.
Table 34 Disk Services page fields Field RAID Array Status: Name RAID Array Status: Size (GB) RAID Array Status: State Description Indicates the name of each RAID-1 array in the system. Indicates the size of the partition in Gigabytes. Indicates a high level state for the array: - ".": indicates the array is functioning normally. - Missing: a disk was removed from the array. - Failed: a disk in the array has failed and needs to be replaced. - Rebuilding: the array is in the process of rebuilding to a fault-tolerant mode.

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

470

Reference

Table 34 Disk Services page fields (contd.) Field RAID Array Status: Disk 0 RAID Array Status: Disk 1 RAID Array Status: Status Description Service status of disk 0. Service status of disk 1. Indicates the status of the array. Values are: - The array is operating normally - Missing - Failed - Rebuild Indicates the disk number in the array, 0, or 1. Total capacity of the disk drive in Gigabytes. Installation state of the disk. Indicates whether a hard disk can be inserted into the RAID array. Indicates the status of individual filesystems on the disk array. Indicates the name of the filesystem on the disk array. Some filesystem names are reserved. Indicates the results of any tests run on the filesystems. Tests are run approximately every 10 minutes to verify that all of the basic filesystem operations are working on each of the filesystems. Total amount of disk space (in MB) allocated for this filesystem. Total amount of disk space (in MB) in use on this file system. Total amount of disk space (in %) in use on this file system. Percentage of total disk space (in MB) free for use on this filesystem. Amount of disk space (in %) available for use by platform processes and applications. maximum amount of disk space (in %) that can be used before a minor alarm is raised indicating that the set threshold has been exceeded.

Disk Maintenance: Disk Number Disk Maintenance: Disk Size (GB) Disk Maintenance: Disk State Disk Maintenance: Disk Action Filesystem Information: Monitor Filesystem Information: Filesystem Name Filesystem Information: Test Results

Filesystem Information: Total Space (MB) Filesystem Information: Total Space Used (MB) Filesystem Information: Total Space Used (%) Filesystem Information: Total Space Available (MB) Filesystem Information: Total Space Available (%) Filesystem Information: Minor Alarm Threshold (%)

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

Operational status of the NCGL platform

471

Table 34 Disk Services page fields (contd.) Field Filesystem Information: Major Alarm Threshold (%) Description Maximum amount of disk space (in %) that can be used before a major alarm is raised indicating that the set threshold has been exceeded. Maximum amount of disk space (in %) that can be used before a critical alarm is raised indicating that the set threshold has been exceeded. Name of the volume group in the array. Total size of the volume group in the array. Amount of volume group space, in Gigabytes, currently allocated to filesystems. Amount of volume group space (in %) currently allocated to filesystems. Amount of unallocated volume group space, in Gigabytes, available for filesystems. Amount of unallocated volume group space (in %) available for filesystems.

Filesystem Information: Critical Alarm Threshold (%)

Volume Group Information: Volume Group Name Volume Group Information: Volume Group Size (GB) Volume Group Information: Total Space Allocated (GB) Volume Group Information: Total Space Allocated (%) Volume Group Information: Total Space Available (GB) Volume Group Information: Total Space Available (%)

Table 35 "Services page fields" (page 471) lists information about Services page fields.
Table 35 Services page fields Field Network Services: Number of Active Command Line Sessions Network Services: Number of Clients with Active Web Sessions NTP Information: Server1 - Server 3 NTP Information: Total Number of Servers Description Number of command line interface (CLI) sessions (both remote and local) on the unit Number of clients running one or more web GUI sessions IP address of up to three Network Time Protocol (NTP) servers in the network, and the status of the connection Number of NTP servers registered with the CS-LAN network

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

472

Reference

Table 35 Services page fields (contd.) Field NTP Information: Accessible Servers NTP Information: Synchronized Servers Description Number of NTP servers accessible to the SST Number of NTP servers to which the unit is synchronized

Table 36 "Administration page fields" (page 472) lists information about Administration page fields.
Table 36 Administration page fields Field Bootload Management: Bootload Bootload Management: Maintenance Description Load ID for the NCGL platform software load Indicates if the Bootload is the default. You can choose a new default bootload if there is more than one load available. Additional loads can come from maintenance releases. File transfer protocol or source location for the platform software upgrade: FTP, Anonymous FTP, HTTP, HTTPS, Local File, or Local CDROM If a logon ID is required to access the upgrade platform load from another server in the network, a logon ID can be entered here. If a password is required to access the upgrade platform load from another server in the network, a password can be entered here. If it is required to access the upgrade platform load from another server in the network, an IP address can be entered here. The target upgrade load path and filename is entered here. The Upgrade button initiates a platform NCGL upgrade. Used to execute the Reboot, Halt, Rebootmate, and Haltmate functions. These are service affecting commands.

Software Upgrade: Protocol

Software Upgrade: Login ID

Software Upgrade: Password

Software Upgrade: IP Address

Software Upgrade: File Software Upgrade: ActionUpgrade button Server Maintenance (active and inactive units)

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

Nsbck command line options

473

Device audits
After a dead office recovery, several data mismatches may be present in the network. An audit must be performed to clear up these data mismatches. Table 37 "Device audit procedures" (page 473) lists the audit procedures available for each component and where those procedures are located.
Table 37 Device audit procedures Component CS 2000 Core Manager or Core and Billing Manager GWC Procedures available Performing a system audit Location Nortel CS 2000 Core Manager Fault Management ( (NN10082-911))

Perform a GWC line data integrity audit Perform a GWC trunk data integrity audit Perform a CS 2000 data integrity audit Perform a GWC V5.2 data integrity audit Performing an audit of the MG 9000 provisioning data

Nortel Gateway Controller Fault Management ( (NN10202-911))

MG 9000

Nortel MG 9000 Configuration ( (NN10096-511))

Nsbck command line options


To display online help for this command, use the -h option on the command line. Step 1 Action Enter the following command as one continuous command: /opt/MagellanNMS/bin/nsbck [-full <devtype> <devname> <devaddr> <id> <pw>] [-incr <devtype> <devname> <devaddr> <id> <pw>] [-view <viewname> <devtype> <devname> <devaddr> <id> <pw>] [-cdir <backup_dir_path> [-f <backup_info_file] [-chost <controller_address>] [-nolog] [-log <logfile>]] [-nc <#_concurrent_connections>] where -full indicates a full backup -incr indicates an incremental backup. -view indicates a selective backup. devtype is the name of the device type, such as PASSPORT. devname is the name of the device.
Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

474

Reference

devaddr is the IP address of the device and has the format n.n.n.n. id is the userID for a Passport and the READ community string for a Passport 4400/4460. pw is the user password for a Passport and the WRITE community string for a Passport 4400/4460. viewname is the name of the view file for a selective backup. -cdir is the backup directory used for this backup operation. -f indicates that the backup information is obtained from a file. backup_info_file is the name of the file containing backup information. Each line in the file has the same format as the -full, incr, or -view options. -chost indicates the remote Controller to be used in place of the Controller running on the local host. controller_address is the address of the Controller and has the format host[:port] -nolog indicates that output messages are discarded. -log indicates that output messages go to stdout/stderr or to a log file. The default is stdout/stderr. logfile is the name of the file to which output messages go. If not specified, the messages go to the file mbrbackup.log in the current directory (where the tool runs). -nc indicates the number of concurrent backups to be performed. By default, SNMP Devices Backup tries to back up 5 different devices concurrently. This parameter is useful when you are backing up a large number of devices. #_concurrent_connections is the number of concurrent backups.
--End--

Backup le naming convention


The service data backup files have the following naming convention:

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

Backup file naming convention

475

./<devtype>/<devname>/timestamp>.<dataset>/\<datafiles. ..>

Note: The period (.) represents the SNMP Devices Backup and
Restore root directory.

where devtype is the device type (PP4400, or PP4460). devname is the device name timestamp has the format yyyymmddhhmmss dataset is the dataset name. datafiles... are the names of one or more files that are backed up. Note: The period (.) represents the SNMP Devices Backup and
Restore root directory.

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

476

Reference

Carrier VoIP Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures NN10450-900 06.02 3 October 2008
Copyright 2008 Nortel Networks

Carrier VoIP

Nortel Carrier Voice over IP Solutions Disaster Recovery Procedures


Copyright 2008 Nortel Networks All Rights Reserved. Printed in Canada, the United States of America, and the United Kingdom Release: (I)CVM12 Publication: NN10450-900 Document revision: 06.02 Document release date: 3 October 2008 To provide feedback or to report a problem in this document, go to www.nortel.com/documentfeedback. www.nortel.com LEGAL NOTICE While the information in this document is believed to be accurate and reliable, except as otherwise expressly agreed to in writing NORTEL PROVIDES THIS DOCUMENT "AS IS" WITHOUT WARRANTY OR CONDITION OF ANY KIND, EITHER EXPRESS OR IMPLIED. The information and/or products described in this document are subject to change without notice. Nortel, the Nortel logo, Business Made Simple and the Globemark are trademarks of Nortel. All other trademarks are the property of their respective owners.

Das könnte Ihnen auch gefallen