Sie sind auf Seite 1von 122

1 X E V- D O

R F P E R F O R M A N C E A N A LY S I S
&
TROUBLESHOOTING
GUIDE

R EL 28.0

RF T OOLS & S ERVICES S YSTEMS


E NGINEERING GROUP

ALL RIGHTS RESERVED ALCATEL-LUCENT 2007 FOR INTERNAL USE ONLY


USE PURSUANT TO COMPANY INSTRUCTIONS
TA B L E O F C O N T E N T S

0. CHANGE HISTORY.............................................................................................................................. 8

1. INTRODUCTION................................................................................................................................. 14

2. USING THIS GUIDE ........................................................................................................................... 16

3. OVERVIEW OF 1XEV-DO SYSTEM AND OPERATION............................................................. 18


3.1 SYSTEM FUNCTIONAL BLOCKS ..............................................................................................18
3.1.1 Access Network ........................................................................................................... 19
3.1.2 Access Terminal .......................................................................................................... 22
3.2 1XEV-DO SYSTEM OPERATION ...........................................................................................22
3.2.1 Forward link................................................................................................................ 23
3.2.1.1 Rel 0 Forward Link ....................................................................................................... 23
3.2.1.2 Rev A Forward Link ..................................................................................................... 24
3.2.2 Reverse Link................................................................................................................ 24
3.2.2.1 Rel 0 Reverse Link........................................................................................................ 24
3.2.2.2 Rev A changes .............................................................................................................. 25
4. SUMMARY OF PERFORMANCE RELATED FEATURES IN PRODUCT RELEASES .......... 26
4.1 PERFORMANCE RELATED FEATURES IN R27.0...................................................................26
4.2 PERFORMANCE RELATED FEATURES IN R28.0...................................................................27
4.3 PERFORMANCE RELATED FEATURES SCHEDULED IN R29.0 ............................................28
5. KPI OPTIMIZATION AND TROUBLESHOOTING ...................................................................... 31
5.1 ESTABLISHED CONNECTION RATE ......................................................................................32
5.1.1 Call setup process ....................................................................................................... 34
5.1.1.1 Multiple Carrier Support............................................................................................... 37
5.1.1.2 Paging Mechanisms ...................................................................................................... 38
5.1.1.3 Fast Connect Calls ........................................................................................................ 38
5.1.2 RF access failures ....................................................................................................... 39
5.1.2.1 Lack of RF coverage ..................................................................................................... 41
5.1.2.2 Excessive Pilots/Non-dominant Pilots .......................................................................... 42
5.1.2.3 Poorly constructed Neighbor Lists................................................................................ 43
5.1.2.4 Improper PN planning................................................................................................... 44
5.1.2.5 External Interference..................................................................................................... 46
5.1.2.6 Inadequate Search Window Size................................................................................... 47
5.1.2.7 Edge of coverage........................................................................................................... 49
5.1.2.8 RNC border issues ........................................................................................................ 50
5.1.3 Blocking ...................................................................................................................... 51
5.1.3.1 TP overload................................................................................................................... 52
5.1.3.2 Maximum number of Connections or UATI Sessions reached..................................... 52
5.1.3.3 High Processor Occupancy rates................................................................................... 54
5.1.4 Failures due to faulty hardware.................................................................................. 56
5.1.5 Heavy Reverse Link traffic .......................................................................................... 56
5.1.6 Traffic Channel Activation failure .............................................................................. 59
1 X EV-DO RF P ER FORM ANCE A NALYS IS & T ROUB LESHOOT ING G U IDE

5.1.7 Authentication error.................................................................................................... 59


5.2 DROPPED CONNECTION RATE ..............................................................................................62
5.2.1 Inter Frequency Handoffs ........................................................................................... 64
5.2.2 Lack of RF coverage ................................................................................................... 65
5.2.3 Excessive pilots/Non-dominant pilots ......................................................................... 66
5.2.4 Poorly constructed neighbor lists ............................................................................... 68
5.2.5 Improper PN planning ................................................................................................ 69
5.2.6 External interference................................................................................................... 70
5.2.7 Insufficient search window size................................................................................... 72
5.2.8 Failures due to faulty hardware.................................................................................. 73
5.2.9 Heavy reverse link traffic ............................................................................................ 74
5.2.10 Maximum number of legs reached .............................................................................. 76
5.2.11 No resources at the candidate leg............................................................................... 77
5.2.12 High Processor Occupancy rates................................................................................ 78
5.2.13 Handoff failures .......................................................................................................... 80
5.2.14 Reverse link lost due to hybrid tuneaways .................................................................. 81
5.3 DATA RATE .............................................................................................................................82
5.3.1 Lack of RF coverage ................................................................................................... 84
5.3.2 Excessive pilots/Non-dominant pilots ......................................................................... 85
5.3.3 Poorly constructed neighbor lists ............................................................................... 87
5.3.4 Improper PN planning ................................................................................................ 88
5.3.5 External interference................................................................................................... 90
5.3.6 Insufficient search window size................................................................................... 91
5.3.7 Failures due to faulty hardware.................................................................................. 93
5.3.8 Heavy reverse link traffic ............................................................................................ 93
5.3.9 High forward link traffic ............................................................................................. 96
5.3.10 Hybrid mode................................................................................................................ 97
5.3.11 Backhaul restrictions .................................................................................................. 98
5.3.12 Handoff rejects .......................................................................................................... 100
5.3.13 Core IP network issues.............................................................................................. 102
APPENDIX A METRIC CROSS-REFERENCE ............................................................................ 103

APPENDIX B OVERVIEW OF SPO TOOL .................................................................................. 106

APPENDIX C OVERVIEW OF PDM TOOL................................................................................. 117

APPENDIX D RECENT PERFORMANCE AFFECTING ALU ALERTS ................................. 121

ALL RIGHTS RESERVED ALCATEL-LUCENT 2007 3


1 X EV-DO RF P ER FORM ANCE A NALYS IS & T ROUB LESHOOT ING G U IDE

LIST OF FIGURES
FIGURE 1 FUNCTIONAL BLOCKS IN A STANDALONE ALCATEL-LUCENT 1XEV-DO SYSTEM........ 18
FIGURE 2 FUNCTIONAL BLOCKS IN A ALCATEL-LUCENT 1XEV-DO OVERLAY ON A ALCATEL-
LUCENT 3G1X HIGH SPEED DATA NETWORK ....................................................................... 19
FIGURE 3 PROTOCOL STACK DIAGRAM FOR ALCATEL-LUCENT 1XEV-DO SYSTEM [9] .............. 22
FIGURE 4 CALL FLOW FOR AN AT INITIATED CONNECTION .......................................................... 36

APPENDIX FIGURE 1 EXECUTIVE SUMMARY ............................................................................... 107


APPENDIX FIGURE 2 SM SUMMARY WINDOW AND PEG COUNT DESCRIPTION .......................... 107
APPENDIX FIGURE 3 SM MAP ..................................................................................................... 108
APPENDIX FIGURE 4 SM TREND .................................................................................................. 109
APPENDIX FIGURE 5 ROP REPORTS CP FAILS AND ALARMS ANALYSIS .................................. 110

L I S T O F TA B L E S
TABLE A-1 ESTABLISHED CONNECTION RATE METRIC CROSS REFERENCE .................... 103
TABLE A-2 DROPPED CONNECTION RATE METRIC CROSS REFERENCE ........................... 104
TABLE A-3 DATA RATE METRIC CROSS REFERENCE ....................................................... 104

ALL RIGHTS RESERVED ALCATEL-LUCENT 2007 4


1 X EV-DO RF P ER FORM ANCE A NALYS IS & T ROUB LESHOOT ING G U IDE

List Of Abbreviations
2G Second Generation (IS95A/IS95B)
3G1x Third Generation 1.25MHz (IS2000)
3GPP2 Third Generation Partnership Project 2

A10 Interface that carries user traffic between PCF and PDSN
A11 Interface that carries signaling between PCF and PDSN
A12 Interface that carries signaling (authentication) between PCF and AAA
A13 Interface that carries signaling between source and target PCF
AAA Authentication, Authorization and Accounting
ACK Acknowledgement
AHO Access Handoff
AN Access Network
AP Application Processor
APHO Access Probe Handoff
AR Assistance Request
AT Access Terminal

BSOC Base Station OA&M Controller


BTS Base Transceiver Station

CAMSHO Channel Assignment into Soft Handoff


CBR CDMA Baseband Radio
CC Control Channel
CDM CDMA Digital Module
CLGC Closed Loop Gain Control
CRC CDMA Radio Controller
CTU Common Timing Unit

DFDP Data Filtering and Data Packaging


DQoS Data Quality of Service
DRC Data Rate Channel
DSM Dedicated SPO Module

EMS Element Management System


EV-DO Evolution Data Optimized
EV-DO Rev 0 Evolution Data Optimized Revision 0
EV-DO Rev A Evolution Data Optimized Revision A
EVM EV-DO Modem

FCA Failed Connection Attempts


FER Frame Error Rate
FLM Forward Link Modem
FMS Flexent Mobility Server
FTP File Transfer Protocol

GPS Global Positioning System

ALL RIGHTS RESERVED ALCATEL-LUCENT 2007 5


1 X EV-DO RF P ER FORM ANCE A NALYS IS & T ROUB LESHOOT ING G U IDE

GRE Generic Routing Encapsulation

HCS HDR Cell Site


HDLC High Level Data Link Control
HDR High Data Rate
HOM Handoff Matrix
HRPD High Rate Packet Data

ID Identification
IM Inter Modulation
IP Internet Protocol

KPI Key Performance Indicator

LCP Link Control Protocol


LIU Line Interface Unit
LWS Lucent Worldwide Services

MAC Medium Access Control


MCC Main CDMA Controller
MCR Multi Carrier Radio
MIP Mobile Internet Protocol
MR Modification Request

NAK/NACK Negative Acknowledgement


NVM Non Volatile Memory

OA&M Operations, Administration & Maintenance


OFS Off Frequency Search
OHM Overhead Manager
OMC-RAN Operations and Maintenance Center Radio Access Network
OMP Operations and Management Platform
OOS Out of Service

PCF Packet Control Function


PDA Personal Digital Assistant
PDM Packet Data Monitoring
PDSN Packet Data Serving Node
PPP Point-to-Point Protocol
PSK Phase Shift Keying
PSMM Pilot Strength Measurement Message
PTT Push-to-Talk

QAM Quadrature Amplitude Modulation


QoS Quality of Service
QPSK Quadrature Phase Shift Keying

RAB Reverse Activity Bit


RATI Random Access Terminal Identifier

ALL RIGHTS RESERVED ALCATEL-LUCENT 2007 6


1 X EV-DO RF P ER FORM ANCE A NALYS IS & T ROUB LESHOOT ING G U IDE

RF Radio Frequency
RLM Reverse Link Modem
RLP Radio Link Protocol
RNC Radio Network Controller
ROP Read Only Printer
RRI Reverse Rate Indicator
RSSI Receive Signal Strength Indicator

SBEVM Single Board EV-DO Modem


SCI Slot Cycle Index
SM Service Measurements
SNR Signal to Noise Ratio
SNMP Simple Network Management Protocol
SIP Simple Internet Protocol
SPAT3G System Performance Analysis Tool 3G
SPO System Performance Optimization
SU Software Update

TCA Traffic Channel Assignment


TCC Traffic Channel Complete
TCP Transmission Control Protocol
TDU Time Delay Unit
TP Traffic Processor

UATI Unicast Access Terminal Identifier


UCR UMTS/CDMA Radio
UDP User Datagram Protocol
UNL Undeclared Neighbor List
URC Universal Radio Controller

VoIP Voice over Internet Protocol

ALL RIGHTS RESERVED ALCATEL-LUCENT 2007 7


1 X EV-DO RF P ER FORM ANCE A NALYS IS & T ROUB LESHOOT ING G U IDE

0. CHANGE HISTORY

Version Changes Date


R21 Initial Release October 2003

R22 Updates reflecting new service measurement counts and metrics, reference October 2004
web sites, metrics cross-reference table and SPAT3G information for
Release 22.

R23 Updates reflecting new service measurement counts and metrics for Release February 2005
23. Updates include:

Updated the guide with information about the new reverse link
overload control translations and the corresponding peg counts
Introduced new chapter summarizing performance related features in
R23 and R24.
Added new information to the hybrid mode section about SCI and
MSM6500 chipset performance.
Added a new subsection on high processor occupancy rates
Added a link for EV-DO release letters
Updated appendix B with information about EV-DO specific features
in SPAT3G
Updated acronyms list, references and the metrics cross-reference table
R24 Updates reflecting new service measurements counts and metrics for August 2005
Release 24.

Added information about Celnet Xplorer in the introduction section


Updated section 4 with performance related features in R24 and 25
Removed negotiation failures from the Established Connection Rate
equation
Modified equation for Fast Connect Failure rate
Added section 5.1.2.7 discussing connection failure rate issues in edge-
of-coverage areas
Added section 5.1.2.8 discussing connection failure rate issues in inter-
RNC border areas
Updated section 5.1.6 with details from Alcatel-Lucent Alert 05-0493
(peg count decrementing issue)
Updated equation for forward throughput in section 5.3
Added section 5.3.13 discussing core IP network issues affecting data
throughput
Updated appendix B (SPAT3G overview) with screenshots for ROP
call processing failure analysis
Updated table of contents, references and acronyms list
Updated metrics cross reference list
R25 Updates reflecting new service measurements counts and metrics for March 2006
Release 25
Added information about Handoff Matrix, Undeclared Neighbor list
and Multiple carriers in the introduction section
Updated section 4 with performance related features in releases 25 and

ALL RIGHTS RESERVED ALCATEL-LUCENT 2007 8


1 X EV-DO RF P ER FORM ANCE A NALYS IS & T ROUB LESHOOT ING G U IDE

26
Updated the equation for the Established Connection Rate
Updated the metric name Established Call Rate to Established
Connection Rate
Added information about 1xEV-DO cell processor overload counts
Updated appendix B (SPAT3G overview) with screenshots for Handoff
Matrix and Undeclared Neighbor List analysis
Updated table of contents, references and acronyms list
Updated metrics cross reference list

R26 Updates reflecting new service measurements counts and metrics for September 2006
Release 26
Updated section 4 with performance related features in releases 26 and
27
Added information about reverse link lost due to tuneaways
Added information about new SM counts for critical system edge
metric preservation

R27 Updates reflecting new service measurements counts and metrics for December 2006
Release 27
Added information on service measurements data for all six sectors by
default
Changed Lucent entries to Alcatel-Lucent where relevant
Updated section 4 with performance related features in product releases
27 and 28
Added information about LUNAR availability for service providers
Updated acronyms list and metrics cross-reference table
Added information on SM changes in R27 SU1 and Soft/Softer handoff
peg count issues
Added information on PCMD analysis is SPAT3G in Appendix B
R28 Updates reflecting new service measurements counts and metrics for October 2007
Release 28
Added information on Release 29 performance affecting features,
removed information on Release 26 performance affecting features
Updated the References Section including information on release letters
Updated and corrected the List of Abbreviations
Added basic Rev A information
Added information on 1xEV-DO cell OA&M convergence
Added information on new R28 peg counts
Updated information on Established Connection Rate metric, including
changes to old formula and new Established Connection Rate [peg] and
added information on the new User Established Connection Rate
Updated information on Fast Connect Success Rate
Updated information on the RF Failure Rate metric to include the catch
all count for post TCA failures
Added RF Failure Rate metric definitions as well as Connection
Blocking metrics
Re-arranged Established Connection Rate topics to reflect new metric
classes
Added more information on the Session Close Critical System Edge
Metric Preservation feature
Added information on the Prior Session: Allow session transfer from/to

ALL RIGHTS RESERVED ALCATEL-LUCENT 2007 9


1 X EV-DO RF P ER FORM ANCE A NALYS IS & T ROUB LESHOOT ING G U IDE

same RNC
Added information on paging mechanisms and some related counts
Added information on maximum number of UATI sessions reached
issues and related features
Changed Drop Call Rate references to Drop Connection Rate
Added basic information on Inter Frequency Handoffs and their effect
on Hard Handoff Failure Rates
Updated RLP Forward and Reverse Link Data Rate metrics added
information on per user RLP data rate metrics
Updated Metric Cross Reference Table with new metrics
Updated SPAT3G section with references to SPO
Added information on the PDM Tool
Added a list of recent performance affecting ALU Alerts

ALL RIGHTS RESERVED ALCATEL-LUCENT 2007 10


1 X EV-DO RF P ER FORM ANCE A NALYS IS & T ROUB LESHOOT ING G U IDE

References
[1] 1xEV-DO RF Translation Application Note # 0 Introduction
[2] 1xEV-DO RF Translation Application Note # 1 Timing, Delay and Access
Parameters
[3] 1xEV-DO RF Translation Application Note # 2 Forward link
[4] 1xEV-DO RF Translation Application Note # 3a Reverse Power control
[5] 1xEV-DO RF Translation Application Note # 3b Reverse Overload control
[6] 1xEV-DO RF Translation Application Note # 4 Handoff
[7] 1xEV-DO RF Translation Application Note # 5 High level Protocol Stack Parameter
Optimization
[8] 1xEV-DO RF Optimization Guidelines
[9] SRD 1853 1x EV-DO Packet Data Call Processing
[10] SRD 1880 OA&M Requirements for the Network (MSC) portion of a 1xEV-DO
System
[11] 1XEV-DO and 3G1X Service Measurement Counts Cross Reference Memo
12/07/02 ( http://nwswww.wh.Alcatel-Lucent.com/fjiang/1XEVSMcrossRef.doc )
[12] 1xEV-DO Troubleshooting Guide ( http://nwswww.ih.Alcatel-
Lucent.com/apxlib/cgi-bin/srchdoc.cgi?entry=d26453+ )
[13] 1xEV-DO Performance Expectations document
[14] White Paper on Base Station and Network Configuration for 1xEV-DO July 30, 2003
(http://nwswww.wh.Alcatel-Lucent.com/yyang7/hdr/do_wp_2.3.pdf)
[15] CDMA RF Performance Analysis and Troubleshooting Guide
[16] Flexent Wireless Networks CDMA2000 1x-EVDO Radio Network Controller
Operations, Administration, and Maintenance (OA&M), 401-614-102
[17] 1xEV-DO RF Translation Application Note # 6 Flexible Scheduler
[18] Alcatel-Lucent 1xEV-DO System Architecture, http://nwswww.wh.Alcatel-
Lucent.com/gaby/do-work.htm
[19] 1xEV-DO Technology Overview (Physical Layer and Signaling)
http://nwswww.wh.Alcatel-Lucent.com/gaby/do-work.htm
[20] Call processing exception reporting for 1x EV-DO: OHM, SFM and PCFA11 CP
failure error list document (d32365)
[21] 1xEV-DO Service Measurements Manual, 401-614-326
[22] System Capacity Monitoring and Engineering Guidelines (SCME) for 1xEV-DO, 401-
614-331.
[23] 1xEV-DO Configuration Parameters Guide, 401-614-324

ALL RIGHTS RESERVED ALCATEL-LUCENT 2007 11


1 X EV-DO RF P ER FORM ANCE A NALYS IS & T ROUB LESHOOT ING G U IDE

[24] Flexent Wireless Networks CDMA2000 1xEV-DO Radio Access System Planning
and Implementation Guide, 401-614-101
[25] Flexent Wireless Networks CDMA2000 1xEV-DO Radio Access System Engineering
Guidelines for Cell OA&M Convergence R25.0, 401-614-108
[26] CDMA 3G-1X/1xEV-DO Cell/RNC Software Release 28.00 Release Letters
[27] Multi-Carrier RF Optimization Guideline for 1X EV-DO Systems

ALL RIGHTS RESERVED ALCATEL-LUCENT 2007 12


1 X EV-DO RF P ER FORM ANCE A NALYS IS & T ROUB LESHOOT ING G U IDE

Useful Web Pages


1xEV-DO Current Engineering http://nwswww.wh.Alcatel-Lucent.com/hdrce/
1xEV-DO FOA http://nwswww.wh.Alcatel-Lucent.com/1xevfoa/index.html
1xEV-DO Perf Management team http://nwswww.wh.Alcatel-Lucent.com/sobers/DO_Perf_CFT/
1xEV-DO Release Letters http://nwswww.wh.Alcatel-Lucent.com/hdrce/
APX Library http://nwswww.ih.Alcatel-Lucent.com/apxlib/
Ask Alcatel-Lucent http://askAlcatel-Lucent.web.Alcatel-Lucent.com/
Cares Tickets (ARs) http://cares.web.Alcatel-Lucent.com/
External Technical Support https://wireless.support.Alcatel-Lucent.com/amps/
FID/FDD Lookup http://wngpjm.web.Alcatel-Lucent.com/cpdr/
LWS Knowledge Store http://servility.ih.Alcatel-Lucent.com/msstpi/
Alcatel-Lucent Alerts http://mss.web.Alcatel-Lucent.com/bulflash/bulflash.html
Alcatel-Lucent Navigator http://navigator.web.Alcatel-Lucent.com/
Alcatel-Lucent Product Documents http://www.cic.Alcatel-Lucent.com/cicmulti.html
SRD website http://nwswww.wh.Alcatel-Lucent.com/seamps/livelink/
Wireless Service Creation http://rfcoresupport.wh.Alcatel-Lucent.com/
& Performance Department
Wireless University https://training.Alcatel-Lucent.com/SabaWeb
Release Letters (External Link) https://wireless.support.lucent.com/amps/rls_info/rls_doc/1XEV-DO/

ALL RIGHTS RESERVED ALCATEL-LUCENT 2007 13


1 X EV-DO RF P ER FORM ANCE A NALYS IS & T ROUB LESHOOT ING G U IDE

1. INTRODUCTION

This document has been compiled to serve as a guide for understanding concepts and techniques
necessary to analyze, troubleshoot and monitor performance in a commercial 1xEV-DO network.
The primary intent is to present information on ways to assess current levels of service quality
offered by the system as well as ways to augment it by identifying and resolving specific
performance issues.

There are two main ways to approach the task of performance troubleshooting and monitoring in
any network. The approach often depends on the phase of network operation. The two
approaches/phases are described below.

For a new network deployment, single-user based drive test techniques are often employed
effectively to optimize performance before turning over the network for commercial operation. It
involves characterizing RF performance under controlled conditions with use of several test
mobiles. There is little real-user traffic to base system level analysis on.

Once in commercial mode, several parameters influence performance over time such as increase
in subscriber usage, shift in traffic pattern, cell additions, carrier growth, hardware problems,
environment changes, etc. that make the ongoing performance management critical. In this
phase, network performance can be gauged via the use of Service Measurements (SM) and
system level tools that derive measurements off the abundant commercial traffic. Controlled
single user tests may be needed to supplement system level analysis, mainly in localized areas to
fine tune performance.

With the above background in mind, it is important to clarify the approach adopted in this guide.
It mainly addresses a 1xEV-DO commercial market with the number of cells and traffic large
enough to offer a statistically meaningful analysis based on system level indicators.

The document follows a top-down approach in that it looks at system level performance and
problem resolution techniques via Key Performance Indicators (KPI) based on available service
measurements for 1xEV-DO1. For each KPI, it supplements the system wide performance
management by selective drive test based optimization techniques.

In terms of the scope of the document, another point to note is that this guide primarily addresses
performance associated with the sub-system involving Access Terminal (AT), Base Transceiver
Station (BTS) and Flexent Mobility Server (FMS) components (explained in Section 3). There is
a separate document for end-to-end network troubleshooting using single user call traces and
associated diagnostics in more detail [12].

1xEV-DO has been deployed extensively by large service providers like Verizon Wireless and
Sprint PCS with additional deployments expected by China Unicom and Reliance. With each
1xEV-DO product release, additional service measurements peg counts as well as new
performance data sources are introduced enhancing the capability to use live traffic measurements

1
Service Measurement counts available through R27.0

ALL RIGHTS RESERVED ALCATEL-LUCENT 2007 14


1 X EV-DO RF P ER FORM ANCE A NALYS IS & T ROUB LESHOOT ING G U IDE

to troubleshoot and improve 1xEV-DO network performance. For example, with R25, Handoff
Matrix and Undeclared Neighbor List data is available to optimize neighbor lists.

This guide is written by applying general concepts of performance management from 2G/3G1x
systems due to many similarities with 2G/3G1x CDMA systems. As the knowledge base grows
from experience in commercial 1xEV-DO networks, additional troubleshooting
techniques/principles will be incorporated.

One of the key assumptions made in this guide is that most 1xEV-DO systems are single carrier
deployments. So at this point, minimum information regarding multiple carriers is included in
this guide (see [27], the Multi-Carrier RF Optimization Guidelines for EV-DO document in the
RF Core Support web site). However, as more multiple carrier deployments are introduced, more
information will be added to this guide as needed.

This guide does not cover 1xEV-DO Rev A changes to any troubleshooting procedures, and is
only mentioned in passing where required. Information on Rev A specific changes will be added
in later updates to this guide.

There are several tools available to retrieve and analyze system level performance data. One such
tool is Vallent Corporations Prospect. It primarily allows creating reports (Microsoft Excel
output) based on Service Measurements data. Another tool that allows a more integrated analysis
of not just SM data, but also other system level information, such as ROP alarms and call
processing failures, Translations/Configuration summary, Neighbor List analysis and Handoff
Matrix/Undeclared Neighbor List analysis, is the Alcatel-Lucent internal tool, SPO (System
Performance Optimization).

Celnet Xplorer is a recently developed tool that was used during performance optimization of
Verizon Wireless markets. Celnet Xplorer collects data for each data flow and contains detailed
information about RF conditions, mobile type and call status. .

ALL RIGHTS RESERVED ALCATEL-LUCENT 2007 15


1 X EV-DO RF P ER FORM ANCE A NALYS IS & T ROUB LESHOOT ING G U IDE

2. USING THIS GUIDE

The document is intended for Alcatel-Lucent internal use. Typical users will be comprised of
LWS personnel specializing in RF performance and troubleshooting as well as Customer
Technical Advocates.

To get the most from the guide, familiarity with 2G/3G1x performance management will prove
helpful, but not required.

Before listing different ways this guide can be employed, it is important to highlight the
fundamental questions the guide addresses related to system performance management. These
are:

KPI/Metrics: How to define/characterize service quality offered to the end user?

Factors: What are some of the factors that can influence these metrics?

Pointers: How does one detect/isolate the performance impacting factor/root cause?

Resolution: How to solve/mitigate the performance impact?

This type of information can be applied effectively in several ways, mainly:

Audit system level performance

It is important to assess network performance on a continual basis. Long-term trending of system


level performance indicators helps identify issues related to system bottlenecks, malfunctioning
hardware, possible translation entry errors, etc as they develop. The information in the guide will
provide useful pointers to aid in the effort.

Note that before embarking on KPI studies, it is important to thoroughly scrub translations per
recommended values (Refer to [1]-[7], [17]). Over time, recommendations may change, or
translation values may get altered mistakenly (such as fine-tuning some other parameter), or not
kept up to date with configuration changes (such as when new resources are provisioned).
Cleaning up translation settings often turns out to be a low hanging fruit to pick when it comes to
improving network performance.

Targeting root cause for a specific problem request

This may be in response to a trouble ticket opened by the customer, citing degradation in a
particular service quality indicator. The goal here is to try to focus on the root cause rather than
performing a system level audit of all metrics.

The basic assumption in the document is that most performance issues can be funneled down to a
few key manifestations. It is left to the user how to categorize the reported anomaly into one of
these few KPIs.

After deducing the impacted KPI, the underlying component or failure cause can be identified
using the repository of information offered in this document. It often requires eliminating other

ALL RIGHTS RESERVED ALCATEL-LUCENT 2007 16


1 X EV-DO RF P ER FORM ANCE A NALYS IS & T ROUB LESHOOT ING G U IDE

causes in order to narrow down to the real one; therefore, it is important to carefully consider all
underlying components. For example, reverse overload may be caused by external interference or
1xEV-DO user load and each has a different resolution strategy.

General Reference

An RF engineer new to the performance management area can learn a great deal on steps to
maintain and troubleshoot network performance using the principles outlined in the document.
The person may or may not have any experience in the area on prior technologies.

KPIs presented in the document are a good way to start delving into the basics of performance
monitoring. The information on failure components can also lead to better understanding of how
to apply/make use of secondary performance metrics.
This version of the CDMA 1X EV-DO RF Performance Analysis & Troubleshooting Guide only
reflects troubleshooting for Lucent 1X EV-DO equipment. While some references of the
document have been updated to reflect the newly created Alcatel- Lucent, most of the references
to the hardware and software are still to Lucent or Lucent Technologies hardware or software.
These references will be updated in future versions of this document as their nomenclature is
updated.

ALL RIGHTS RESERVED ALCATEL-LUCENT 2007 17


1 X EV-DO RF P ER FORM ANCE A NALYS IS & T ROUB LESHOOT ING G U IDE

3. OVERVIEW OF 1XEV-DO SYSTEM AND OPERATION

Before delving into the core subject of this document, it is worthwhile to present a basic view of
the 1xEV-DO system in terms of architecture and operational principles. These are discussed in
the subsequent sections.

3.1 System Functional Blocks

Figure 1 illustrates functional blocks in a standalone Alcatel-Lucent 1xEV-DO system. Figure 2


shows the same for an Alcatel-Lucent 1xEV-DO overlay on an Alcatel-Lucent 3G1x network;
color codes differentiate the shared from unique components in each system.

FIGURE 1 FUNCTIONAL BLOCKS IN A STANDALONE ALCATEL-LUCENT 1XEV-DO


SYSTEM

ALL RIGHTS RESERVED ALCATEL-LUCENT 2007 18


1 X EV-DO RF P ER FORM ANCE A NALYS IS & T ROUB LESHOOT ING G U IDE

1xEV EMS

FIGURE 2 FUNCTIONAL BLOCKS IN A ALCATEL-LUCENT 1XEV-DO OVERLAY ON A


ALCATEL-LUCENT 3G1X HIGH SPEED DATA NETWORK
A 1xEV-DO system can generically be categorized into 2 macro segments, these are:

AN Access Network is the equipment providing data connectivity between a packet


switched data network (typically the Internet) and the access terminals.

AT Access Terminal is the device providing data connectivity to a user. An access


terminal may be connected to a computing device such as a laptop personal computer or
it may be a self-contained data device such as a personal digital assistant.

Next, we present some of the components in each segment of the network.

3.1.1 Access Network

PDSN Packet Data Serving Node is one of the key components in a wireless Data network. In
the downlink direction (Internet to AT), it converts IP packets received from the Internet into PPP
stream and passes on to the 1x EV Controller. In the uplink direction, it terminates the PPP layer,
structures PPP packets coming from PCF into IP packets, and directs them away from the DO
network to the Internet.
AAA server AAA stands for Authorization, Authentication and Accounting. The AAA server
stores the database that contains service and protocol information for each user registered in the
1xEV-DO network. This information is returned to the PDSN using a secure protocol when the
user first establishes a session in the network. The PDSN uses this information for authentication.
AAA also administers the billing function by storing accounting information obtained from the
PDSN.

ALL RIGHTS RESERVED ALCATEL-LUCENT 2007 19


1 X EV-DO RF P ER FORM ANCE A NALYS IS & T ROUB LESHOOT ING G U IDE

1xEV-DO Flexent Mobility System (FMS) It serves as the heart of Alcatel-Lucent 1xEV-DO
call processing and management. The 1xEV-DO FMS frame is also referred to as the 1xEV-DO
Application Processor Frame (DO-APF) or 1xEV-DO RNC frame [24]. Its main hardware
components include a bank of servers, known as APs and TPs, described below.

AP Each FMS frame can contain up to 8 Application Processors (AP). These are actually
4 pairs of APs - both APs in a pair run in full active/backup mode, meaning either one can
take control if the other goes down. AP server is based on the Solaris operating system. Each
AP has 2 Traffic Processors (TP) and several cells to manage.

TP TP runs on the VxWorks operating system. A Data session served by a FMS has an
associated TP. TP handles the signaling and traffic processing for the data flow between the
cell sites and PDSN.

Each FMS contains 2 Cajun switches for traffic exchange between the APs, TPs and the router
(described later).

An Alcatel-Lucent 1xEV-DO FMS can also be classified into software functional blocks. The
specific functions are mainly implemented on the AP and the TP:

1xEV-DO Controller The 1xEV-DO control function provides signaling and traffic
processing control for each session. These functions include session establishment and release
(performed by a functional entity called the Overhead Manager or OHM); frame selection and
Radio Link Protocol (RLP) processing (performed by functional entities called Selector
Function Main or SFM, and RLP and Signaling Manager or RSM), see [24] for more
information.

During setup of a Data session, AP assigns and binds a TP for rest of the session until and unless
an idle (dormant) mode inter-PCF transfer occurs. Among several functions, the TP is
responsible for physical layer channel assignment and managing virtual handoffs between cells.
From Data protocol stack perspective, it receives GRE (Generic Routing Encapsulation) packets
from PCF, sequences GRE stream into RLP packets to pass on to the serving cell in the forward
link. In the other direction, it performs frame-selection to choose the best received physical layer
frame from various cells in the Active set of the call, terminates RLP packets and performs GRE
wrapping to transport PPP packets to the associated PCF for the session. Another important
function it handles is the RLP recovery mechanism. On the reverse link, it is responsible for
verifying the integrity of RLP frames received over the physical layer, and requesting re-
transmission of RLP frames received in error via NAKs. On the forward link, it buffers and re-
transmits RLP packets in response to RLP NAKs received from the AT.

PCF The term Packet Control Function (PCF) refers to the general PCF functionality, which
includes both the A10-PCF and the A11-PCF functionality. The A10-PCF is responsible for the
data traffic (A10) connection and resides physically on the TP (Traffic Processor). The A11-PCF
handles the signaling (A11) interface and resides on the AP within the FMS hardware
architecture. The primary function of the PCF is to buffer and transport PPP packets between the
PDSN and FMS using GRE (Generic Routing Encapsulation) protocol. As it is evident from the
protocol structure in Figure 3, the GRE encapsulated PPP packets are carried over an Ethernet
connection (which could consist of one or more routers) using IP datagrams; this is a local IP

ALL RIGHTS RESERVED ALCATEL-LUCENT 2007 20


1 X EV-DO RF P ER FORM ANCE A NALYS IS & T ROUB LESHOOT ING G U IDE

network separate from the end-to-end IP packet delivery mechanism between PDSN and AT.
Also, note that the interface between PCF and PDSN is also often called A10/A11 connection,
per IS2001. The PCF is also responsible for generating accounting information needed by PDSN
for each call.

Router The router performs an important function of distributing traffic from the controller to
the destination cell(s) and aggregating uplink traffic from the cell to the Controller. In addition, it
also routes traffic between PCF and the PDSN.

1xEV-DO Element Management System (EMS) The EMS interfaces with FMS to provide
OA&M facilities for 1xEV-DO cells and the FMS (as of Release 25 and later, for mixed
CDMA/EV-DO cells, the OA&M for 1xEV-DO cells has been moved to the CDMA RCS
platform after implementing the convergence feature, see [25] for more information). It resides on
the standard Alcatel-Lucent OMP server platform. Its OA&M tasks include configuration
management on 1xEV-DO cells and FMS translation change and verify via the EMS GUI2
(Graphical User Interface), software installation, reboot/restart/status, network provisioning; fault
management alarm monitoring, fault detection and recovery, diagnostics; and, performance
management service measurements gathering and storage.

1xEV-DO Base Transceiver station (BTS) It is the cell site servicing over-the-air physical
connections with the AT to transport RLP frames in either direction. It has an EVM (1xEV-DO
Modem) hardware unit (similar to Channel Card Unit or CCU in 2G/3G1x BTS), which consists
of Forward Link and Reverse Link Modems also known as FLMs and RLMs to transmit and
receive physical/MAC layer frames in the two directions. Another important circuit pack is
CDMA Radio Controller (CRC). It has two functional subunits one is Line Interface Unit
(LIU) whose primary function is to terminate the HDLC protocol over T1/E1 backhaul, and the
other is Main Cell Controller (MCC) which is responsible for BTS wide OA&M control, such as,
alarm management, interface to other circuit packets, NVM updates and diagnostic testing. CBR
(CDMA Baseband Radio) is another circuit pack responsible for digital and RF signal inter-
conversion.

On the forward link, at any given instant, AT is tuned to traffic channel from any one cell. On the
reverse link, there can be multiple cells in the Active set that are actively receiving, decoding and
passing on traffic channel frame contents to the Controller. In addition, cell transmits certain
Control Channels to assist AT with the idle mode operation and call set-up process. It includes
broadcast messages (similar to overhead messages such as System and Access Parameters
messages in 3G1x) and mobile specific messages (such as page, traffic channel assignment, etc.).
Another key element is its assignment of MAC Indices. MAC Indices are similar to Walsh Codes.
There are a total of 64 MAC Indices, one reserved for each of the overhead channels and the rest
assigned to active calls. MAC Indices are important to distinguish signaling messages as well as
reverse power control bits sent on the forward link for individual calls. Finally, BTS is also
responsible for managing the inner and outer closed loops of reverse power control with the AT.

2
EMS GUI is the interface in the EMS that allows entering / modifying translation parameter values for various
components in the BTS and the FMS. After convergence, the cell hardware configuration for mixed CDMA/EV-DO
cells moved to the CDMA RCS platform, while call processing parameters remain in the EMS.

ALL RIGHTS RESERVED ALCATEL-LUCENT 2007 21


1 X EV-DO RF P ER FORM ANCE A NALYS IS & T ROUB LESHOOT ING G U IDE

1xEV Controller
Call
AAA
Control

Frame RLP Fixed


1xEV 1xEV Router Ethernet
Selector PCF PDSN End
AT BTS System

100
Airlink T1/E1 Internet
Base T
A10
Interface
TCP TCP
UDP UDP
IP IP IP IP
PPP PPP
RLP RLP GRE GRE
TCP TCP
UDP UDP IP IP
IP IP IP IP
MAC MAC HDLC HDLC
802.3 802.3802.3 802.3
PHY PHY T1 T1

FIGURE 3 PROTOCOL STACK DIAGRAM FOR ALCATEL-LUCENT 1XEV-DO SYSTEM [9]

3.1.2 Access Terminal

Also known as the mobile, it can operate in one of 2 different modes, based on the layers of the
protocol stack it manages.

Connected to laptop: In this split function mode, AT terminates physical, MAC and RLP
layers in the Protocol stack. All higher layers (PPP, TCP/UDP, IP, Application, etc) are
terminated in the laptop itself.

Self-connected unit (such as PDA): In contrast to the above mode, AT terminates all the
layers in itself. All the functions mentioned on the AN side have peers in the AT (e.g.,
physical layer channel establishment, reverse power control, RLP recovery,
sequencing/de-sequencing between RLP packets and PPP stream, protocol conversion
between PPP and IP, as well as between TCP/UDP and IP, etc).

The analysis and troubleshooting discussions in the remainder of the document mainly involve
AT, BTS and 1xEV Controller components of the network.

3.2 1xEV-DO System Operation

Unlike 3G1x technology (defined by IS2000 standards), which can support both Voice and Data
calls in a standard 1.25Mhz carrier, 1xEV-DO is a Packet Data only technology. It is defined by
IS856 standards. The 1xEV-DO air interface, AT and AN components have been carefully

ALL RIGHTS RESERVED ALCATEL-LUCENT 2007 22


1 X EV-DO RF P ER FORM ANCE A NALYS IS & T ROUB LESHOOT ING G U IDE

designed to offer much higher data rates/throughputs to the end-user on the forward link
compared to 3G1x systems. The reverse link is largely similar to that of 3G1x.

3.2.1 Forward link

3.2.1.1 Rel 0 Forward Link

The 1xEV-DO Rel 0 forward link is the key differentiator that sets the technology apart from
3G1x. It offers peak rates of 2.4 Mbps compared to 153.6 kbps in 3G1x. The various rates
include 38.4, 76.8, 153.6, 307.2, 614.4, 921.6, 1228.6, 1843.2 and 2457.2 kbps. The forward link
is time multiplexed; there is no concept of Walsh code sharing among logical channels/users.
Since the forward link is time multiplexed, a MAC index is used to identify what type of data is
being transmitted: control or traffic data. If transmitting traffic data, the MAC index is used to
identify what user the data is for, if transmitting control data, the MAC index indicates whether
the type of control channel used. There are a total of 64 indices in Rel 0, with some being
reserved. There is also no concept of separate fundamental and supplemental traffic channels per
user. Each user is allotted entire sector-carrier power at any instant without any power control for
the transmission at a given rate.

The forward link transmission is in terms of slots; each slot is 1.66ms. The Pilot channel structure
also differs considerably compared to 3G1x systems. All cells, for a small portion within every
slot, send the Pilot channel synchronously. The AT computes received Ec/Io (or Signal to Noise
Ratio) for each Pilot during this synchronous interval. This SNR measurement is one of the key
parameters used by the AT to select and request a specific rate. Unlike 3G1x systems, the rate
decision process is in the AT and not in the cell. This allows for quicker rate adaptation to
changing RF conditions, quicker transition to a stronger sector to receive forward link data, and
minimal signaling overheads, compared to 3G1x systems. The AT selects a specific rate to meet a
target packet error rate (hard-coded to 1% in current ATs). It is important to note that the
transmitted cell site power to the user stays constant, so the AT has to adjust rates to maintain the
QoS (in 3G1x systems, cell has to adjust rate and transmit power to maintain QoS).

1xEV-DO uses a more sophisticated modulation technique to support high-speed data rates 16-
QAM or 8-PSK modulation (lower rates use conventional QPSK). The other distinguishing
1xEV-DO attribute is the use of Incremental Redundancy on forward link. Basically, for multiple
slot transmissions (except the higher couple rates, all other rates span over multiple slots), the cell
sends data first and then redundancy bits in an incremental manner only if the AT is having
trouble decoding the contents. This results in high utilization of the physical layer.

On the forward link, only one sector is transmitting all the data to a given user. As mentioned
above, it is the AT that requests cell to transmit on a specific rate. While in an active call, the AT
constantly sends these requests via Data Rate Control (DRC) bits on the reverse link. The DRC
indicates the AN all the forward link transmit parameters: what data rate, packet length, number
of slots and coding rate to use when transmitting data to the AT. The AT addresses these DRCs to
a specific cell from which it desires transmission. Such a cell is called DRC pointed cell. When
another pilot in its Active set becomes stronger, the AT starts pointing its DRC to this new cell.
This process of switching active sector transmission is called Virtual Soft handoff. It is similar to
the anchor transfer of forward link supplemental channel performed by the cell site in Alcatel-

ALL RIGHTS RESERVED ALCATEL-LUCENT 2007 23


1 X EV-DO RF P ER FORM ANCE A NALYS IS & T ROUB LESHOOT ING G U IDE

Lucent 3G1x systems; in case of 1xEV-DO, the process is managed by the AT and inherently
faster.

The initial Alcatel-Lucent 1xEV-DO system uses a proportionally fair scheduler, which
attempts to maximize the aggregate sector data rate while managing delay (data starvation)
experienced by the disadvantaged users. Cell evaluates DRCs requests from all ATs and chooses
the one with the highest rate. Due to short-term RF variations, different users experience SNR
peaks at different times; the one experiencing strong signal gets the data - this is the
proportional part where user served and hence, the assigned rate is based on signal strength.
Considering long-term averages, everybody gets roughly equal transmission time (number of
slots) this is the fair part.

3.2.1.2 Rev A Forward Link

As in Rel 0, Rev A systems continue using the DRC to determine data rates and other forward
link transmit characteristics. However in Rev A the DRC is changed to include two more data
rates (with a maximum of 3.072Mbps) and it no longer indicates one single set of transmit format
parameters as in Rel 0, but a set of transmit formats. The AN then chooses one of these formats
taking into account other information such as packet size, packet type, quality of service and
other active users requirements. This gives the AN some flexibility in deciding forward link data
rates.

Rev A systems also introduce an ARQ channel which is used to support reverse link hybrid ARQ.

Additionally, Rev A changes the number of MAC indices to 128, however Rev A uses some of
the new MAC indices for other purposes, such as Multi-User packets, a new control channel.
Additionally, MAC index 5, (which is used in Rel 0 for user traffic) is used for a broadcast
channel in Rev A.

3.2.2 Reverse Link

3.2.2.1 Rel 0 Reverse Link

As mentioned earlier, there are many similarities in the reverse link between 1xEV-DO Rel 0 and
3G1x systems. The supported rates are 9.6, 19.2, 38.4, 76.8 and 153.6 kbps. There is also a Pilot
channel transmitted by each AT on the reverse channel to aid coherent demodulation at the cell
site. The soft/softer handoff process and the Active set maintenance procedures are similar (Tadd,
Tdrop, Tcomp, TTdrop concepts of IS2000). The AT can enter into a maximum of 6-way soft
handoff, depending on translation settings. The AT transmission is power controlled by the cell
site to maintain a 1% reverse frame error rate. Unlike on the forward link where the target error
rate is hard coded at the AT, the one on the reverse link can be controlled by a translation called
Reverse Target Frame Error Rate on the Sector EMS GUI page.

There are some differences, however. The physical layer frames are 26.66ms in duration (20ms
frames used for 3G1x), thus data rates can change only every 26.66ms. The reverse link carries

ALL RIGHTS RESERVED ALCATEL-LUCENT 2007 24


1 X EV-DO RF P ER FORM ANCE A NALYS IS & T ROUB LESHOOT ING G U IDE

Data Rate Control (DRC) bits to report forward link requested rates. There is no concept of
separate fundamental and supplemental channels. There is only one logical traffic channel per AT
the AT dynamically changes rate based on available data backlog and a set of rate transition
probabilities. The AT starts with low transmission rate and ramps it up based on these
probabilities. The AT informs the AN of its current data rate transmission by including Reverse
Rate Indication (RRI) information on each frame.

As mentioned above, each AT decides the transmission rate on its own. To control the reverse
link integrity and prevent RF overload, the cell site has two options. One is to transmit a Rate
Limit message on the forward link control channel limiting the maximum rate used by the ATs.
The other mechanism is to set the Reverse Activity Bit (RAB) on. RAB is sent on the Medium
Access Control channel the MAC channel contains RAB, reverse power control commands and
DRC lock bit for Active calls on the sector. When the RAB bit is on, some of the users can
probabilistically halve their rate on the next frame. The percentage of such users is specified in
translations. If the bit is off, the users can double their transmission rate unless they are already at
maximum rate (153.6kbps).

3.2.2.2 Rev A changes

For Rev A systems, the number, and rate, of supported reverse link data rates increases, with the
minimum being 19.2 kbps and the maximum being 1843.2 kbps). To support the higher reverse
link data rates the frames are divided into sub-frames with duration of 6.67ms, enabling faster
data rate changes. Additionally, an Auxiliary Pilot is introduced to support the higher reverse link
data rates.

The Rev A reverse link also introduces the Data Source Control (DSC) channel to aid the
continuity of data transmission on handoff regions, and the DRC is updated to support the higher
forward link data rates. Other changes to the reverse link include changes to the RRI to indicate
the data payload size and a sub-packet ID to enable reverse link Hybrid ARQ.

ALL RIGHTS RESERVED ALCATEL-LUCENT 2007 25


1 X EV-DO RF P ER FORM ANCE A NALYS IS & T ROUB LESHOOT ING G U IDE

4. SUMMARY OF PERFORMANCE RELATED FEATURES IN


PRODUCT RELEASES

This section summarizes the key performance related features that are part of the current release
(R28.0) and the previous release (R27.0). Features that are scheduled for the next release (R29.0)
are also discussed to provide the user with information about future performance impacting
features in the product. For a comprehensive list of features available in each release, the user is
encouraged to refer to the release letters.

4.1 Performance related features in R27.0


The following performance related features are available in Cell/RNC release 27.0.

FID-8219.12 1x EV-DO multiple T1/E1 uplink: This feature covers the platform work to
support sending reversed link traffic data upstream over multiple T1s/E1s in 1xEV-DO, with
proper priority control. This capability is important for the support of multiple carriers and
the support of 1xEV-DO RevA.

FID-9138.16 Session Timeout in the EVDO RNC: This security feature provides session
time-out for inactive UNIX sessions in EVDO RNC. Inactivity is the lack of user input to the
session from a keyboard. The session is terminated along with its associated processes when
the inactivity timeout value is exceeded. The inactivity timeout value is configurable by the
security administrator. The value of this parameter establishes the timeout value for all users
on all the RNC APs in the service node. The security administrator can configure a user to be
exempt from this timeout limitation. The security administrator makes changes to the
configuration at the OMP, which runs scripts to implement the changes in the subtending
RNC APs. The relevant interfaces impacted by the inactivity timeout include the EVDO RNC
console port, an EVDO RNC IP accessible port providing UNIX-like remote login
(telnet/ssh) capability, and an EVDO RNC application supported by UNIX-like remote login
(telnet/ssh) capability. This feature is dependent on SSH for secure provisioning. This feature
has been requested by Sprint.

FID-9253.1 1x EV-DO performance monitoring tools enhancements: This FID provides


the following two 1xEV-DO tools support capabilities: Special Engineering Studies tool to
provide histogram counts of performance data for capacity planning purposes including
Reverse Link Short and Long Term RSSI, Forward Link Traffic Percent Busy Slots and
Datalink Utilization and a tool to identify active users per EVM (i.e., provide a Reverse Link
Channel Element Status Display capability - RDAF #706248)

FID-12269.2 Enhancements to EVDO Per Call Service Measurements: This feature


includes requirements listed in SRD 5677 as "deferred", Per-Call Measurement Data (PCMD)
support for basic Rev A EVDO, and general PCMD and Service Measurements
Enhancements. The deferred requirements from 5677 include the option of reporting either no
RUM data, or reporting the data from the first and last RUM received during a PCMD
Interval, and enhancements of the RUM data reported. The PCMD requirements to support
Basic Rev A will be to generate PCMD Records for Personality Changes (GAUP) and
optionally for flows in Multi-Flow Packet Applications (MFPA).

ALL RIGHTS RESERVED ALCATEL-LUCENT 2007 26


1 X EV-DO RF P ER FORM ANCE A NALYS IS & T ROUB LESHOOT ING G U IDE

FID 12528.0 Rev 0 RNC to handle idle handoff from Rev A session: FID-12528.0 will
add software changes for a Rel 0 target RNC to close a Rev A session idle handoff from a
Rev A source RNC. For all three idle handoff methods, Assign First, Color Code and Prior
Session, once the target RNC receives the old UATI of an AT idle handoff to it, it will send
the A13-Session Information Request message to the source RNC and wait to receive the
A13-Session Information Response message. If the target RNC is with Rel 0 software
releases and the A13-Session Information Response message contains any unknown
protocols indicating the AT is not with a Rel 0 session, the target RNC will close the session.
The Rel 0 RNC is required to retrieve the Enhanced Idle State Protocol attributes from the
A13-Session Information Response message. The information is needed in case the A13-
Session Information Response message come back late and the Rel 0 RNC will have to send
the SessionClose message to the AT during its awake time defined in the Enhanced Idle State
Protocol.

4.2 Performance related features in R28.0


The following performance related features are available in Cell/RNC release 28.0.

FID-9053.3 Increasing the UATI Sessions per OHM: This optional feature increases the
maximum number of UATI sessions per OHM beyond 65K (to 100K). The feature also adds
the OA&M functionality needed to view the memory size of the equipped AP CP2140 cards
in the 1xEV-DO RNC (a continuation of 9053.2).

FID-9123.2 Bulk Provisioning Capability for 1x EV-DO with OMC-RAN: This feature is
to ensure that the EVDO bulk provisioning capability via the OMP-FX is available regardless
of whether the Element Manager is the EMS or OMC-RAN.

FID-11003.25 1x EV-DO Standalone support with FMS based RNC phase 1: This feature
implements support with the FMS RNC, of a standalone EV-DO system using the converged
(RCS based) cell OA&M implemented in R25 for mixed mode networks. This feature
includes all required functions outside of FMM development covered in FID-11003.2. This
includes changes to support Generic Retrofit, and documentation.

FID-12078.12 Data Over Signaling Protocol: This feature enables the support of
transmission of data over the access channel (reverse link) using the Data Over Signaling
protocol defined in the HRPD Rev A standard. The support of forward link DoS is provided
in 12078.36. This feature is useful for application such as sending messages for PTT, etc.
where real time performance is critical.

FID-12078.13 Enhanced Idle State Protocol: This feature supports the Enhanced Idle State
protocol specified in the HRPD RevA standard. The Enhanced Idle State protocol enhances
the control of idle state monitoring and provides benefits such as reduced paging delay and
better battery life to the mobile users.

FID-12373.1 Prior Session: Allow Session Transfer from/to Serving RNC: Prior to this
feature, the system will block any prior session requests that are made by the AT on the RNC
where it currently has its previous session. In the field, this apparently occurs frequently,
especially in areas of confused or poor RF, such that, in some areas, over 40% of all prior

ALL RIGHTS RESERVED ALCATEL-LUCENT 2007 27


1 X EV-DO RF P ER FORM ANCE A NALYS IS & T ROUB LESHOOT ING G U IDE

session requests are for sessions from the serving RNC and are thus blocked. The same-RNC
prior session request blocking was put in place so that the software would not perform a prior
session request for sessions that were already removed by the existing hardware ID matching
software. This feature will allow the previous session with matching hardware IDs to persist
long enough to be transferred, and will remove the same-RNC restriction, allowing the same-
RNC prior session transfers to occur. If the new session with a matching hardware ID does
not request the matching prior session during session negotiation, the system will then
continue on and remove the older orphaned session, as before.

FID-12456.1 RNC Grouping to Improve Idle Handoff Performance: This feature allows
multiple RNC's to form a group in which AT can be served by one RNC but controlled by
another during in a dormant state. The PCF entity with the A10 termination will be able to
send page messages across RNC boundaries to any cell within the multiple RNCs in a Group.
This feature provide the capability so that AT does not need to wake up at RNC boundary or
it wakes up across the RNC boundary but it can still keep its session. This feature will
improve the overall system performance to avoid the idle handoff ping-ponging and RATI
ping-ponging at RNC boundaries. The above functionalities will be available to customers in
12456.1 and this feature will provide the Infrastructure for 12456.1.

FID 12458.01 Per OHM Service Measurements in 1x EV-DO: This feature changes the
existing AP service measurement counts to per OHM service measurement counts. OHM
failover alarms will also be provided with this feature. Internally this feature also enhances
the session memory management infra-structure to enable the support of 40K UATI sessions
for the release and to allow the memory to be used more efficiently on "unused" sessions
(sessions without R-P connections).

FID-12766.0 Customer Critical Service Measurements and Translations: This feature


will allow fast turnaround for features that require SM or Translations on EVDO. It contains:
1. Spare EMS translations, whose values are delivered to, and are accepted by the RNC, and
are available for use by the software, but are not used by this feature, and
2. Spare Service Measurements, which are available at the RNC to be pegged, but are not
actually pegged by this feature, and delivered to the OMP, and to entities using SMs.

FID 13378.0 Service Measurements Re-architecture: Prior to FID-13378.0, 1x EV-DO


RNC sector-carrier service measurements always contained data for six sectors for every
carrier, regardless of how many sectors the carrier may have.
Feature 13378.0 modifies the 1xEVDO RNC sector-carrier SM reporting, the system only
generates RNC SM data for six sectors if sectors numbered 4, 5, or 6 have service
measurements except when the modem that supports sectors 4, 5, and 6 was OOS for the
entire SM collection interval. In this case, only three sectors RNC SM data will be reported.
If both modems were OOS for the entire SM collection interval, then no SMs for the cell will
be reported.

4.3 Performance related features scheduled in R29.0


The following performance related features are scheduled to be available in Cell/RNC release
29.0. It must be noted that the feature content may change by the R29 GA time frame.

ALL RIGHTS RESERVED ALCATEL-LUCENT 2007 28


1 X EV-DO RF P ER FORM ANCE A NALYS IS & T ROUB LESHOOT ING G U IDE

FID 10862.5 1xEV-DO Gold BCMCS: This feature enables the support of
Broadcast/Multicast Service (BCMCS) in the 1xEV-DO system. In 1xEV-DO, data is sent to
the end users typically via unicast. With BCMCS, data is broadcasted over the air and
multiple users can receive the information simultaneously. This capability provides a more
efficient use of the air interface resource for applications where one to many communications
is used, generating new service opportunities. The feature provides BCMCS service using
the Basic BCMCS Broadcast Channel (Gold BCMCS) specified in the 3GPP2 HRPD
BCMCS air interface standard (3GPP2 C.S0054-0 v2.0).

FID 12078.11 More QoS Enhancements for the HRPD Rev A network: This feature will
support the following functionalities to enhance Quality of Service (QoS) operation for
1xEV-DO Rev A network.

1. Enhanced carrier selection in mixed network (Rev0 /Rev A).

2. There will be an additional threshold during carrier selection that enables the
customer to bias the AN to select the best personality for a UATI in a given
sector. Carrier selection will also be thresholded by the maximum allowed Rev A
users in a sector. If number of users are fewer than this threshold and RL RF
loading is within the RL loading differential threshold then the originating
carrier will be selected. If no Rev A carriers are available then a Rel0 carrier will
be selected.

3. FL Scheduler work to support additional states.

4. The enhancement increases the maximum number of priority states from 3 to 5 to


improve scheduler performance.

5. Infrastructure for clock accuracy and SM counts to support Delay.

6. Customers have requested to have delay SMs per Service Category for Backhaul
(between RNC and BTS) and BTS queue delay.

7. Enhanced RNC Overload Control and Session Status Command support for Rev
A and MFPA. This functionality is added to make sure that when TP or AP is in
overload, the overload control algorithm is enhanced to ensure that the QoS
flows are given preferential treatment over Best effort flows. The session status
command is also enhanced to include Rev A and MFPA states.

8. Enhanced Service Measurements support. This includes many new Service


Measurements to monitor the system performance in presence of QoS flows.

9. Synchronizing RTCMAC3 attributes between AN and AT after they change in


the database. Whenever an RTCMAC3 parameter is changed in the database, the
same is re-GAUPed with the AT to ensure that the AT and the AN are in sync.

FID 12267.30 Ethernet Backhaul for both CDMA and 1xEV-DO for Modcell 4.0: This
feature provides Ethernet backhaul for Modcell 4.0, Compact 4.0 and HD 4.0 for both
CDMA and 1xEV-DO applications. As one of L1/L2 options that IP backhaul offers,
Ethernet backhaul will be an alternative to T1/E1 backhaul facility with broad bandwidth

ALL RIGHTS RESERVED ALCATEL-LUCENT 2007 29


1 X EV-DO RF P ER FORM ANCE A NALYS IS & T ROUB LESHOOT ING G U IDE

ranged up to 100 Mbps. Ethernet backhaul will benefit the cell site where multiple T1s cost
more. In addition, customer will also benefit from simplified provisioning over Ethernet
backhaul transport. Ethernet backhaul will be offered with URC and URCII. Each URC or
URCII can be assigned for 3G1X or 1xEV-DO, but not mixed. An Ethernet Interface Unit
(EIU) as part of the BTS will provide one or two 100 Base-T connections towards User-to-
Network Interface (UNI). Manual configuration and auto negotiation on each Ethernet
interface will be supported. However, manual configuration to fix each Ethernet interface at
100 Mbps speed and full duplex mode is recommended. This feature is for NAR markets
only.

FID 12267.31 Ethernet Backhaul for both CDMA and 1xEV-DO for Modcell
1.0/2.0/3.0: This feature is to provide Ethernet interface for Modcell 1.0/2.0/3.0 for CDMA
and 1x EV-DO. 10/100 BaseT will be supported as Ethernet interface at the BTS. Modcell
1.0 and 2.0 will also requires a field upgrade kit providing physical connection between
URCm and Ethernet port in I/O module at the frame.

FID 13378.1 Service Measurement Rearchitecture - Phase 2: This feature will Improve
AP memory utilization by removing the standby SM dictionary on the APs. Add new SM
output version number to the hdrfms files including both hourly and short-interval SM files.
Ensure sector-carrier SM output in the hdrfms files report 6 sectors data if the cell is
configured with sectors numbered 4, 5, or 6 regardless of whether the cell is active or OOS.
Otherwise, the SM data should contain only 3 sectors data.

FID 13571.0 Pilot Channel Power Measurement: This feature allows the cell to measure
the power of the Pilot channel. The main driver behind this feature is to provide more
troubleshooting capability for the RF Tx path. This is an on-demand test that can be
requested via TI. The following values will be provided upon request for Pilot channel power
measurement: Average value (in dBm) of the Pilot channel power, Actual Translation
parameter value for Pilot Gain, Expected Pilot power measurement. This feature is restricted
to the BTS 8440 (M4.0B with MCPA). Note that the pilot power for the carriers on the UCR
can also be measured, but in this release there is no way of getting the IAOC scalar and
hence, the expected pilot power will not be available for these UCR carriers.

ALL RIGHTS RESERVED ALCATEL-LUCENT 2007 30


1 X EV-DO RF P ER FORM ANCE A NALYS IS & T ROUB LESHOOT ING G U IDE

5. KPI OPTIMIZATION AND TROUBLESHOOTING

At a higher level, the performance of a 1xEV-DO system can be quantified via a few primary
metrics, named here, the KPIs. When used collectively, they are indicative of the quality of
service experienced by a typical end user. In a more general sense, they may also be applied to
characterize performance in any wireless Data network.

These KPIs are:

Established Connection Rate

Dropped Connection Rate

Forward and Reverse Data Rates

As may be evident from the names, the KPIs together help gauge the quality of a network in
terms of its ability to offer successful call set-ups, minimal unintended call interruptions and fast
download speeds to an end user all essential ingredients for a rich Data experience.

In the following sections, we treat the KPIs individually. For each KPI, we first provide a high
level definition and then, explore the underlying elements/parameters that may negatively
influence it along with the associated troubleshooting/rectification techniques. Complete formulas
for the KPIs are available in [25] and [22].

One key point to note is that the ability to isolate a factor influencing any of the KPIs hinges on
the degree to which the problem manifests. The latter itself depends on many factors such as the
extent of the problem area, amount of call activity in the problem area, usage patterns, persistence
of the problem, etc. In general, larger than normal degradation in a metric stands out much easier
and hence, facilitates its own detection and resolution.

In 1x EV-DO R27, irrespective of how many sectors are equipped, service measurements are
always reported for all six sectors. However, with the service measurements re-architecture
feature in R28, service measurements data for sectors 4, 5 and 6 will be reported only if those
sectors have any SM data or if the modem that supports those sectors is in OOS condition.

ALL RIGHTS RESERVED ALCATEL-LUCENT 2007 31


1 X EV-DO RF P ER FORM ANCE A NALYS IS & T ROUB LESHOOT ING G U IDE

5.1 Established Connection Rate

In essence, this KPI represents the success rate associated with setting up a connection.
Mathematically, for releases prior to Release 28, it can be expressed as follows:

Connection Requests Connection Failures


Establishe d Connection Rate x100
Connection Requests

The metric can be computed on a per sector-carrier basis.

The various terms in the above equation are computed from SM peg counts collected on a per-
sector-carrier basis as follows:

Connection Failures sum of:


AN_INIT_CONN_ATTMPT_FAIL_RL_NOT_ACQ
AT_INIT_CONN_ATTMPT_FAIL_RL_NOT_ACQ
AN_INIT_CONN_ATTMPT_FAIL_NO_TCC_RCVD
AT_INIT_CONN_ATTMPT_FAIL_NO_TCC_RCVD
AN_INIT_CONN_ATMPT_FAIL_OTHER_PRETCA
AN_INIT_CONN_ATMPT_FAIL_OTHER_POSTTCA
AT_INIT_CONN_ATMPT_FAIL_OTHER_PRETCA
AT_INIT_CONN_ATMPT_FAIL_OTHER_POSTTCA
AN_INIT_CONN_ATTMPT_FAIL_NO_RES_AVLBL
AT_INIT_CONN_ATTMPT_FAIL_NO_RES_AVLBL
Connection Requests sum of:
AN_INIT_CONN_REQ
AT_INIT_CONN_REQ
As of Release 26 and later, the connection requests and failures are modified to take into account
the cross carrier attempts:
CROSS_CARR_ATTMPT_RCVD_TCC_RCVD
CROSS_CARR_ATTMPT_SENT_TCC_RCVD
And
AN_INIT_CONN_ATTMPT_DIFF_SEC_CARR_SENT
AN_INIT_CONN_ATTMPT_DIFF_SEC_CARR_RCVD
AT_INIT_CONN_ATTMPT_DIFF_SEC_CARR_SENT
AT_INIT_CONN_ATTMPT_DIFF_SEC_CARR_RCVD

A Connection Request is an event where the AT makes a request to establish a call with the AN,
or vice versa. If the cell is able to assign a Traffic channel, the connection may still fail before
establishing a stable traffic link for several reasons, which are aggregated as Connection Failures.

ALL RIGHTS RESERVED ALCATEL-LUCENT 2007 32


1 X EV-DO RF P ER FORM ANCE A NALYS IS & T ROUB LESHOOT ING G U IDE

The cell may not be able to assign Traffic channel for several reasons. One reason could be it
experiences trouble activating traffic channel element typically due to hardware error in the
EVM.

The other reason could be that the cell is already processing maximum number of connections
and cannot free up resources to serve additional users. Any type of call processing overload
(such as both the TPs of an AP going in overload due to excessive traffic load) can also prevent
Traffic channel allocation. These two causes essentially represent call blocking due to resource
shortage. Note that per the current implementation, the call is never denied due to any power
overload conditions unlike in 2G/3G1x networks.

Once assigned a traffic channel, the connection may still fail for several reasons. A call setup
failure in this mode is primarily linked to some type of RF impairment (poor RF coverage, pilot
pollution, excessive RF loading, etc.)

The Established Connection Rate may alternatively be viewed as the deficit between Connection
Requests and the connections that did manage to come up stable on the Traffic Channel. A
complementary metric commonly used is called Access Failure rate. It can be represented as (1
Establish Connection Rate) and signifies failed call or access attempts. The deficit between
connection requests and the connections that were successfully established is caused by RF
Failures, Blocking and other reasons, this breakdown is covered later on.

Service providers define their own metrics to track performance. Verizon wireless has created
Failed Connection Attempts (FCA) as an equivalent metric for Established Connection Rate.
Alcatel-Lucent engineers might encounter this metric when attending to Verizon wireless specific
performance issues. At this point, the FCA metric equation is the same as Alcatel-Lucents
Established Connection Rate equation.

To improve the Established Connection Rate, it is important to understand the constituent factors
influencing the deficit (RF Failures, Blocking, and other factors). We catalogue and explain these
in the subsequent sections. But, first we briefly describe the call setup process.

It must be noted that R27 SU1 changes RNC to handle the connection close by closing the
connection. This is expected to release the call drop rate but since the call is not considered
established and therefore will increase the failed connection attempts. In R27 SU0, this scenario
is pegged as a drop call. The connection close is considered as TCC received and therefore the
call is considered as established. In R27 SU1, compared to R26, the AT/AN initiated
connection attempts failed due to no TCC received peg count is expected to decrease.
Compared to R27 SU0, the AT/AN initiated established connections as well as the Connection
Release due to RLL are expected to decrease.

As of Release 28, a new way of computing the Established Connection Rate is introduced that
uses the Established Connection Rate [Peg]. Mathematically, this new equation can be expressed
as

Established Connection Rate [ Peg ]


Established Connections
x100
Connection Requests

ALL RIGHTS RESERVED ALCATEL-LUCENT 2007 33


1 X EV-DO RF P ER FORM ANCE A NALYS IS & T ROUB LESHOOT ING G U IDE

This metric can be computed on a per sector-carrier level. The Established Connections consist of
the sum of the established connection counts and the cross carrier traffic channel confirmation
counts:
AN_INIT_ESTABLISHED_CONN
AT_INIT_ESTABLISHED_CONN
CROSS_CARRIER_ATTMPT_RCVD_TCC_RCVD
CROSS_CARRIER_ATTMPT_SENT_TCC_RCVD

The Connection Requests can be computed as in the previous equation. See [25] for more
information on this new approach for calculating the Established Call Rate.

These two computation methods include all connections, those used as part of session setup, and
those that are user initiated (e.g. data transmission) except Fast Connection attempts. For Release
28, an additional Established Connection Rate metric is introduced to take into account only user
initiated connections. This new metric is expressed as follows:

Established Connections AT Init Established Conn For Session Setup


Established Connection Rate User [ Peg ] x100
Connection Requests AT Init Conn Request For Session Setup

Notice how both the numerator and denominator subtract those established connections, and
connection requests used for session setup (which are not user initiated).

5.1.1 Call setup process

Figure 4 illustrates a typical flow of messages between AT and the AN for an AT initiated call. It
bears many similarities with a typical origination call attempt in a 2G/3G1x network.

When the AT receives the command to establish a call (such as, from the laptop connected to AT
when user tries to establish a fresh PPP session or reactivate a dormant3 session), AT first checks
to see it is current with the overhead messages, and if not, updates them. If the mobile is
powering up, one of the first steps of setting up a call is sending a Route Update & UATIRequest
message (RATI). This message lets the AN know this is a new connection request and assigns the
AT a Unicast AT Identifier (UATI). The UATI uniquely identifies an AT and persists until the
AT leaves the RNC coverage (this depends on how Inter-RNC Idle Handoffs are set, see [6] for
more information) or the keepAliveTimer timer expires (this parameter is found in the servicenode
configuration form).

3
The first time AT establishes traffic channel with the AN, it is called a new session. The session may last for several
hours or days depending on the AN configuration. Within a session, the AT can switch between active and dormant
modes numerous times. Data is exchanged between AT and AN in active mode over traffic channels. After a short
period (typically of the order of a few seconds depending on the AT/AN setting) of inactivity, the AT goes into
dormancy, that is, tears the traffic link and goes into idle mode. When a fresh set of data appears, either side (AT or
AN) can initiate a connection out of dormancy (generically termed reactivation) depending on who has data to send.
Note that a new session is always initiated by the AT. The reactivation can either be from the AT or AN.

ALL RIGHTS RESERVED ALCATEL-LUCENT 2007 34


1 X EV-DO RF P ER FORM ANCE A NALYS IS & T ROUB LESHOOT ING G U IDE

For Release 28 a new feature was introduced that increases the number of UATI sessions per
OHM to 100K (FID 9053.3, see [26] for more information). This feature requires additional
hardware be installed in APs, but may help reduce blocking due to lack of UATI. Additionally,
Release 28 introduces a new peg count, the SESSION_BLOCKED_NO_UATI count, which can
be used to track blocking due to no UATI resources (see [21] for more information).

On the next available access channel slot, the AT sends a Connection Request message via an
access probe. Along with the Connection Request message, the AT always sends a Route Update
message listing the received pilot strengths. If a response is not received within a finite time, the
AT will retransmit the probe. It will keep sending the probes (based on the Number of Access
Probes configuration parameter value in the Sector and Service Node forms), each at
incrementally higher power, until it receives AcAck (Acknowledgement) from the cell.

Once the cell is ready with the necessary resources and once the cell receives the DRC Cover
Indication from the controller, it sends a Traffic Channel Assignment message (TCA) to the AT. If
there were multiple pilots in the Route Update message, cell will transmit TCA on all the sectors.

On receipt of the TCA, the AT initiates reverse link traffic channel transmission. There is no
traffic channel information sent at this time, only DRC and reverse pilot bits. If there were
multiple pilots in the TCA, the mobile will include all these Pilots in its Active set. The AT will
gradually increase its transmit power until it receives acknowledgement from the cell site or it
times out.

ALL RIGHTS RESERVED ALCATEL-LUCENT 2007 35


1 X EV-DO RF P ER FORM ANCE A NALYS IS & T ROUB LESHOOT ING G U IDE

AT BTS Controller

Command received by the AT to


initiate a call. AT updates overhead
messages and waits for the next
Access Slot to transmit

Route Update & Connection Request Message

Allocate Traffic Channel Request


AcAck
Allocate Traffic Channel Response

DRC Cover Indication

Traffic Channel Assignment Send Traffic Chn Assignment

Send DRC + Pilot and ramp up


Reverse Traffic Channel Mobile Acquired Indication

RTC Ack Send RTC Ack

Traffic Channel Complete Traffic Channel Complete

Ack

Configuration Negotiation Procedures

FIGURE 4 CALL FLOW FOR AN AT INITIATED CONNECTION

When cell site acquires the reverse link transmission, it sends an acknowledgement called
RTCAck to the AT. Once the mobile receives RTCAck message, it sends a Traffic Channel
Complete (TCC) message to the cell site. Cell site acknowledges it back with an Ack message.

If there are any negotiation procedures to be completed, a set of messages is exchanged next to
arrive at the proper configuration. For example, there are certain parameter values that the
mobile assumes by default per standards. This saves the control channel bandwidth (in 2G/3G1x
networks, such information is transmitted via overhead messages such as via System
Parameters message and Access Parameters message). If the cell site is configured with non-
default parameter settings, it must negotiate (exchange) these with the mobile. After the
negotiation is finished, the call setup is complete at this time and the actual Data contents can
now be carried over the air. However, any negotiation failure does not contribute to the
Established Call Failure rate metric.

ALL RIGHTS RESERVED ALCATEL-LUCENT 2007 36


1 X EV-DO RF P ER FORM ANCE A NALYS IS & T ROUB LESHOOT ING G U IDE

Many of the access enhancements first introduced by IS95B standards are also available in a
1xEV-DO system. For example, after sending the first probe and while waiting for the AcAck,
the mobile can switch to a stronger Pilot. This is similar to Access Probe Handoff (APHO)
specified in IS95B/IS2000 standards. After receiving the AcAck and while waiting for the TCA,
the mobile can again switch to the previous or different stronger Pilot. This is similar to Access
Handoff (AHO) specified in IS95B/IS2000 standards. Finally, the TCA can contain multiple
Pilots that the mobile can place in its Active set as it enters Traffic channel. This is similar to
Channel Assignment into Soft Handoff (CAMSHO) specified in IS95B/IS2000 standards. Notes
that these features are implemented as core functionalities in Alcatel-Lucent 1xEV-DO system
and cannot be disabled via translations.

The process for AN initiated call (akin to a mobile termination attempt in 2G/3G1x network) is
similar to the above except that the cell site first pages the mobile to invoke Connection Request
message from the AT. In a wireless Data network, all AN initiated calls are typically
reactivations from dormancy; the network does not initiate a fresh session with the mobile. A
mobile on the other hand is capable of initiating a new session or reactivation from dormancy.

5.1.1.1 Multiple Carrier Support

From Release 26 onwards, 1xEV-DO deployments support multiple carriers. Sites with multiple
carriers can be deployed in areas of heavy traffic, where the extra capacity is required.

In systems with multiple carriers Mobile Hashing distributes ATs evenly across the carriers. This
balances traffic load across carriers and reduces the number of cross-carrier traffic channel
assignments.

In multiple carrier sectors, the AT selects a carrier based on the hash algorithm defined in the IS-
856 standard. The hash algorithm uses the Random Access Terminal Identifier (RATI) and the
number of channels in the SectorParameters message. The AT runs the hash algorithm whenever
the number of channels listed in the SectorParameters message changes.

During session setup, the AN tries to balance traffic by assigning ATs to carriers based on the
Load Balancing Method parameter in the Sector and ServiceNode forms. This parameter has two
possible values: Originating Carrier and Number of Active Users. In 1xEV-DO, there is no
algorithm based on forward RF Loading as the forward link is always transmitting at full power.

For sectors using the Originating Carrier setting, ATs are assigned to the carrier they originated
on as long as the number of active users is less than the Maximum Number of Users Supported (in
the Sector and ServiceNode forms too), see 11 for more information.

For sectors using the Number of Active Users setting, ATs are assigned based on the number of
active users on each carrier. Another parameter, the Load Differential parameter (also in the
Sector and ServiceNode forms) is used in the carrier selection algorithm. For each carrier in the
sector, the algorithm computes the difference between the number of active users on the
originating carrier and the other carriers in that sector. If the calculated values are less than this
parameter, the AT is assigned to the originating carrier. Otherwise, the AT is assigned to the least
loaded carrier in this sector. The value of the Load Differential parameter can be increased to
minimize cross-carrier assignments to reduce call setup failures due to RF mismatch between
carriers, see 11 for more information..

ALL RIGHTS RESERVED ALCATEL-LUCENT 2007 37


1 X EV-DO RF P ER FORM ANCE A NALYS IS & T ROUB LESHOOT ING G U IDE

5.1.1.2 Paging Mechanisms

There are two types of paging mechanisms the AN may use when trying to start data transfer to
an AT, a Normal Paging Method (very similar to the paging used in 3G1x HSPD Networks), and
a Fast Connect paging method.

The Normal Paging method is used by the AN when the suspend timer at a dormant AT has
expired and the AT has entered the slotted mode of operation. This paging method is
implemented via a two-step process:

1) Page the last known active set: In other words, the AN sends a page message on all
sectors last accessed by AT.

2) If there is no response from the AT after step 1), page all cells within the same RNC

Starting with Release 28, there are two translation parameters associated with the Normal Paging
method. The Number of Times to Page the Last Active Set parameter sets the number of page
attempts for Step 1. The Number of Times to Page the Last Seen parameter sets the number of
page attempts for Step 2, see [2] for more information (keep in mind that as of Release 28 the
Paging Strategy parameter is obsolete.). When the last known active set contains Pilots from
different RNCs, only cells from the RNC governing the AT session (UATI) are paged for Step 2.

Release 28 a set of new paging peg counts have been introduced which may be used for
measuring paging effectiveness, see [25] for more information.

Additionally, for Release 28, the RNC Grouping to Improve Idle Hand-off Performance feature
allows combining of RNCs into RNC groups and introduces new paging mechanisms that can be
used to improve paging effectiveness. This feature requires translations to be set and an FAF
entry, see the Release 28 release letter for more information. The paging mechanisms for this
feature are enabled through four translation parameters in the ServiceNode form:

- Number of Times to Page Entire RNC Group

- Number of Times to Page Neighboring RNC

- Number of Times to Page the Last Active Set

- Number of Times to Page the Last Seen RNC

A value of 0 assigned to any of these parameters indicates that the corresponding paging
mechanism should be skipped.

5.1.1.3 Fast Connect Calls

Fast Connect is a special type of AN initiated call, and as previously explained, the Established
Connection Rate does not include these connections. When the AT has been dormant for 5
seconds or less (controlled by the tunable parameter called Suspend Timer) and the AN has a
fresh set of data to transmit to this AT, the AN uses this procedure to bring a dormant session
back onto the Traffic channel. Basically, it just sends the TCA to the AT, which consists of Pilots

ALL RIGHTS RESERVED ALCATEL-LUCENT 2007 38


1 X EV-DO RF P ER FORM ANCE A NALYS IS & T ROUB LESHOOT ING G U IDE

in the ATs Active set at the time prior call release. The idea is that 5 second is a relatively short
interval and the AT must have been in the vicinity, so it is safe to locate it on the prior set of
Pilots. It is appropriately called Fast Connect since there is not much time spent on the
Control/Access channel message exchange and the attempt is reactivate the Traffic channel
quickly.

The AN sends the TCA once it has ascertained necessary resources on the AN. This results in the
peg for Fast Connect Requests. If the cell times out waiting for TCC from the mobile, it pegs it
as Fast Connect Failure No response to traffic channel assignment. If the cell does not
acquire the reverse link after the TCA was sent, then the Fast Connect Failure Reverse link
not acquired count is pegged. Prior to Release 28, the above counts are pegged on the AP on a
per AP basis. Per sector granularity is not available. These counts are not pegged towards any of
the counts for AN Initiated calls.

For such calls, the Fast Connect Success rate metric (for AT/AN Initiated calls) can be derived.
It can be defined, for releases prior to Release 28 on a per AP basis as follows:

Fast Connect Requests Fast Connect Failures


Fast Connect Success rate
Fast Connect Requests

One may want to compute a composite metric for Established Connection Rate consisting of
counts for AT Initiated, AN Initiated and Fast Connect calls together at the per sector-carrier
level. However, since the pegs for Fast Connect do not have the granularity down to the sector-
carrier basis, the Established Calls rates will have to be computed separately for AT/AN Initiated
calls (per sector-carrier) and Fast Connect calls (per AP).

Many of the factors commonly impact AT/AN Initiated calls as well as Fast Connect Calls. These
are discussed in the subsequent sections.

For Release 28, a new way of computing this metric is also available. The new computation uses
the Fast Connect Established Calls peg, so the computation becomes:

Fast Connect Success Rate [ Peg ]


Fast Connect Established Calls
x100
Fast Connect Requests

Another difference with the pre-Release 28 computation is that the new Fast Connect
Established Calls peg is available at the OHM level, not at the AP level.

Notice how the new Release 28 calculation methods use the [Peg] label to distinguish their
calculation from the pre-Release 28 metrics. This is done for the Established Connection Rate
metrics as well as the Fast Connect Success Rate metric.

5.1.2 RF access failures

As the name suggests, this type of failures impacts access performance due to RF performance
limiting factors. By definition, the RF access failures influencing the deficit between Connection

ALL RIGHTS RESERVED ALCATEL-LUCENT 2007 39


1 X EV-DO RF P ER FORM ANCE A NALYS IS & T ROUB LESHOOT ING G U IDE

Request and successful call setup occur after a Traffic channel is assigned4 by the AN, but before
Traffic Channel Complete message is received from the AT. The failures represent the inability
to complete call setup due to inadequate signal strength received by the AT and/or AN.
Basically, the RF impairments break down the message flow and/or the signal acquisition
resulting in an incomplete call setup.

Call setup is a multi-stage process, and hence there are various places where RF breakdown may
occur. There are separate SM counts to trap failures (for both AN initiated and AT initiated
connections) at each stage. These are:

After sending a Traffic Channel Assignment Message to the AT, AN times out waiting to
acquire DRC channel from the AT. The SM count to trap this is called Connection
Attempt Failure Reverse link not acquired.

After AN successfully acquires AT on the reverse link, it sends an Acknowledgement


message, called RTCAck, to the AT. AT is expected to Ack it back with the TCC
message. If AN does not receive the TCC after 3 attempts of transmitting the RTCAck, it
will result in a failed call attempt. The SM count to capture this event is called
Connection Attempt Failure - No Traffic Channel Complete Message received.

After AN sends a Traffic Channel Assignment Message to the AT, a count called
Connection Attempt Failures - Other Reasons Post TCA. This is a catch-all count to
capture all other possible connection failures not captured by the previous counts.

All the above stages apply regardless of an AT or AN initiated Connection Request.


Accordingly, there are separate SM counts for connection requests (that is, AT or AN initiated
Connection Request), connection failures and separate metrics (AN initiated RF failure rate
and AT initiated RF failure rate) to track them.

The failure counts above can be used to calculate an RF Failure Rate. Prior to Release 28, the RF
Failure Rate could be mathematically expressed as follows:

RF Failure Rate
RF Failure Counts
x100
Traffic Channel Assignments Sent

As in the Established Connection Rate and Fast Connect Rate, the RF Failure Rate for R28 can
be calculated using another method:

Traffic Channel Assignments Sent Established Connections


RF Failure Rate [ Peg ] x100
Traffic Channel Assignments Sent

Both equations also have cross carrier traffic channel attempts components in them (which are
not shown in the above definition). See [25] for more information and the detailed equations.
4
From AT perspective, failure may occur much sooner but AN will not know until it times out waiting for response to
the Traffic Channel Assignment.

ALL RIGHTS RESERVED ALCATEL-LUCENT 2007 40


1 X EV-DO RF P ER FORM ANCE A NALYS IS & T ROUB LESHOOT ING G U IDE

Next, we elaborate on potential RF factors influencing the failures along with some mitigation
strategies. Their impact may be felt anywhere along the call setup process although with varying
degrees. It is not uncommon to see similar symptoms for problems induced by more than one of
the RF factors. Thus, a comprehensive understanding of all RF factors is called for to be able to
eliminate unrelated RF factors and single out the root cause.

5.1.2.1 Lack of RF coverage

Concept/Reason for failure

AT in weak coverage area (such as AT receive power less than 105 dBm and best Pilot SNR < -
10dB, mobile transmit power > 20 dBm), by definition, suffers from excess pathloss resulting in
its inability to close one or both the links. Typically, the reverse link runs out of coverage first
given that the 1xEV-DO forward link has roughly 10 dB advantage (Pilot channel transmitted at
full power). Areas suffering from heavy shadowing such as in-building locations or stretch of
route heavily blocked by building, tree, etc can likely experience this type of failure.

Failure Symptoms and Identification strategies

Such problems are best detected via drive tests. The drive test area can be narrowed to within the
coverage of a few cells by identifying cells via SM, which tend to exhibit unusual degradation in
many QoS indicators, including:
Low Established Connection Rate
High Dropped Connection Rate (defined in section 5.2)
High Page Failure rate or low Paging Effectiveness rates
High RLP re-transmission rates (ratio of RLP Octets Re-transmitted to RLP Octets
Transmitted on the forward link, ratio of Missing RLP Octets Requested to RLP
Octets Received on the reverse link these counts are collected in the TP for the DRC
pointed sector).

Suggested Mitigation Techniques

Having identified select areas, drive test based RF optimization techniques can be employed as
detailed in [8]. The process typically entails analyzing drive test data to arrive at a suitable
conclusion, such as, increasing cell power or adjusting antenna azimuth, or in the worst case
adding repeater5 (especially for in-building environments) or cell to fill in low signal strength
areas.

Improving the coverage may also result in secondary effects such as overall performance
improvement as interference is mitigated due to reduced traffic channel power requirement. Also,
see section 5.1.2.7 for more information on the Session Close Critical System Edge Metric
Preservation feature which can be used to mitigate lack of RF coverage problems.

5
Current 1xEV-DO deployments are based on regular cells; generic repeaters may first require thorough
characterization to ensure they maintain decent transmit SNR in the forward direction.

ALL RIGHTS RESERVED ALCATEL-LUCENT 2007 41


1 X EV-DO RF P ER FORM ANCE A NALYS IS & T ROUB LESHOOT ING G U IDE

5.1.2.2 Excessive Pilots/Non-dominant Pilots

Concept/Reason for failure

Presence of too many pilots in an area often impacts forward link performance due to interference
it creates to the pilot(s) being actively tracked by the AT. Typically, there is no dominant pilot in
such a problem area, that is, there are several pilots with weak comparable strengths, often
accompanied by constant pilot thrashing (dominant sector changing quickly). Another scenario is
where a distant sector overshoots to create/contribute to a localized non-dominant pilot region.

Call setups are susceptible to failure in multiple pilot regions since AT has no single strong pilot
to tune to. While AT can enter into Traffic Channel with multiple pilots in the Active set, there
may still be some degradation since AT can tune to only a single pilot at any given time on the
forward link. Any delay in completing the call set-up (for example, AT may miss the first
RTC_ack message before it tunes to a stronger pilot, the message will be resent a few frames
later) could impair the Established Connection Rate.

Failure Symptoms and Identification strategies


There are several ways to attain a pointer to the potential existence of this problem.

On a relative basis, cells exhibiting higher than normal rate of connection requests with 3 or more
Pilots provide one such indication. There are also separate SM counts to flag Connection
Requests with 2 Pilots and Connection Requests with 3 Pilots events. All these counts are
reported in the AP for the sector that received the Connection Request. [Sum of AT Initiated
Connection Requests and AN Initiated Connection Requests] minus [Sum of these multiple
pilot counts] reflect the number of Connection Requests with single pilot. The following four
metrics in SPO can verify the existence of the pilot pollution problem:
Connection Request with One Pilot (%)
Connection Request with Two Pilots (%)
Connection Request with Three Pilots (%)
Connection Request with Four or more Pilots (%)
Guidelines similar to the one used to detect pilot pollution in 2G/3G1x networks, based on SM
counts for the number of Tadd pilots reported on the PSMM, can be used. Higher than normal
rate of Soft/Softer HO fail due to maximum number of legs reached (ratio of
SO_SOR_HOFF_FAIL_MAX_NO_LEGS_REACHED and SO_SOR_HOF_ATTMPT)
when the Maximum Legs in Handoff configuration parameter is set to a value higher than 3 can
also indicate pilot pollution problems.

Another pointer, though a weak one6, is excessive activity for Soft and/or Softer handoff
attempts. It is likely that other KPIs such as Dropped Connection Rate and Data Rate may also
exhibit impairment in these areas.

6
Soft/softer handoff activity is more symbolic of mobility, but along with several other indicators tends to suggest a
potential multi-pilot region.

ALL RIGHTS RESERVED ALCATEL-LUCENT 2007 42


1 X EV-DO RF P ER FORM ANCE A NALYS IS & T ROUB LESHOOT ING G U IDE

For one-to-one overlay on a Alcatel-Lucent 2G or 3G1x network, data and tools from the
underlying 2G/3G1x infrastructure may also be employed for identification of excess-pilot areas.
Note, however, that this type of comparative analysis will be valid only if the antennas are shared
across technologies, or in physical proximity with similar attributes (azimuth, downtilt, height,
beam width)

2G/3G1x cells showing high secondary CE usage relative to Primary (or Total) CE usage,
or Total Walsh Code relative to Primary Erlangs, are potential indicators to excess pilot
regions.

Handoff Matrix and Undeclared Neighbor List analysis tools in SPO can be used on the
underlying 2G/3G1x network to flag cells overshooting from a distant region. Every
effort must be made to contain their coverage/reduce their interference in order to not
only benefit call set-up rate, but also to improve overall performance (Dropped
Connection Rate, data rate, etc).

Suggested Mitigation Techniques


Similar to the Lack of RF coverage reason, drive tests can be employed to analyze and resolve
such problem areas. Typical RF optimization techniques include adjusting cell site power and/or
antenna downtilt of one or more neighboring cells to create a dominant pilot or fewer visible
pilots. Antenna downtilt usually are more effective in controlling coverage of overshooting
sectors. Refer to [8] for more details.

5.1.2.3 Poorly constructed Neighbor Lists

Concept/Reason for failure

Not having a fine tuned neighbor list is a common source of access failures. In essence, it
prevents AT from being on the best pilot during the access process. A poor neighbor list may
typically be missing a nearby pilot to which AT will otherwise handoff to, or may have integrity
issues, such as, lack of reciprocity or suffer from PN ambiguity, discussed below.

Failure Symptoms and Identification strategies

The first check for cells with high access failures should be to review the neighbor list entries and
ensure they follow basic rules, such as other sectors of the same cell as well inward facing sectors
of the surrounding first tier cells are included as neighbors and the correct AP IP addresses are
populated for the neighbor sectors.

If the Soft Handoff Attempt to Assign rate metric is high, it typically indicates that some of the
neighbor sectors have incorrect AP IP addresses listed in the NeighborSector form.

The Neighbor List Alert (NL Alert) feature in the SPO tool can be employed effectively to fine
tune neighbor lists. The integrity can be verified in terms of:

Reciprocity - if cell X is on neighbor list of cell Y, then so should be Y on the neighbor


list of X.

ALL RIGHTS RESERVED ALCATEL-LUCENT 2007 43


1 X EV-DO RF P ER FORM ANCE A NALYS IS & T ROUB LESHOOT ING G U IDE

No PN ambiguity - cell X should not have two cells with same PN on its neighbor list7, or
if cell X and Y are in handoff, where X sponsors neighbor cell A and cell Y sponsors cell
B, A and B should not be configured with the same PN8. Otherwise, it is possible that
AT ends up establishing a traffic link on the weaker of the two cells with the same PN,
thereby, impacting access performance due to noise component brought in by the
stronger non-traffic bearing PN at the AT demodulator. The 2-way ambiguity problem is
likely to have larger impact on the dropped calls

Cross-face omission It is a good idea to include other sectors in the cell as high priority
neighbors.

AP IP alert The IP addresses of the neighbor sectors APs have to be defined correctly
in order for handoffs to occur.

With 1xEV-DO R25, Handoff Matrix (HOM) and Undeclared Neighbor List (UNL) data is
available. In addition to NL Alert analysis, the Handoff Matrix analysis and Undeclared
Neighbor List analysis reports and maps in SPO can be used to add required neighbors and
remove unneeded neighbors.

Suggested Mitigation Techniques

Fix any reciprocity issues and resolve PN ambiguity by changing PN offset of one of the two
cells with common PN. Ensure that the neighbor sectors have the correct AP IP addresses
populated.

If it is a standalone deployment or during initial optimization, it is best to shape neighbor lists


with drive tests. Neighbor list omissions can be detected for example, where a call drops and
shows up on a strong Pilot that was not in the prior Active set or the Neighbor List message sent
to the mobile. Opening the Remaining set can also help identify missing neighbors, if they are
strong enough for the AT to detect and report them, either via Route Update or Searcher info
message. Refer to [8] for more details on drive test based neighbor list refinement techniques.

With adequate traffic in the network, an add/delete neighbor analysis can be performed using the
Handoff Matrix analysis reports and maps. In addition, UNL reports and maps can be used to
study strong remaining set pilots that should either be considered to be added into the neighbor
list or considered for reduced coverage.

5.1.2.4 Improper PN planning

Concept/Reason for Failure

7
Commonly known as one-way PN ambiguity.

8
Commonly known as two-way PN ambiguity.

ALL RIGHTS RESERVED ALCATEL-LUCENT 2007 44


1 X EV-DO RF P ER FORM ANCE A NALYS IS & T ROUB LESHOOT ING G U IDE

Improper PN planning could result in PN assignments such that AT cannot resolve signal from 2
different cells. There are two common scenarios in which this problem manifests.

PN offsets are time-shifted versions of the same PN sequence. Each PN offset is 64 chips
apart. Large propagation delays, as in the case of signal from a distant overshooting cell,
can alter the identity of a PN at the AT, which in the worst case could make it
indistinguishable from a nearby cell. When AT sends Route Update message, it will
include this common PN it sees. The call will be set-up only on the close-in cell, not on
the far-away cell since it is actually transmitting on a different PN. As a result, only the
close-in cell will key up traffic. When AT attempts to demodulate traffic channel, it will
not only include valid traffic energy from the close-in cell, but also weigh in noise
component from the distant cell. Consequently, the traffic signal-to-noise ratio will nose-
dive, impacting access performance.

Another scenario is where two cells are assigned the same PN, but are not physically
separated enough such that at the AT the two cells becomes indistinguishable. This
triggers the same performance impacting phenomenon as described above in the case of
two far-way cells with different PNs.

In both cases, the problem appears because there is not enough separation in the PN offset of the
two cells. Prudent PN planning can help minimize the problem. The first scenario is mitigated by
using adequately large pilot_increment (available as Pilot PN Sequence Offset Index Increment in
the Service Node translation form) so that AT can still resolve signals from cells received with
large relative difference. Special attention should be heeded for situations where signal is
expected to propagate much farther than typical range, for example, due to low loss experienced
over water bodies, signal from elevated cell sites, etc. The second scenario requires proper PN re-
use margin.

Failure Symptoms and Identification strategies

The NL Alert feature in SPO is designed to uncover PN ambiguity issues and can potentially help
identify the problem with inadequate PN reuse margin. See the previous section (5.1.2.3) for
more details. However, this approach cannot detect all problem configurations. In such cases,
drive tests are often effective, though not simplistic, in isolating potential problem(s).

A common symptom to look for is where FER on the forward link appears poor while the mobile
receive power and Pilot Ec/Io remain at decent levels. This follows from the discussion earlier
that traffic signal to noise ratio weakens since the AT ends up demodulating noise from the cell
not in the Active set. Usually, this is a very telling indicator of two or more cells being received
with same PN (either of the two problem scenarios) assuming there are no other issues (such as
cell software/hardware defects). To confirm the problem, the cell suspected to be facing PN
interference should be turned off to verify if the Ec/Io associated with this PN still remains
strong. If it does, the next step will be to identify cells in the vicinity configured with same PN or
explore the possibility of a distant overshooting cell. Latter may require meticulous investigation
of terrain/morphology/cell elevation to identify the potential for a benign propagation
environment in the area.

Suggested Mitigation Techniques

ALL RIGHTS RESERVED ALCATEL-LUCENT 2007 45


1 X EV-DO RF P ER FORM ANCE A NALYS IS & T ROUB LESHOOT ING G U IDE

Once the failure scenario is identified, it is relatively easy to resolve the problem. For the first
scenario (overshooting cell), standard RF optimization techniques, such as antenna downtilt
and/or azimuth adjustments can be attempted to contain its coverage. The other option is PN re-
assignment, but care should be exercised to ensure it does not become a problem elsewhere.
Using a large PN increment will also mitigate the problem, though will require quite a bit of PN
planning re-work.

The second problem scenario can be resolved by maintaining sufficient number of cells between
PN re-use. A judicious design at the outset will go a long way in minimizing a potentially large
amount of effort spent resolving it later.

5.1.2.5 External Interference

Concept/Reason for Failure

Presence of interference raises the noise floor, which raises the amount of power needed to
overcome it, further elevating the noise levels. Since the cell always transmits at fixed full
power, AT may not be able to decode the signal in presence of strong interference levels on the
forward link. On the reverse link, the AT can run out of power needed to close the link in the
wake of strong external interference. In either case, the Established Connection Rate will be
sacrificed. The level of impact, and hence, the ability to detect/resolve, may depend on the
amount of interference. An intermittent/weak transmitter may often be very difficult to identify.

There are several flavors of external interference that could impact performance of a network. It
can exist on the forward or the reverse link.

An unlicensed transmission within the CDMA band is the most common cause of
external interference. Usually spectral clearance tests are performed in a new deployment
to identify and rectify such sources. If the tests were not executed, such as, when
additional spectrum is allocated to an existing operator, continued operation of
interference sources could cause significant performance issues as the commercial service
is activated on the carrier.

Interference can also be caused by spill over of a legitimate transmission from the
adjacent channel due to narrow spectral spacing, and/or, inadequate filter roll-offs
implemented by the transmitter or the receiver.

Another flavor of interference is where inter-modulation (IM) products, generated in the


receiver front-end amplifier due to a strong signal on a neighboring channel (although
within the amplifier band), cause significant performance degradation.

Failure Symptoms and Identification strategies

Detecting presence of interference is not straightforward and locating the exact source may be
even more difficult. It often requires shutting down the desired transmission (that is, turn off
service) to pinpoint the root cause. Thus, it is best to isolate external interference upfront before
pressing system into commercial system.

ALL RIGHTS RESERVED ALCATEL-LUCENT 2007 46


1 X EV-DO RF P ER FORM ANCE A NALYS IS & T ROUB LESHOOT ING G U IDE

The most telling indicator on the reverse link is much higher levels of RSSI rise, Eb/No setpoint
and RFER, all happening as a result of inflated noise floor. There are SM counts available to
inspect these per sector performance metrics RSSI (Short and Long Term Average RSSI Rise
pegged in the MCC), reverse Eb/No setpoint (Total Setpoint for Reverse Outer Loop Power
Control separate count for each reverse frame rate ranging from 0 to 153.6kbps, pegged in the
TP), and RFER (Reverse Link Frame Error and Total Reverse Link Frame counts, pegged in
the TP). Corresponding performance metrics for these peg counts available in SPO include:
RSSI (One Second Average RSSI Rise, One Second Peak RSSI Rise, Ten Second Average
RSSI Rise, and Ten Second Peak RSSI Rise), and RFER (Reverse Frame Error rate, Reverse
Retransmission Failure rate and EVM DRC Erasure rate).

From prior field experience with such type of interference in 2G/3G1x networks, several sectors
facing the interference source will exhibit a simultaneous increase in the RSSI, Eb/No setpoint
and RFER. This is one of the surest indicators of external interference. To pinpoint the problem,
more expensive/elaborate means such as use of spectrum analyzer at/around the cell site may be
called for.

On the forward link, drive testing may help provide initial pointers to the presence of external
interference. Basically, the higher interference will result in very high mobile receive power, but
poor Ec/Io (Io goes up due to interference) and poor fwd FER. This, however, may also be caused
to due to missing a neighbor or an inadequate search window size; hence, these have to be
eliminated first along with any potential hardware/software issues. Turning off some cells or
service in the affected area may be required to allow for thorough spectral clearance tests.

On either link, one way to isolate the problem related to IM interference is by using an in-line
attenuation to the mobile or the cell. If the measured signal power at either end (mobile receive
power on the forward link, or cell RSSI on the reverse link which is supposedly much higher than
the noise floor) increases at a much faster rate than the additional attenuation value, it suggests
the existence of IM products. Basically, the IM harmonics created due to amplifier non-linearity
de-emphasize at a larger rate than the additional attenuation value while the desired inband signal
attenuates at the nominal rate; therefore, the composite signal still attenuates at a larger rate.

Suggested Mitigation Techniques

As above, identifying the precise location/source of the interference is the most difficult first step.
Once the source is identified, the remedy is simply to turn it off.

5.1.2.6 Inadequate Search Window Size

Concept/Reason for Failure

There are two main search windows whose sizes are of interest Active and Neighbor sets.
When searching the PN space to track/detect pilots in its various sets, the AT utilizes a finite
window width in order to keep the search time small. The size is optimized to suit local terrain
and cell topology in order to maximize the likelihood of AT finding a pilot and its multipath
components in the smallest possible time.

Different considerations go into determining Active and Neighbor set search window size.
Accordingly, there are two failure modes.

ALL RIGHTS RESERVED ALCATEL-LUCENT 2007 47


1 X EV-DO RF P ER FORM ANCE A NALYS IS & T ROUB LESHOOT ING G U IDE

When Active set search window size is not configured large enough to allow AT to
receive/track a strong multipath component. Typically, the default search window size
is good enough to cover most of the significant components. In some rare cases,
especially in large delay spread environments, this problem may surface. AT cannot
combine/demodulate the multipath since it falls outside the search window, rendering it
as an interferer.

When AT fails to detect a neighbor due to an inadequate Neighbor search window size.
More than the presence of multipath, it is the relative path difference at the AT between
the reference and the neighbor cells that drives this window width. Since PN offset
transmitted by each cell is a time-shifted version of a single PN sequence, the difference
in propagation delays to the AT requires the Neighbor search window to be wide enough
to span the largest delay differential. Otherwise, a particular neighbor may fall outside
the search window, causing it to interfere with the Active set.

The call may fail in either case due to RF impairment if the mobile is not able to detect a strong
pilot or a multipath component, and instead attempts to setup a call on a weaker one.

When changing Active set search window size, it is usually a good idea to make the one used at
the cell site for reverse link demodulation, called Reverse Traffic Window Size, as well as for
acquiring soft handoff leg, called Data Window size, comparable. Both the translation parameters
are on the EMS GUI for BTSs.

Failure Symptoms and Identification strategies

SM counts do not provide a clear clue as to the existence of a search window problem.

For an overlay network, a basic sanity check is recommended to ensure Active and Neighbor
search window sizes are comparable to the 2G/3G1x underlay.

Drive test is often the most productive method for identifying search window issues. LDAT3G
can be used to alert and recommend proper neighbor search window sizes. Based on the mobile
logs, it employs the phase-offset information for various pilots reported on the Route Update
Message for an active call to identify potential for larger neighbor search windows. However, it
cannot identify situations where the existing neighbor search window is small enough to let AT
even detect the neighbor Pilot. Towards that end, another tool that could be of particular
significance, especially in tough to resolve problem areas, is a pilot scanner, which employs
independent GPS time reference to provide an accurate view of multipath components from each
cell.

Suggested Mitigation Techniques

Once the need for opening the search window for an Active or Neighbor set is identified, it can be
increased via translations. If a proper size is not known, it can be expanded in smallest possible
increments until performance is improved and/or mobile logs confirm proper detection at the
problem area.

ALL RIGHTS RESERVED ALCATEL-LUCENT 2007 48


1 X EV-DO RF P ER FORM ANCE A NALYS IS & T ROUB LESHOOT ING G U IDE

5.1.2.7 Edge of coverage

Concept/Reasons for failure

At the edge of a 1xEV-DO system, where 1xEV-DO coverage ends and 3G1x coverage
continues, ATs attempt session setups even in poor RF conditions. The ATs attempt to setup
sessions several times if they fail. This results in the number of connection attempts peg count to
increase resulting in a poor value for the Established Connection Rate metric.

Failure Symptoms and Identification strategies

For sectors at the edge of 1xEV-DO coverage, it is likely that the Established Connection Rate
metric is degraded. The primary cause for failure will likely be the connection attempt failures
due to no traffic channel complete received from the AT. It is also likely that a majority of the
failures in these regions are caused by connection requests associated with RATI session setup
requests. RATI session setup requests is computed as the difference between the Session setup
requests and the Inter-subnet idle transfer attempts counts.

This problem can also be identified by collecting data using the Celnet Xplorer tool for the cells
in the coverage border. Analysis reports will provide a view of how many of the connection
attempts are coming from ATs in poor coverage area and are far away from the border cells.

With R26, the Critical System Edge Metric Preservation feature provides the short-term solution
for preserving the usefulness of service measurements in poor RF coverage areas by allowing the
service provider to optionally deny connection setup requests from ATs in those areas. A long-
term solution is under discussion for a future release. At the edge of the Alcatel-Lucent
Technologies 1xEV-DO system, many ATs are autonomously attempting to start up new sessions
in extremely poor RF conditions outside of the nominal service area. These attempts use up
Reverse Link resources often taking them away from other ATs that have a better chance of
establishing a data connection. Battery life of the ATs in question is also most often needlessly
diminished. The number of such requests is sufficient to skew Service Measurements so that they
no longer provide an accurate account of the service in the valid coverage area.

This feature provides operators with the choice to deny attempts that are in poor RF conditions
and too far away (those having a negligible chance of successfully establishing a data
connection). It also provides new service measurement counts and modifies existing service
measurement counts to more accurately represent the effective coverage area. This feature only
impacts ATs that are undergoing connection setup in poor RF conditions at the edge of the
service area. Users on already established sessions are not impacted.

With R26, the AT_INIT_CONN_REQ_POOR_RF_ALLOW_SESS and


AT_INIT_CONN_REQ_POOR_RF_DENY_SESS service measurements counts measure the
number of connection requests that are allowed and denied due to poor RF conditions.

Suggested Mitigation Techniques

A R24 private load is being proposed to trial a potential solution for this problem. In this private
load, connection attempts from ATs in poor RF (poor Ec/Io values) and too distant from the cell

ALL RIGHTS RESERVED ALCATEL-LUCENT 2007 49


1 X EV-DO RF P ER FORM ANCE A NALYS IS & T ROUB LESHOOT ING G U IDE

site can be denied. If this mitigates the problem, this proposal will become part of a future
software load.

In R26, with the Session Close Critical System Edge Metric Preservation feature, operators are
allowed to deny session setup attempts in poor RF conditions and when they are too far away
from a cell site. This is done by setting the RF-Border Session Setup Deny parameter (called RF-
Border Session Setup Connection Deny prior to Release 27) to Deny Connections in Poor
Conditions, which enables this feature.

In addition to enabling the feature, operators can set the Minimum Distance Threshold for Session
Setup Request Denial and the Signal Strength Threshold for Session Setup Request Denial
parameters as needed to control the distance and the signal strength for admitting session setup
requests. See [2] for more information.

5.1.2.8 RNC border issues

Concept/Reasons for failure

ATs are required to re-register with the system as it crosses an RNC border. RNC borders are
defined by the change in color codes. In such border areas, it is likely that the ATs ping-pong
between the RNCs resulting in paging failures, misdirected data and connection failures.

Failure Symptoms and Identification strategies

For sectors at the RNC border, it is likely that the Established Connection Rate metric and the
AT/AN initiated connection attempt failure rate are degraded. However, with the current service
measurements, it is not easily possible identify this problem. Drive data analysis in the RNC
border area would provide a clear picture of the problem.

To identify RNCs that may have issues with idle handoff ping-ponging or RNC paging failures,
the following counts could be utilized: Inter-Subnet Idle Transfer Attempts, Inter-Subnet Idle
Transfer Attempts for Prior Session, Inter-Subnet Idle Transfer Failures - Insufficient
Information in AT Prior Session Configuration Request, and Number of Dangling Sessions
Closed. Additionally, look for high values in the Inter Subnet Idle Transfer Failure Rate
metric.

Suggested Mitigation Techniques

RNC borders need to be carefully engineered in order to minimize such performance problems.
The following factors need to be considered while engineering RNC borders:

Define RNC border in low data usage areas

Avoid RNC border that runs parallel to major highways/roads

Avoid RNC borders near major office buildings, water bodies and island cells (a few cells in
a RNC surrounded by cells on a different RNC)

ALL RIGHTS RESERVED ALCATEL-LUCENT 2007 50


1 X EV-DO RF P ER FORM ANCE A NALYS IS & T ROUB LESHOOT ING G U IDE

If a new RNC is needed in heavy usage area (capacity need), instead of adding new cells on
to a new RNC, re-distribute cells evenly across RNCs

Implement the RNC Grouping to Improve Idle Hand-off Performance feature, introduced in
Release 28. This feature allows combining multiple RNCs into an RNC group, so that an AT
can keep its session when it moves across RNC borders within the RNC group. This should
improve the idle handoff performance by cutting down the session transfers and eliminating
the session transfer ping-ponging for ATs moving around RNC borders within the same RNC
group, see the Release 28 release letter for more information. This feature also introduces
additional Paging mechanisms to enhance paging effectiveness, see 5.1.1.2.

5.1.3 Blocking

As the name suggests this type of failures impact access performance due to lack of resources.
By definition, any connection attempt failures that occur after receiving a Connection Request
Message and prior to sending the Traffic Channel Assignment are connection blocks. These are
AN failures such as TCA allocation failures, resource overloads or overflows, internal errors or
messaging failures.

Connection blocks do not include RF failures (there is no over-the-air interaction with the AT
once a Connection Request Message is received until after the Traffic Channel Assignment is
sent). Connection blocks also do not include any failures associated with external network
elements beyond the AN (such as AAA and PDSN). Finally, connection blocks do not include
any PPP authentication failures since PPP negotiation between the PDSN and the AT occurs on
the traffic channel after connections are already established.

The Connection blocking rate can be defined as follows, see [25] for more information and the
detailed formula:

Connection Requests Assignments Sent Session Transfer Failures


Connection Blocking Rate x100
Connection Requests Session Transfer Failures

This connection blocking rate includes all possible connection attempts including cross carrier
attempts (which are not shown in the above definition). For Release 28, a new blocking metric
that captures only connection blocking for user initiated data transmissions is introduced. This
metric is defined as:

Connection Requests Assignments Sent AT Init SessionSetup Conn Req


AT Init SessionSetup Assignments Sent
Connection Blocking Rate User x100
Connection Requests AT Init SessionSetup Conn Req

As already mentioned, connection blocking can be due to lack of resources, resource overload, or
internal errors. Some of these causes are examined next.

ALL RIGHTS RESERVED ALCATEL-LUCENT 2007 51


1 X EV-DO RF P ER FORM ANCE A NALYS IS & T ROUB LESHOOT ING G U IDE

5.1.3.1 TP overload

Concept/Reasons for failure


There is a safeguard mechanism to block further calls on the Traffic Processor (TP) as its
processor occupancy exceeds a certain threshold. This is to preserve Data performance for
existing users as the traffic served by the TP increases beyond the designed limits. Doing so will
minimize packet delays or drops as TP is unable to keep up with the load. Calls blocked due to
TP overload will result in the deficit between Connection Requests and Established calls.

Note that there are two TPs per Application Processor (AP) in the FMS, so blocking will only
occur if both of them are in overload.

The TP occupancy threshold to trigger an overload is controlled by a tunable parameter called


TP Utilization Threshold. It is currently set to 85%.

Failure Symptoms and Identification strategies

There are a host of SM counts and metrics on TP utilization. There is a direct count for Number
of New Sessions Denied due to TP overload (corresponding metric in SPO is Sessions Denied
due to TP Overload). The count is actually pegged at the AP when both its TP are in overload.
Other metrics at the TP level related to overload are Number of Times TP in Overload as well
as Total Overload Duration.

However, it is a sound practice to monitor TP Processor Usage, pegged per TP as well as the
packet drop rate (Ratio of counts on number of dropped packets and packet arrival rate, when
TP is in overload, pegged in each TP) on a regular basis to allow preemptive action.

Suggested Mitigation Techniques

Normally, if Alcatel-Lucent recommendations for system design and provisioning are followed, it
will allow proper projections to minimize TP overload. Refer to [14] for Alcatel-Lucent
recommended Network Engineering Guidelines for 1xEV-DO systems. Should such a situation
arise, however, one solution may be to re-configure and allocate fewer BTSs per AP/TP
assembly. This way each TP will serve smaller fraction of the traffic in a geographical area,
providing relief to its processor occupancy.

Over time, software/hardware upgrades may also become available to reduce the occupancy for
the same amount of serviced traffic, alleviating the need to re-configure cell to AP/TP mapping.

5.1.3.2 Maximum number of Connections or UATI Sessions reached

Concept/Reasons for failure


Currently, the scheduler in the ASIC at a BTS can support a maximum of 48 users per sector.
When the number of active users reaches 48 (including soft/softer connections), successive call
attempts are denied on the sector, lowering the Established Connection Rate.

ALL RIGHTS RESERVED ALCATEL-LUCENT 2007 52


1 X EV-DO RF P ER FORM ANCE A NALYS IS & T ROUB LESHOOT ING G U IDE

There is also another ASIC limitation that a 3-sector cell can support a maximum of 93 channels.
Although these are pooled across 3 sectors, it is possible to trigger this per-cell bottleneck before
any individual sector hits the 48-user limit.

Note that standards allow 64 MAC Indices per sector for 1xEV-DO Rel 0. After allowing for
overhead channels, only 59 are available for assignment to active users. If and when the existing
ASIC limitations are removed, the available MAC Indices will become the per-sector bottleneck.

The maximum allowed users on a sector are controlled by a translation called Maximum
Number of Users (as of Release 28, an additional parameter has been added for Rev A users as
well), refer to [1] and [5] for more information. When the cell reaches this threshold and receives
a new call attempt, it first attempts to make some room by attempting to push an active user into
dormancy if its dormancy timer is about to expire9. However, if there is no such user, the
Connection Request is rejected by sending Connection Deny message to the AT. This lowers the
Established Connection Rate.

In 1xEV-DO, a critical problem is that many ATs (mobiles) follow the logic of reconnecting to a
prior session as part of an Inter-RNC Idle Hand-off as specified by the standards. However, the
Alcatel-Lucent system did not perform this functionality, and instead brought up a new session
and stranded the older session, causing interruptions in service and blocking system resources.
This caused Established Connection Rate to degrade even in good RF coverage areas.

This has been fixed in RNC/Cell Release 24.0, with the implementation of the Prior Session
method to bring the system into compliance with the standards. New service measurements have
been added to count Prior Session metrics separately. In addition, if this feature is mapped back
into releases where service measurements cannot be added, only the non-SM part will be mapped.

Before Release 28, prior sessions that also occurred when the source and target RNCs were the
same were not handled properly. This would increase the number of session requests and increase
the probability of blocking as the system blocked any prior session requests made by an AT on
the RNC where it currently had its previous session. In Release 28 the Prior Session: Allow
Session Transfer from/to Serving RNC feature has been introduced to resolve this issue.

For the maximum number of UATI Sessions reached, you need to keep in mind that to set up a
new session, the OHM assigns UATI to ATs. Depending on traffic, this maximum number may
be reached and this would cause session setups to fail.

Failure Symptoms and Identification strategies

The bottlenecks described above are easy to spot in SM. They are measured at the sector level as
Number of Times Maximum Connections Reached.

In R24.0 RNC load, there is a software error that causes the counter for the number of active
users per sector to not be decremented and resulting in blocked calls/connections if the counter

9
There is a mechanism to release calls that have been idle for certain time and nearing their dormancy timer expiration.
The idea is that these calls are likely to go into dormancy soon, so releasing them sooner may not cause much end-user
impact while at the same time reducing interference to others.

ALL RIGHTS RESERVED ALCATEL-LUCENT 2007 53


1 X EV-DO RF P ER FORM ANCE A NALYS IS & T ROUB LESHOOT ING G U IDE

reaches the provisioned maximum. Refer to Alcatel-Lucent alert # 05-0493 for additional
information and mitigation techniques.

For identifying OHMs that may be blocking session setups because of a large number of UATI
assigned, a new peg count called Session Blocked, No UATI has been introduced in Release 28.
Additionally, it is recommended to keep an eye on the Peak Session per OHM and Peak UATI
Sessions without R-P Connections per OHM peg counts, which were also introduced in
Release 28.

In Release 28 the Prior Session: Allow Session Transfer from/to Serving RNC feature introduced
new CP Fail error codes for keeping track of prior sessions. Additionally, this feature modified
the Inter-Subnet Idle Transfer Failures - Insufficient Information in AT Prior Session
Configuration Request peg count such that it will not peg when prior session idle transfers are
done to the same RNC.

Suggested Mitigation Techniques


One solution is to reduce the coverage of the cell in an attempt to offload traffic to the
neighboring cells/sectors. This, however, assumes the neighboring cells have the room to handle
more traffic, which may not be the case all the time.

An alternative is to add multiple carriers. This is a more graceful solution than adjusting
coverage, which may not always yield desired traffic offloading. At this point, only single carrier
deployments are considered.

Where there are spectrum availability issues, adding a cell may be the last resort solution. This, of
course, requires longer lead times in order to accommodate zoning, cell installation, RF
calibration and optimization stages before turning the cell over to commercial service.

For Release 28 the Increasing the number of UATI sessions per OHM to 100K feature was
introduced to alleviate issues with session setup failures due to large number of UATI sessions
(this feature was already available in Release 27 SU3 using a tunable parameter, however Release
28 introduced the database parameters and GUI needed to allow operators to enable this feature
per RNC). This feature can help alleviate this issue; however it requires additional hardware to be
activated. See the Release 28 release letter for more information.

5.1.3.3 High Processor Occupancy rates

Concept/Reasons for failure

When equipment processor occupancy levels exceed certain thresholds, it is likely that new
connections are blocked and existing connections are dropped. There are several components in
the 1xEV-DO system for which occupancy levels can be monitored including the Application
Processor (AP), Traffic Processor (TP), Line Interface Unit (LIU), Forward Link Modem (FLM)
and Reverse Link Modem (RLM).

In order to improve the performance of the RLM, the DRC per rate peg counts have been disabled
in R23.0. A new implementation to peg these counts without impacting the RLM is being

ALL RIGHTS RESERVED ALCATEL-LUCENT 2007 54


1 X EV-DO RF P ER FORM ANCE A NALYS IS & T ROUB LESHOOT ING G U IDE

devised and is likely to be fixed as an MR in release 24.0. Cell sites loaded with older releases
would still be pegging the DRC per rate counts.

Failure Symptoms and Identification strategies

With R23, there are several service measurement counts that have been introduced to monitor the
occupancy levels. High values for these counts over several days could potentially result in
blocked and dropped calls. These counts include:

AP Peak Processor Occupancy (AP_PEAK_PROC_OCCUPANCY)

AP Average Processor Occupancy (AP_AVG_PROC_OCCUPANCY)

LIU Peak Processor Occupancy (LIU_PEAK_PROC_OCCUPANCY)

LIU Average Processor Occupancy (LIU_AVG_PROC_OCCUPANCY)

FLM Peak Processor Occupancy (FLM_PEAK_PROC_OCCUPANCY)

FLM Average Processor Occupancy (FLM_AVG_PROC_OCCUPANCY)

RLM Peak Processor Occupancy (RLM_PEAK_PROC_OCCUPANCY)

RLM Average Processor Occupancy (RLM_AVG_PROC_OCCUPANCY)

With R25, there are additional peg counts in support of the 1xEV-DO cell processor overload
control feature. High values for the following counts could result in blocked and dropped calls:

LIU_Overload_Num

LIU_Overload_Duration

FLM_Overload_Num

FLM_Overload_Duration

RLM_Overload_Num

RLM_Overload_Duration

The Peak_TP_Utilization peg count in R25 provides the maximum TP processor occupancy
rate.

Suggested Mitigation Techniques

If the AP or TP occupancy levels are high and with high traffic volume, consider re-homing the
cell site to another AP with lower occupancy levels (this may require re-homing cells to another
RNC frame). If LIU, FLM and/or RLM occupancy levels are high, consider off-loading traffic
from the cell in question to surrounding cell sites.

ALL RIGHTS RESERVED ALCATEL-LUCENT 2007 55


1 X EV-DO RF P ER FORM ANCE A NALYS IS & T ROUB LESHOOT ING G U IDE

5.1.4 Failures due to faulty hardware

Concept/Reason for failure

Presence of faulty or mis-configured hardware can influence the performance on the affected cell
or neighboring cells. Hardware failures can occur in various forms and intensity. For example, a
CRC or EVM board entering into faulty operation registers sudden increase in access failures. At
times, the affected hardware device may only show one errant behavior, such as, increase in call
setup failures, but other indicators such as dropped call rate remain sane.

A different flavor of hardware problem could be a faulty RF antenna cable with high return loss
likely resulting in smaller than desired cell footprint. Or an RF Amplifier operating nonlinearly
that is causing RF interference. In many cases, the hardware simply goes into a faulty state, the
hardware itself is fine software restore or stable clear rectifies the problem.

It is also possible to witness intermittent hardware outage instead of an outright malfunction, such
as loss of T1/E1 during certain hours.

Failure Symptoms and Identification strategies

A first check should involve careful review of alarms generated and flagged by fault management
module of the network. Certain issues associated with receive path (such as RSSI imbalance
across main and diversity paths), malfunctioning hardware devices, backhaul outages, etc. can
easily be identified through the alarms. These alarms can be correlated with service measurement
data for the same time frame. Positive correlation would confirm that the performance
degradation was to the faulty hardware.

The ROP analysis report in SPO provides information on these alarms at the AP, TP and BTS
levels. A pareto analysis of call setup failures by specific hardware type based on the ROP data
can offer a tell-tale sign of issues with a specific hardware/cell based on the over-abundant
concentration of the failures. To make this type of analysis meaningful, however, care should be
taken to normalize the failures with the amount of carried traffic.

Suggested Mitigation Techniques

Once the suspected malfunctioning hardware is identified, the first step is to attempt to rectify
through a remote software restore or download. If the performance continues to suffer, it may
require cell visit to reseat the hardware and in some cases, to ensure tight cable connection. If all
on-site diagnostics fail, the last step will be to replace the hardware.

5.1.5 Heavy Reverse Link traffic


Concepts/Reasons for failure

As the reverse link load increases, either due to higher number of active users or increased data
upload activity, the overall interference floor seen by each cell increases. In order to overcome
the raised interference floor, AT has to transmit at much higher power levels. An AT already at
the cell edge can no longer close the reverse link as its signal arrives with a weak signal-to-noise
ratio at the cell. This would sacrifice access performance. In many cases, the access probe itself

ALL RIGHTS RESERVED ALCATEL-LUCENT 2007 56


1 X EV-DO RF P ER FORM ANCE A NALYS IS & T ROUB LESHOOT ING G U IDE

may not be heard at the cell; as a result, no SM counts will be pegged, but end-user experience
will degrade.

Failure Symptoms and Identification strategies

The indication of high reverse overload condition can be inferred by increase in several SM
metrics, relative to normal levels observed on other sectors

High values for peg counts introduced in R23 to measure Reverse Link Overload Control
including Short Term Average Fast Control Low Load count
(SM_SHTM_AVGLLC_FASTCTRL), Short Term Average Fast Control Medium
Load count (SM_SHTM_AVGMLC_FASTCTRL) and Short Term Average Fast
Control High Load count (SM_SHTM_AVGHLC_FASTCTRL). These peg counts
are available on a per sector basis.

High Reverse Frame Error rate (ratio of Reverse Frame Error and Total Reverse
Frame counts, pegged in the TP)

High average setpoint for various rates (ratio of Total Setpoint for Reverse Outer Loop
Control and Total Number of Frames for Reverse Outer Loop Control at each rate,
pegged in the TP)

Increase in Number of Connections Force Released and Abnormal Release rate,


pegged in the MCC

Higher values for One Second and Ten Second Average/Peak RSSI Rise, pegged in
the MCC

Large number of Total Connection Requests per hour, pegged in the AP

Some of these indicators may also be influenced by reasons other than heavy reverse traffic, such
as:

External interference resulting in elevated RSSI levels, high RFER and high average set
points.

A malfunctioning device in the RF receive path, such as UCR or low noise amplifier,
causing consistent or intermittent RSSI spikes, thereby, reducing the coverage.

A component breakdown in the RF receive path, or improper antenna cable connection,


incapacitating one of the two diversity branches. This would tend to increase the Eb/No
requirement as well as the power control variation. Not only will this require increase in
power transmitted by the mobile, but the variation will also push the cell further into
instability.

Hence, careful investigation of all possible causes may be required before a particular one can be
confirmed. Though not always necessary, one distinction may be that there are large number of
users consistently on the cell, as can be judged from Total Connection Requests per hour (sum
of AT and AN Initiated Connection Requests). It is less likely that just a few users consistently
load up the cell, usually a pointer, to external interference or faulty hardware.

ALL RIGHTS RESERVED ALCATEL-LUCENT 2007 57


1 X EV-DO RF P ER FORM ANCE A NALYS IS & T ROUB LESHOOT ING G U IDE

Suggested Mitigation Techniques

It is always a sound practice to track/trend various performance indicators, especially, Connection


Requests and short/long term average/peak RSSI rise to be able to take a timely corrective action
instead of waiting till the last moment when performance is impacted.

Once the cell is identified as a heavy traffic-bearing cell impacting access performance due to
reverse link loading or overload, there are a few options available to offload the traffic:

Reduce the overload control thresholds to lower the reverse transmission rates. With
R23, the HDR Reverse Link Overload Control (HROC) tunable parameters have
become translation parameters. The Alcatel-Lucent Reverse Overload Control
mechanism utilizes these translatable thresholds to manage the integrity of reverse
link performance. Refer to [5] for an overview of operation and associated
translations. Lowering these thresholds will reduce the probability that AT will
transmit at a higher rate, under load. Lower transmission rates require less AT
transmit power on an average. In essence, reverse overload will kick-in at lighter
loading levels thereby mitigating coverage reduction at the expense of data rate
performance on the reverse link.

This is not a preferred solution in the long-term. It may, however, be acceptable as a


stopgap measure instead of denying service to users until longer-term measures, stated
next, are pursued. However, the customer must be made aware of the trade-offs
before attempting any threshold adjustments.

One solution is to increase Pilot overlap such that other links get into the active set of
the call, mitigating the traffic channel power requirement lowering the overall
interference. From the perspective of mitigating AT transmit power requirement,
higher handoff diversity should never be undesirable. However, careful consideration
is warranted to characterize potential impact on the forward link performance as well
as increase in cell resource utilization (e.g., channel element hardware).

Add a 1xEv-DO carrier. In general, assuming the service provider has the required
spectrum, this is the most common approach used to offload traffic used in 2G/3G1x
networks.

Add a cell to serve the heavy traffic area. If adding a carrier is not a possibility due to
lack of spectrum, adding a cell (geographically separated from the high traffic cell(s))
is often an effective, although a long-lead, solution to improve coverage and hence,
permit higher levels of offered traffic per unit area. Care must be taken to ensure
proper RF optimization is performed to minimize excessive pilot with cells in the
vicinity otherwise it may impact forward link performance.

ALL RIGHTS RESERVED ALCATEL-LUCENT 2007 58


1 X EV-DO RF P ER FORM ANCE A NALYS IS & T ROUB LESHOOT ING G U IDE

5.1.6 Traffic Channel Activation failure

Concept/Reasons for failure

This represents a traffic channel allocation failure at the time of call setup. In essence, it can be
construed as Channel element initialization failure. AN will send a Connection Deny message to
the AT upon encountering the failure. The problem is likely to be due to some type of hardware
failure in the EVM board at the cell site. Note that this case is different than when cell has
reached the Maximum Number of Users limitation.

Failure Symptoms and Identification strategies

This type of failure is accumulated into a SM count called Traffic channel Allocation Failure
for setup Resources Not Available in the Candidate Sector. The count is pegged in the AP
for the candidate sector not able to allocate traffic channel. Note that this count is not separated
for AT and AN Initiated call attempts. It is also important to note that the Connection Request
will only be denied if all sectors listed in the Route Update message sent along with the
Connection Request message cannot allocate any resources.

It is also possible that ROP logs this failure, which could help with easier identification.

Suggestion Mitigation Strategies

When there is a lot of traffic channel activation failures concentrated on a particular CE or


FLM/RLM, one corrective action will be to stable clear the cell. If that doesnt help, running
remote diagnostics or reloading the card is suggested. If that does not help, site visit is called for.
First, reseat the card manually and bring it to back to service. If the problem persists, as a last
resort, replace the affected channel element hardware.

5.1.7 Authentication error

Concept/Reasons for failure

There are two stages of authentication. The first one is for new session that involves
authentication with the PDSN and AAA as part of PPP setup. Any failures are logged in the
AAA. This information exchange occurs after the traffic channel has been established; so these
failures do not impact Established Connection Rate.

ALL RIGHTS RESERVED ALCATEL-LUCENT 2007 59


1 X EV-DO RF P ER FORM ANCE A NALYS IS & T ROUB LESHOOT ING G U IDE

The second stage of authentication is of more interest, since it directly influences the Established
Connection Rate. Once a session is successfully established, each subsequent transition from
dormant to Active state undergoes this authentication stage using the protocol parameters set at
the beginning of the session. In essence, this is authentication of the access channel and it is
performed before any traffic channel is assigned. The purpose is to validate that the requesting
AT is the true owner of the session previously established.

Note that this stage does not involve much message exchange. Security information from the
MAC header of the Connection Request message is decoded and validated at the AN. As a result,
any failure is more likely to be due to a real authentication failure rather than RF reasons. In this
case, AN sends a Connection Deny message instead of Traffic Channel Assignment message. It
contributes to the deficit between Connection Requests and Traffic channel assignments.

Failure Symptoms and Identification strategies

There are several counts and metrics to indicate authentication failure during various sub-phases
of the access layer authentication. These are pegged on the TP. These include:

RAN Authentication Failure rate

Number of Security Protocol Negotiation Failures

Number of Session Key Length Negotiation Failures

Number of key Response message timeouts

Number of Session Security Digest Mismatches

Number of Session Security ATKeyComplete message time-outs

Number of packet discards in SHA-1 Authentication Protocol

Number of Authentication Failures for postponed Security packets on old


1XEV_call Control

Number of authentication failures for postponed security packets on new 1XEV-


Call Control

Number of Encryption Attribute negotiation failures

Given the lack of per-sector granularity, the impact of authentication failures on the Established
Connection Rate on a sector basis is difficult to establish. In any event, it is important to
determine if particular TP(s) is experiencing unusual amount of authentication failures in an
attempt to curb them.

Suggested Mitigation Techniques

Once an authentication breach is suspected, first step should be to determine the type of AT. In
any event, the goal is to detect any incompatibility between the AT and AN. This can be
confirmed if the failures are pre-dominantly occurring on the same make/model of ATs.

ALL RIGHTS RESERVED ALCATEL-LUCENT 2007 60


1 X EV-DO RF P ER FORM ANCE A NALYS IS & T ROUB LESHOOT ING G U IDE

After the failures are limited to a few ATs, a logical step will be to turn in their identification to
the operator for further action.

ALL RIGHTS RESERVED ALCATEL-LUCENT 2007 61


1 X EV-DO RF P ER FORM ANCE A NALYS IS & T ROUB LESHOOT ING G U IDE

5.2 Dropped Connection Rate

This metric is commonly viewed as a measure of unintended interruptions during a Data session,
primarily due to loss of RF link or handoff failures. Prior to Release 28, this metric was defined
mathematically like so,

Dropped Connection Rate


Unintended Connection Releases
Connection Requests Connection Failures

As of Release 28, this metric has been changed to use the new Established Connection counts, so
the Dropped Connection Rate for Release 28 and later releases can be defined as:

Dropped Connection Rate [ Peg ]


Unintended Connection Releases
Established Connections

As of Release 26 and later, the connection requests and failures are modified to take into account
the cross carrier attempts, so both definitions also have cross carrier components, see [25] for the
detailed equations.

As defined above, it is the ratio of unintended or abnormal connection releases to the total
number of Established Connections. Established Connections are those that were assigned Traffic
channel and came up stable on the traffic channel, that is, they completed call setup process
successfully. The metric can be computed on a per-carrier basis.

There are two SM counts, the sum of which provides the total number of dropped calls or the
unintended connection releases. Both these counts are collected in the TP and pegged on the DRC
pointed sector at the time of call drop. The counts are:

Connection Released RF Link Lost this happens when AN declares loss of RF link
when all the legs lose lock on the reverse link DRC (see below)

Connection Released Other Reasons as the name suggests, there could be several
reasons for this, such as, call states not in sync between the AP and TP, cell cannot
build/send messages, timeouts, internal software error, etc.

Soft/Softer Handoff Failures No AT Response this represents connection releases


due to failure to receive the Traffic Channel Confirmation message from the AT to the
Traffic Channel Assign message sent by the AN to add or drop a handoff leg. As of
Release 26, this count is no longer used in the Dropped Connection Rate definition as it is
included as part of the Connection Released Other Reasons count.

The term Established Connections is simply the numerator in the metric for Established
Connection Rate discussed in section 5.1.

It must be noted that R27 SU1 changes RNC to handle the connection close by closing the
connection. This is expected to release the call drop rate but since the call is not considered
established and therefore will increase the failed connection attempts. In R27 SU0, this scenario
is pegged as a drop call. The connection close is considered as TCC received and therefore the

ALL RIGHTS RESERVED ALCATEL-LUCENT 2007 62


1 X EV-DO RF P ER FORM ANCE A NALYS IS & T ROUB LESHOOT ING G U IDE

call is considered as established. In R27 SU1, compared to R26, the AT/AN initiated
connection attempts failed due to no TCC received peg count is expected to decrease.
Compared to R27 SU0, the AT/AN initiated established connections as well as the Connection
Release due to RLL are expected to decrease.

A connection may be dropped in one of the following common ways:

Cell can no longer decode DRCs. When cell receives 4 or fewer good DRCs in any 5-
second sliding interval, it declares loss of RF link and drops the call. If there are multiple
legs supporting the call, all legs must lose the lock on the DRCs before the call is
released. This can occur under following conditions:

o Although AT is transmitting, cell cannot hear the mobile (due to pathloss,


external interference, mobile has tuned away to receive a voice call on 3G1x or
has spent too much time on a particular 3G1x search in hybrid mode11, etc.).

o AT can no longer decode the forward link and cannot recover before the DRC
supervision timer expires (240ms), and stops transmitting.

o AT times out, waiting for response to an Ack required message (such as Route
Update Message or Traffic Channel Complete message) even after transmitting it
3 times, and stops transmitting.

Handoff failure. Cell sends a Traffic Channel Assignment message to indicate change in
the Active set (drop or add a Pilot) to the AT. AT responds with Traffic Channel
Complete (TCC) message. However, if the cell times out waiting for the TCC, it de-
allocates the traffic channel to the mobile. This is another flavor of dropped call12 and
currently gets pegged as Soft/Softer Handoff Failures No AT Response13. A separate
metric Dropped Call rate due to HO failures has been defined in SPO to measure the
effect of handoff failures.

In all of the above mechanisms, cell sets the FT_valid field in the Quick_config message sent
on the forward link Control channel to 0, thereby, signaling the phone to get off the traffic
channel.

11
In hybrid mode, AT periodically searches 3G1x while in an active 1xEV-DO connection. If it receives an incoming
voice termination, it will take the voice call and not tune back to 1xEV-DO until after the voice call is released. At
times, it may spend a long time on 3G1x, such as, in weak 3G1x coverage areas or when performing idle handoffs,
autonomous registrations, etc. This could manifest as a Lost call on the 1xEV-DO system. A future enhancement will
attempt to remove such tune-away induced releases from Lost call peg by estimating ATs designated 3G1x page slot
and the time it disappears from 1xEV-DO relative to this slot.

12
A future enhancement at the cell may include ignoring the TCC timeout. Instead the cell will keep repeating TCA
until it receives TCC from the mobile, or the DRC supervision timer at the mobile expires. The latter will lead to cell
not being able to decode the DRCs and forcing it to release the call. This follows from the line of thinking that a call
should not be torn down due to failed handoff; if the RF link impairment persists, the call should be dropped
subsequently when the RF link is lost.

13
In a future release, a separate SM peg count will be available to represent connection releases due to handoff failures.
At that time, it can replace the soft/softer handoff failure count in the dropped call equation.

ALL RIGHTS RESERVED ALCATEL-LUCENT 2007 63


1 X EV-DO RF P ER FORM ANCE A NALYS IS & T ROUB LESHOOT ING G U IDE

Given that the AT always listens to only one Pilot any given time on the forward link, it results in
a make before break type transition when the AT changes the serving sector. This is called
Virtual Soft handoff and is unique to 1xEV-DO. One may think the lack of full soft handoff
diversity may increase the likelihood of handoff failures and drop calls. However, 1xEV-DO AT
has the ability to switch to a dominant sector much faster, compensating for the lack of soft
handoff continuity.

Note that, to a Data user, a dropped call itself may not inhibit experience so much as it does to a
voice user. This is because either the network or the AT may be able to re-establish the
connection soon after a Drop call without requiring end-user intervention. This is a feature
available on many wireless Data systems including 1xEV-DO.

A re-activation upon drop is possible, especially, if cell or RF conditions are conducive for a
successful call setup. For example, a neighbor list omission can result in a dropped call, but the
AT can quickly acquire Sync on this strong Pilot leading to successful call setup/resumption of
data transfer. However, if the dropped call was due to lack of RF coverage or cell problem, re-
activation will likely fail.

One can argue that the inherent ability to re-establish connection makes the Dropped Connection
Rate less significant for Data systems. Nevertheless, this is an important metric tracked by Data
service operators for several reasons:

Carry over of mindset from voice networks that drop calls offer tangible view of
performance interruption on an otherwise stable call.

Drop calls are often reflective of prevailing poor RF conditions, which will impact Data
performance in other ways (e.g., low data rate).

Some drop calls may not necessarily result in successful re-activation, especially in areas
of poor RF coverage or cell software/hardware malfunction.

Perceived delay due to drop/successive call setup may be unacceptable, especially, for
certain time-critical applications, such as voice over IP, or audio/video streaming.

Next we discuss each of the potential contributors to Dropped Connection Rate in detail. Many
of these are similar in nature to those responsible for impact on the Established Connection Rate.

5.2.1 Inter Frequency Handoffs

As of Release 26, 1xEV-DO systems support multiple carriers. Multiple carrier deployments
can be provisioned to support inter-frequency handoffs. The Support for Multiple 1xEV-DO
Carriers Inter-Frequency Handoff (IFHO) feature provides the capability to perform
handoffs of an active 1xEV-DO connection from one carrier to a different carrier.

With this feature, an AT moving into a sector that does not have the current serving carrier can
handed off to a new carrier without dropping the connection. Additionally this feature
introduced configuration parameters used to provision inter-frequency handoffs. Some of the
changes include modifications to the neighbor list to indicate all carriers supported by each

ALL RIGHTS RESERVED ALCATEL-LUCENT 2007 64


1 X EV-DO RF P ER FORM ANCE A NALYS IS & T ROUB LESHOOT ING G U IDE

neighbor sector and thresholds used for supporting Directed and Mobile Assisted Infer
Frequency Handoffs (MAIFHO).

Directed Inter frequency handoffs can be used to support ATs which are not able to carry out
Off Frequency Search (OFS) measurements. For these ATs, the AN can force an inter
frequency handoff provided some thresholds are met. For those ATs that support OFS
measurements, the MAIFHO algorithm uses actual AT provided measurements of the other
carrier along with thresholds to perform the inter frequency handoffs, see [6] and [27] for
more information.

5.2.2 Lack of RF coverage

Concept/Reason for failure

AT in weak coverage area (such as AT receive power less than 105 dBm and best Pilot SNR < -
10dB, mobile transmit power > 20 dBm), by definition, suffers from excess pathloss resulting in
its inability to close one or both the links. Typically, the reverse link runs out of coverage first
given that the 1xEV-DO forward link has roughly 10 dB advantage (Pilot channel transmitted at
full power). Areas suffering from heavy shadowing such as in-building locations or areas
blocked by buildings, trees, etc. can likely experience this type of failure.

A drop call due to insufficient RF coverage is generally preceded by high frame error activity on
one or both the links, and increased mobile transmit power.

Failure Symptoms and Identification strategies

Cells suffering from lack of RF coverage tend to exhibit simultaneous degradation in several
performance metrics. Besides Dropped Connection Rate, these include Established
Connection Rate, Page Failure rate, Reverse Frame Error rate, RLP re-transmission rates
(ratio of RLP Octets Re-transmitted to RLP Octets Transmitted on the forward link; ratio of
Missing RLP Octets Requested to RLP Octets Received on the reverse link, all these counts
are pegged in the TP), and data rate (discussed in section 5.3) performance. Hence, it requires
correlating several performance indicators to identify the root cause.

Even when the aforementioned metrics show a correlated degradation, it may not be
straightforward to establish lack of RF coverage as the root cause for dropped calls. The problem
may be best detected and resolved via drive tests. The drive test area can be whittled down to the
coverage of a few cells by identifying suspect cells via SM.

For one-to-one overlay on a 2G/3G1x network, comparison with the underlying network
performance as well as review of the existing coverage information can help isolate the root cause
quicker. For example, weak coverage areas often exhibit high average transmit power per user on
the forward link. Of course, other possible factors discussed in the following sections (Pilot
pollution, Neighbor list/search window issue, Island mode problem) will have to be eliminated
first.

Suggested Mitigation Techniques

ALL RIGHTS RESERVED ALCATEL-LUCENT 2007 65


1 X EV-DO RF P ER FORM ANCE A NALYS IS & T ROUB LESHOOT ING G U IDE

Having identified specific areas, drive test based RF optimization techniques can be employed as
detailed in [8]. The process typically entails analyzing drive test data to arrive at a suitable
conclusion, such as, increasing cell power or adjusting antenna azimuth, or in the worst case
adding repeater14 (especially for in-building environments) or cell to fill in low signal strength
areas.

Improving the coverage typically has a positive effect on the overall performance. This also
follows from the claim above that lack of RF coverage impacts several other performance
indicators. Improvement in coverage provides higher link margin to combat RF performance
inhibitors such as shadowing, fading, etc.

5.2.3 Excessive pilots/Non-dominant pilots

Concept/Reason for failure

Presence of too many pilots in an area often impacts forward link performance due to interference
it creates to the pilot(s) being actively tracked by the AT. Typically, there is no dominant pilot in
such a problem area, that is, there are several pilots with comparable strengths, all weak, often
accompanied by constant pilot thrashing (dominant sector changing quickly). Another scenario is
where a distant sector overshoots to create/contribute to a localized non-dominant pilot region.

Calls are susceptible to getting dropped in multiple pilot regions since AT has no single strong
pilot to tune to. Any dropped calls will primarily be due to inability to complete handoff
sequences due to break in the signaling.

Failure Symptoms and Identification strategies


There are several ways to attain a pointer to the potential existence of this problem.

On a relative basis, cells exhibiting higher than normal rate of connection requests with 3 or more
Pilots provide one such indication. There are also separate SM counts to flag Connection
Requests with 2 Pilots and Connection Requests with 3 Pilots events. All these counts are
reported in the AP for the sector that received the Connection Request. [Sum of AT Initiated
Connection Requests and AN Initiated Connection Requests] minus [Sum of these multiple
pilot counts] reflect the number of Connection Requests with single pilot. The following four
metrics in SPO can verify the existence of the pilot pollution problem:
Connection Request with One Pilot (%)
Connection Request with Two Pilots (%)
Connection Request with Three Pilots (%)
Connection Request with Four or more Pilots (%)
Guidelines similar to the one used to detect pilot pollution in 2G/3G1x networks, based on SM
counts for the number of Tadd pilots reported on the PSMM, can be used. Higher than normal

14
Repeaters have not been applied to 1xEV-DO deployments so far and should be verified for their transmit signal to
noise level integrity.

ALL RIGHTS RESERVED ALCATEL-LUCENT 2007 66


1 X EV-DO RF P ER FORM ANCE A NALYS IS & T ROUB LESHOOT ING G U IDE

rate of Soft/Softer HO fail due to maximum number of legs reached (ratio of


SO_SOR_HOFF_FAIL_MAX_NO_LEGS_REACHED and SO_SOR_HOF_ATTMPT)
when the Maximum Legs in Handoff configuration parameter is set to a value higher than 3 can
also indicate pilot pollution problems.

Another pointer, though a weak one15, is excessive activity for Soft and/or Softer handoff
attempts. It is likely that other KPIs such as Established Connection Rate and Data Rate may also
exhibit impairment in these areas.

For one-to-one overlay on a Alcatel-Lucent 2G or 3G1x network, additional data and tools from
the underlying 2G/3G1x infrastructure may also be employed for identification of excess-pilot
areas. Note, however, that this type of comparative analysis will be valid only if the antennas are
shared across technologies, or in physical proximity with similar attributes (azimuth, downtilt,
height, beam width).

2G/3G1x cells showing high secondary CE usage relative to Primary (or Total) CE usage,
or Total Walsh Code relative to Primary Erlangs, are potential indicators to excess Pilot
regions.

Handoff Matrix and Undeclared Neighbor List analysis tools in SPO can be used on the
underlying 2G/3G1x network to flag cells overshooting from a distant region. Every
effort must be made to contain their coverage/reduce their interference in order to not only
benefit call set-up rate, but also to improve overall performance (Dropped Connection
Rate, data rate, etc).

Suggested Mitigation Techniques


Similar to the Lack of RF coverage reason, drive tests can be employed to analyze and resolve
such problem areas. Best way to resolve the problem is to attend to the root cause that is, to
mitigate pilot pollution. Typical RF optimization techniques for this include adjusting cell site
power and/or antenna downtilt of one or more neighboring cells to create a dominant pilot or
fewer visible pilots. Antenna downtilt usually are more effective in controlling coverage of
overshooting sectors. Refer to [8] for more details.

Many customers prefer to increase the Maximum Legs in Handoff to 6 to reduce dropped call
rate in such problem areas. Larger Active set not only increases the likelihood of having a pilot at
ATs disposal when it momentarily becomes dominant16, it also tends to mitigate reverse link
power. This, however, comes at the expense of increased resource utilization on either link. On
the forward link, the cost is mostly limited to increase in MAC Index usage, on the reverse link
the impact is in terms of increase in CE hardware and backhaul occupancy. Increasing the

15
Soft/softer handoff activity is more symbolic of mobility, but along with several other indicators tends to suggest a
potential multi-pilot region.

16
In 1xEV-DO, since AT tunes to only one Pilot at a time on the forward link, there is no increase in combining
diversity due to a larger Active set. However, a larger Active set may still be beneficial in some cases. For example, it
is better to keep a bouncing Pilot in the Active set longer, potentially saving the call when it momentarily becomes
strong rather than let it swap in and out of the Active set. The latter could result in a dropped call if there is break in
handoff messaging due to marginal RF conditions.

ALL RIGHTS RESERVED ALCATEL-LUCENT 2007 67


1 X EV-DO RF P ER FORM ANCE A NALYS IS & T ROUB LESHOOT ING G U IDE

Maximum Legs in Handoff translation (Service Node/General section of EMS GUI) may also
serve as a stopgap measure while more resource intensive changes such as antenna adjustments
are being planned.

5.2.4 Poorly constructed neighbor lists

Concept/Reason for failure

Not having a fine tuned neighbor list is a common source of dropped call failure. A poor
neighbor list may typically be missing a nearby pilot to which AT will otherwise handoff to, or
may have integrity issues, such as, lack of reciprocity or suffer from PN ambiguity, discussed
below. In essence, any of these neighbor list issues renders a nearby pilot a strong source of
interference to the Active set pilots, causing a call to drop.

Failure Symptoms and Identification strategies

The first check for cells with high dropped call rate should be to review the neighbor list entries
and ensure they follow basic rules, such as other sectors of the same cell as well inward facing
sectors of the surrounding first tier cells are included as neighbors and the correct AP IP
addresses are populated for neighbor sectors.

If the Soft Handoff Attempt to Assign rate metric is high, it typically indicates that some of the
neighbor sectors have incorrect AP IP addresses listed in the NeighborSector form.

The Neighbor List Alert (NL Alert) feature in the SPO tool can be employed effectively to fine
tune neighbor lists. The integrity can be verified in terms of:

Reciprocity - if cell X is on neighbor list of cell Y, then so should be Y on the neighbor list of
X.

No PN ambiguity - cell X should not have two cells with same PN on its neighbor list17, or if
cell X and Y are in handoff, where X sponsors neighbor cell A and cell Y sponsors cell B, A
and B should not be configured with the same PN18. Otherwise, it is possible that AT ends up
establishing a traffic link on the weaker of the two cells with the same PN, thereby, impacting
access performance due to noise component brought by the stronger non-traffic bearing PN at
the AT demodulator.

Cross-face omission It is a good idea to include other sectors in the cell as high priority
neighbors.

AP IP alert The IP addresses of the neighbor sectors APs have to be defined correctly in
order for handoffs to occur.

17
Commonly known as one-way PN ambiguity.

18
Commonly known as two-way PN ambiguity.

ALL RIGHTS RESERVED ALCATEL-LUCENT 2007 68


1 X EV-DO RF P ER FORM ANCE A NALYS IS & T ROUB LESHOOT ING G U IDE

With 1xEV-DO R25, Handoff Matrix (HOM) and Undeclared Neighbor List (UNL) data is
available. In addition to NL Alert analysis, the Handoff Matrix analysis and Undeclared
Neighbor List analysis reports and maps in SPO can be used to add required neighbors and
remove unneeded neighbors.

Neighbor list issues often result in call drags before eventually causing call drops. Depending
on the severity of the impact, this may manifest in terms of high RSSI levels on the approaching
cell as the AT power ramps up in the absence of connection to this cell.

Suggested Mitigation Techniques

Fix any reciprocity issues and resolve PN ambiguity by changing PN offset of one of the two
cells with common PN. Ensure that the neighbor sectors have the correct AP IP addresses
populated.

If it is a recent deployment without a 2G/3G1x overlay or during initial optimization, it is best to


shape neighbor lists with drive tests. Neighbor list omissions can be detected for example, where
a call drops and shows up on a strong Pilot that was not in the prior Active set or the Neighbor
List message sent to the mobile. Opening the Remaining set can also help identify missing
neighbors, if they are strong enough for the AT to detect and report them, either via Route Update
or Searcher info message. Refer to [8] for more details on drive test based neighbor list
refinement techniques.

With adequate traffic in the network, an add/delete neighbor analysis can be performed using the
Handoff Matrix analysis reports and maps. In addition, UNL reports and maps can be used to
study strong remaining set pilots that should either be considered to be added into the neighbor
list or considered for reduced coverage.

5.2.5 Improper PN planning

Concept/Reason for Failure

Improper PN planning could result in PN assignments such that AT cannot resolve signal from 2
different cells. There are two common scenarios in which this problem manifests.

PN offsets are time-shifted versions of the same PN sequence. Each PN offset is 64 chips
apart. Large propagation delays, as in the case of signal from a distant overshooting cell, can
alter (shift) the identity of a PN at the AT, which in the worst case could make it
indistinguishable from a nearby cell. When AT sends Route Update message, it will include
this common PN it sees. The call will be setup only on the close-in cell, not on the far-away
cell since it is actually transmitting on a different PN. As a result, only the close-in cell will
key up traffic. When AT attempts to demodulate traffic channel, it will not only include
valid traffic energy from the close-in cell, but also weigh in noise component from the
distant cell. Consequently, the traffic signal-to-noise ratio will take a hit, impacting drop call
performance.

Another scenario is where two cells are assigned the same PN, but are not physically
separated far enough such that they become indistinguishable at the AT. This produces the
same performance-limiting phenomenon as described above.

ALL RIGHTS RESERVED ALCATEL-LUCENT 2007 69


1 X EV-DO RF P ER FORM ANCE A NALYS IS & T ROUB LESHOOT ING G U IDE

In both the cases, the problem appears because there is not enough separation between the PN
offset of the two cells. Prudent PN planning can help minimize the problem. Using adequately
large Pilot_increment (available as Pilot PN Sequence Offset Index Increment in the Service Node
translation form) helps mitigate the first scenario by allowing the AT to resolve signals from two
distant cells. Special attention should be heeded for situations where signal is expected to
propagate much farther than typical range, for example, due to low loss experienced over water
bodies, signal from elevated cell sites, etc. The second scenario requires proper PN re-use margin.

Failure Symptoms and Identification strategies

The NL Alert feature in SPO is designed to uncover PN ambiguity issues and can potentially help
identify the problem with inadequate PN reuse margin. See the previous section for more details
(5.2.4). Note, however, that this approach cannot detect all problem configurations. In such
cases, drive tests are often effective, though not simplistic, in isolating potential problem(s).

A common symptom to look for is where FER on the forward link appears poor while the AT
Receive power and Pilot Ec/Io remain at decent levels. This follows from the discussion earlier
that traffic signal to noise ratio weakens since the AT ends up demodulating noise from the cell
not in the Active set. Usually, this is a very telling indicator of two or more cells being received
with same PN (either of the two problem scenarios) assuming there are no other issues (such as
cell software/hardware defects).

To confirm the problem, the cell suspected to suffer from PN interference should be turned off to
verify if the Ec/Io associated with this PN still remains strong. If it does, the next step will be to
identify cells in the vicinity configured with same PN or explore the possibility of a distant
overshooting cell. Latter may require meticulous investigation of terrain/morphology/cell
elevation in an attempt to identify the potential for a benign propagation environment in the area.

Suggested Mitigation Techniques

Once the failure scenario is identified, it is relatively easy to resolve the problem. For the first
scenario (overshooting cell), standard RF optimization techniques, such as antenna downtilt
and/or azimuth adjustments can be attempted to contain its coverage. The other option is PN re-
assignment, but care should be exercised to ensure it does not become a problem elsewhere.
Using a large PN increment will also mitigate the problem, though will require quite a bit of PN
planning re-work.

The second problem scenario can be addressed by maintaining sufficient number of cells between
PN re-use. A judicious design at the outset will go a long way in minimizing the amount of effort
spent fixing it later.

5.2.6 External interference

Concept/Reason for Failure

Presence of interference raises the noise floor, which raises the amount of power needed to
overcome it, further elevating the noise levels. As the cell or the AT runs out of power to close
the link, call reliability will suffer, dropped call being one of the common consequences. The

ALL RIGHTS RESERVED ALCATEL-LUCENT 2007 70


1 X EV-DO RF P ER FORM ANCE A NALYS IS & T ROUB LESHOOT ING G U IDE

level of impact, and hence, the ability to detect/resolve, may depend on the amount of
interference. An intermittent/weak transmitter may often be very difficult to identify.

There are several flavors of External Interference that could impact performance of a network. It
can exist on the forward or the reverse link.

An unlicensed transmission within the CDMA band is the most common cause of
interference. Usually spectral clearance tests are performed in a new deployment to identify
and rectify such sources. If the tests were not executed, such as, when additional spectrum is
allocated to an existing operator, continued operation of interference sources could cause
significant performance issues as the commercial service is activated on the carrier.

Often, interference is also caused by spill over of a legitimate transmission from the adjacent
channel due to narrow spectral spacing, and/or, inadequate filter roll-offs implemented by
the transmitter or the receiver.

Another flavor of interference is where inter-modulation (IM) products generated at the


receiver amplifier due to a strong signal on neighboring channel (although within the
amplifier band) causes significant performance degradation.

Failure Symptoms and Identification strategies

Detecting presence of interference is not that straightforward and locating the exact source may
be even more difficult. It often requires shutting down the desired transmission (that is, turn off
service) to pinpoint the root cause. Thus, it is best to isolate external interference upfront before
pressing system into commercial system.

The most telling indicator on the reverse link is much higher levels of RSSI rise, Eb/No setpoint
and RFER, all happening as a result of inflated noise floor. There are SM counts available to
inspect these per sector performance metrics RSSI (Short and Long Term Average RSSI
Rise pegged in the MCC), reverse Eb/No setpoint (Total Setpoint for Reverse Outer Loop
Power Control separate count for each reverse frame rate ranging from 0 to 153.6 kbps,
pegged in the TP), and RFER (Reverse Link Frame Error and Total Reverse Link Frame
counts, pegged in the TP). Corresponding performance metrics for these peg counts available in
SPO include: RSSI (One Second Average RSSI Rise, One Second Peak RSSI Rise, Ten
Second Average RSSI Rise, and Ten Second Peak RSSI Rise), and RFER (Reverse Frame
Error rate, Reverse Retransmission Failure rate and EVM DRC Erasure rate).

On the forward link, drive testing may help provide initial pointers to the presence of external
interference. Basically, the interference will result in very high mobile receive power, but poor
Ec/Io (Io goes up due to interference) and poor fwd FER. This, however, may also be caused due
to a missing neighbor or an inadequate search window size; hence, these have to be eliminated
first along with any potential hardware/software issues. Turning off some cells or service in the
affected area may be required to allow for thorough spectral clearance tests.

On either link, one way to isolate a problem related to IM interference is by using an in-line
attenuation to the mobile or the cell. If the measured signal power at either end (mobile receive
power on the forward link, or cell RSSI on the reverse link which is supposedly much higher than
the noise floor) increases at a much faster rate than the additional attenuation value, it suggests
the existence of IM products. Basically, the IM harmonics created due to amplifier non-linearity

ALL RIGHTS RESERVED ALCATEL-LUCENT 2007 71


1 X EV-DO RF P ER FORM ANCE A NALYS IS & T ROUB LESHOOT ING G U IDE

de-emphasize at a larger rate than the additional attenuation value while the desired inband signal
attenuates at the nominal rate; therefore, the composite signal still attenuates at a larger rate.

Suggested Mitigation Techniques

As above, identifying the precise location/source of the interference is the most difficult first step.
Once the source is identified, the remedy is simply to turn it off.

5.2.7 Insufficient search window size

Concept/Reason for Failure

There are two main search windows of interest active and neighbor sets. When searching the
PN space to track/detect pilots in its various sets, the AT utilizes a finite window width to keep
the search time small. The size is optimized to suit local terrain and cell topology in order to
maximize the likelihood of AT finding a Pilot and its multipath components in the smallest
possible time.

Different considerations go into determining Active and Neighbor set search window sizes.
Accordingly, there are two failure modes.

When the Active set search window size is not configured large enough to allow AT to
receive/track a strong multipath component. Typically, the default search window size is
good enough to cover most of the significant components. In some rare cases, especially
in large delay spread environments, this problem may surface. AT cannot
combine/demodulate the multipath since it falls outside the search window, rendering it
as an interferer.

When the AT fails to detect a neighbor due to an inadequate Neighbor search window
size. More than the presence of multipath, it is often the relative path difference at the
AT between the reference and the neighbor cells that drives this window width. Since PN
offset transmitted by each cell is a time-shifted version of a single PN sequence, the
difference in propagation delays to the AT requires the neighbor search window to be
wide enough to span the largest delay differential. Otherwise, a particular neighbor may
fall outside the search window, causing it to interfere with the Active set.

Not adding a strong Neighbor to the Active set may cause the call to drag, potentially increasing
the reverse link interference to other users, and also causing dropped calls on the vicinity cells.

When changing Active set search window size, it is usually a good idea to make the one used at
the cell site for reverse link demodulation, called Reverse Traffic Window Size, as well as for
acquiring soft handoff leg, called Data Window size, comparable. Both parameters on the BTSs
section of the EMS GUI.

Failure Symptoms and Identification strategies

SM counts do not provide a clear clue as to the existence of a search window issue.

ALL RIGHTS RESERVED ALCATEL-LUCENT 2007 72


1 X EV-DO RF P ER FORM ANCE A NALYS IS & T ROUB LESHOOT ING G U IDE

For an overlay network, a basic sanity check is recommended to ensure Active and Neighbor
search window sizes are comparable to the 2G/3G1x underlay.

Drive test is often the most productive method to identify search window issues. LDAT can be
used to alert and recommend proper neighbor search window sizes. Based on the mobile logs, it
employs the phase-offset information for various pilots reported on the Route Update Message
for an active call to identify potential for larger neighbor search windows. However, it cannot
identify situations where the existing neighbor search window is small enough to let AT even
detect the neighbor pilot. Towards that end, another tool that could be of particular significance,
especially in tough to resolve problem areas, is a Pilot scanner, which employs independent GPS
time reference to provide an accurate view of multi-path components from each cell.

Suggested Mitigation Techniques

Once the need for opening the search window for an Active or Neighbor set is identified, it can be
increased via translations. If a proper size is not known, it can be expanded in smallest possible
increments until performance is improved and/or mobile logs confirm proper detection at the
problem area.

5.2.8 Failures due to faulty hardware

Concept/Reason for failure

Presence of faulty or mis-configured hardware can influence the performance on the affected cell.
Hardware failures can occur in various forms and intensity. For example, a CRC or EVM board
entering into faulty operation registers sudden increase in dropped calls. At times, the affected
hardware device may only show one errant behavior, such as, increase in drop calls, but other
indicators such as access failure rate remain sane. A different flavor of a hardware problem could
be a faulty RF antenna cable with high return loss likely resulting in smaller than desired cell
footprint. In many cases, the hardware simply goes into a faulty state, the hardware itself is fine
software restore or stable clear rectifies the problem.

It is also possible to witness intermittent hardware outage instead of an outright malfunction, such
as loss of T1/E1 during certain hours.

Often issues with the hardware on a cell prevent the AT from entering into soft handoff as it
approaches this cell. This leads to the call being dragged until the AT can no longer
communicate with the existing Active set cells, eventually registering a dropped call on one of the
neighboring cells supporting the call. One particular case is where the timing unit in the cell
drifts away abnormally due to a hardware defect. This puts the affected cell out of sync with the
neighboring cells. As a result, soft handoffs are blocked in both directions (traffic entering or
leaving the cell) since a neighboring PN takes a different identity due to time shift. This problem
is commonly known as island cell phenomenon and impacts drop call performance more than
access performance.

Failure Symptoms and Identification strategies

A first check should involve careful review of alarms generated and flagged by fault management
module of the network. Certain issues associated with receive path (such as RSSI imbalance

ALL RIGHTS RESERVED ALCATEL-LUCENT 2007 73


1 X EV-DO RF P ER FORM ANCE A NALYS IS & T ROUB LESHOOT ING G U IDE

across main and diversity paths), malfunctioning hardware devices, backhaul outages, timing
units, etc. can easily be identified through the alarms. It is also likely that hardware related
failures will result in increased count for Connection Released Other Reasons (pegged in the
TP) and available as a metric in SPO as Connection Release rate Other Reasons.

The ROP analysis report in SPO provides information on these alarms at the AP, TP and BTS
levels. A pareto analysis of call set-up failures by specific hardware type based on ROP data can
offer a tell-tale sign of issues with a specific hardware/cell based on the over-abundant
concentration of the failures. To make this type of analysis meaningful, however, care should be
taken to normalize the failures with the amount of carried traffic.

It is easy to detect the Island cell problem via SM. Typically, only the dropped call rate shoots
up while the access failures remain the same. Further, the soft handoff activity dwindles in both
directions. Handoff activity can be inferred from SM count Soft and Softer handoff attempts
(pegged in the AP). Note, however, that softer handoffs (and soft handoffs within the same cell)
may still complete successfully, since the sectors share the same timing unit. However, soft
handoff with the neighboring cells will not be requested since cell will view phase reported on the
Route Update Message potentially as a Remaining Set Pilot. Similarly, soft handoff activity on
the neighboring cell will also reduce.

An Island cell problem can also be confirmed via drive tests by looking for strong unexpected
pilot PNs near the problem cell as well as absence of strong pilot PNs associated with the cell. A
Pilot scanner with its independent GPS source can facilitate this investigation.

Suggested Mitigation Techniques

Once the suspected malfunctioning hardware is identified, the first step is to attempt to rectify
through remote software restore or download. If the performance continues to suffer, it may
require cell visit to reseat the hardware and in some cases, to ensure tight cable connection. If all
on-site diagnostics fail, the last step will be to replace the hardware.

5.2.9 Heavy reverse link traffic


Concepts/Reasons for failure

As the reverse link load increases, either due to higher number of active users or increased data
upload activity, the overall interference floor seen by each cell increases. In order to overcome
the raised interference floor, AT has to transmit at much higher power levels. An AT already at
the cell edge can no longer close the reverse link as its signal arrives with a weak signal-to-noise
ratio at the cell. This would sacrifice call reliability leading to call drops in some instances.

Failure Symptoms and Identification strategies

The indication of high reverse activity can be inferred by increase in several SM counts, relative
to normal levels observed on other cells:

High values for peg counts introduced in R23 to measure Reverse Link Overload Control
including Short Term Average Fast Control Low Load count
(SM_SHTM_AVGLLC_FASTCTRL), Short Term Average Fast Control Medium

ALL RIGHTS RESERVED ALCATEL-LUCENT 2007 74


1 X EV-DO RF P ER FORM ANCE A NALYS IS & T ROUB LESHOOT ING G U IDE

Load count (SM_SHTM_AVGMLC_FASTCTRL) and Short Term Average Fast


Control High Load count (SM_SHTM_AVGHLC_FASTCTRL). These peg counts
are available on a per sector basis.

High Reverse Frame Error rate (ratio of Reverse Frame Error and Total Reverse
Frame counts, pegged in the TP)

High average setpoint for various rates (ratio of Total Setpoint for Reverse Outer Loop
Control and Total Number of Frames for Reverse Outer Loop Control at each rate,
pegged in the TP)

Increase in Number of Connections Force Released and Abnormal Release rate,


pegged in the MCC

Higher values for One Second and Ten Second Average/Peak RSSI Rise, pegged in
the MCC

Large number of Total Connection Requests per hour, pegged in the AP

Some of these indicators may also be influenced by reasons other than heavy reverse traffic, such
as:

External interference resulting in elevated RSSI levels, high RFER and high average set
points.

A malfunctioning device in the RF receive path, such as UCR or low noise amplifier,
causing consistent or intermittent RSSI spikes, thereby, reducing the coverage.

A component breakdown in the RF receive path, or improper antenna cable connection,


incapacitating one of the two diversity branches. This would tend to raise Eb/No
requirement as well as cause more power control fluctuations. Not only will this heighten
AT transmit power requirements, but also push the cell further into instability.

Hence, careful investigation of all possible causes may be required before a particular one can be
confirmed. Though not always necessary, one distinction may be that there are large number of
users consistently on the cell, as can be judged from Total Connection Requests per hour (sum
of AT and AN Initiated Connection Requests). It is less likely that just a few users consistently
load up the cell, usually a pointer, to external interference or faulty hardware.

Suggested Mitigation Techniques

It is always a sound practice to track/trend various performance indicators, especially, Connection


Requests and RSSI rise to allow preemptive action.

Once the cell is identified as a heavy traffic-bearing cell impacting dropped call performance due
to reverse link loading or overload, there are a few options available to offload the traffic:

Reduce the overload control thresholds to lower the reverse transmission rates. With
R23, the HDR Reverse Link Overload Control (HROC) tunable parameters have become
translation parameters. The Alcatel-Lucent Reverse Overload Control mechanism

ALL RIGHTS RESERVED ALCATEL-LUCENT 2007 75


1 X EV-DO RF P ER FORM ANCE A NALYS IS & T ROUB LESHOOT ING G U IDE

utilizes these translatable thresholds to manage the integrity of reverse link performance.
Refer to [5] for an overview of operation and associated translations. Lowering these
thresholds will reduce the probability that AT will transmit at a higher rate, under load.
Lower transmission rates require less AT transmit power on an average. In essence,
reverse overload will kick-in at lighter loading levels thereby mitigating coverage
reduction at the expense of data rate performance on the reverse link.

This is not a preferred solution in the long-term. It may, however, be acceptable as a


stopgap measure instead of denying service to users until longer-term measures, stated
next, are pursued. However, the customer must be made aware of the trade-offs before
attempting any threshold adjustments.

One solution is to increase pilot overlap such that other links get into the active set of the
call, mitigating the traffic channel power requirement lowering the overall interference.
From the perspective of mitigating AT transmit power requirement, higher handoff
diversity should never be undesirable. However, careful consideration is warranted to
characterize potential impact on the forward link performance as well as increase in cell
resource utilization (e.g., channel element hardware).

Add a 1xEV-DO carrier.

Add a cell to serve the heavy traffic area. If adding a carrier is not a possibility due to
lack of spectrum, adding a cell (geographically separated from the high traffic cell(s)) is
often an effective, although a long-lead, solution to improve coverage and hence, permit
higher levels of offered traffic per unit area. Care must be taken to ensure proper RF
optimization is performed to minimize excessive Pilot with cells in the vicinity
otherwise it may impact forward link performance.

5.2.10 Maximum number of legs reached


Concept/Reasons for failure
Handoff additions may only happen to the extent there is room in the ATs Active set. The
maximum Active set size is governed by the translation called Maximum Legs in Handoff. It is
in the Service Node/General section of the EMS GUI. When the Active set gets full, the only way
to add a pilot is to wait for one of the Active set members to drop19. To drop a pilot, its Ec/Io
strength must stay below the Pilot Detection Threshold for at least Drop Timer Value
seconds. These two translation parameters are in the EMS GUI section for Sectors.

One can see that in certain situations, waiting to drop off a weak pilot can delay addition of a
more worthwhile pilot. The delay could result in a dropped call if the mobile has already
dragged into the coverage of a candidate pilot before establishing a connection with it.

19
Alcatel-Lucent 1xEV-DO system supports a mechanism to simultaneously add and drop Pilots. However, when the
Active set is full and a candidate Pilot is available, this shuffle does not take place until the drop condition for one of
the Active set Pilots has been met.

ALL RIGHTS RESERVED ALCATEL-LUCENT 2007 76


1 X EV-DO RF P ER FORM ANCE A NALYS IS & T ROUB LESHOOT ING G U IDE

Such events are more likely to happen in pilot polluted environments. Usually, Alcatel-Lucent
recommendation of setting the Maximum Legs in Handoff to 4 is adequate to minimize such
dropped calls. However, it is conceivable that the problem may still happen in poorly optimized
or tough to optimize RF environments.

Failure Symptoms and Identification strategies

It is easy to spot this phenomenon from SM. There is a count in the AP called Soft and Softer
handoff attempts not processed Maximum number of Legs Reached to peg it.

Suggested Mitigation Techniques


One method to mitigate the dropped calls occurring due to full active set is to allow more pilots in
the Active set. That is, the Maximum Legs in Handoff translation can be increased to 6.

Note that there could be potential downsides to expanding the Active set (e.g., use up more MAC
Indices on the forward link and increase hardware utilization on the reverse link). It may still be a
good interim step, however, especially if the problem areas are localized and not severe in nature.

A better/longer-term solution would be to resort to traditional drive test based RF optimization


techniques and attempt to improve dominant coverage in the region (see section 5.2.2). This will
limit the likelihood of encountering a full Active set when a strong candidate pilot needs to be
added. Another option after exhausting efforts to improve dominant coverage in the area is to
experiment with higher values of Pilot Detection Threshold and Pilot Drop Threshold.
These translation parameters are in the Sectors section of EMS GUI and the generally
recommended values are 9 and 11 dB respectively. Higher values will tend to keep Active set
smaller. In future, the availability of dynamic add/drop thresholds (similar to IS95B handoffs
enhancements) will also help reduce Active set size.

5.2.11 No resources at the candidate leg

Concept/Reasons for failure

When the candidate leg to be added to the Active set has run out of resources, the handoff add
attempt will not go through. As a result, the effective SNR on one or both the links will degrade
due to growing pathloss as AT moves away from the existing Active set cells and approaches the
cell it was denied handoff with.

The poor SNR levels due to call drag at some point will diminish the ATs ability to decode the
forward link. The mobile can also make reverse link power control errors and after a while unable
to reach the cell as the transmit power reaches its ceiling. The high transmit power will also raise
the interference levels to the nearby cells. All this could lead to increase in drop calls on the cells
in the vicinity.

Failure Symptoms and Identification strategies

There is a separate SM count pegged at the AP level to facilitate investigation of handoff rejects
due to resource shortages. It is called Soft and softer handoff failures lack of resources in the

ALL RIGHTS RESERVED ALCATEL-LUCENT 2007 77


1 X EV-DO RF P ER FORM ANCE A NALYS IS & T ROUB LESHOOT ING G U IDE

candidate sector. Increase in this count on a particular sector will likely be accompanied by
higher Dropped Connection Rate on the neighboring sectors (prior to the drop, the call will be in
active connection with these cells) as well as on this resource constrained sector (due to higher
reverse link interference caused in the vicinity by the call drags).

With R24, a new set of target sector soft/softer handoff service measurements is available. In
addition to the count listed in the previous paragraph, the Soft/Softer handoff failures due to no
resources peg count, pegged at the target sector, provide a view of handoff blocking in the target
sectors. Analyzing such counts for neighboring sectors, for example in a geographical map,
provides a high level view of where handoffs are being blocked.

Any lack of resources will typically be due to cell running out of channel elements. Note that
with one FLM/RLM (forward link/reverse link modem card), there are 93 CEs available on a 3-
sector cell. In rare cases, where a sector takes on a large share of traffic, it could run out of MAC
Indices (max 59 MAC Indices available per sector) before all CEs on the cell are used or the other
limitation of 48 Max Number of Users20 per sector limit is reached.

It must be noted that with R26, when the number of users in a sector reaches the maximum limit,
the Soft/Softer handoff failures due to no resources at the target sector is pegged instead of
the Soft/Softer handoff failures due to no resources peg count.

Suggested Mitigation Techniques

One solution is to reduce the coverage of the cell in an attempt to offload traffic to the
neighboring cells/sectors. This, however, assumes the neighboring cells have the room to handle
more traffic, which may not be the case all the time.

An alternative is to add multiple carriers. This is a more graceful solution than adjusting
coverage, which may not always yield desired traffic offloading. Release 25 will support
multiple carriers for 1xEV-DO.

Where there are spectrum availability issues, adding a cell may be the last resort solution. This, of
course, requires longer lead times in order to accommodate zoning, cell installation, RF
calibration and optimization stages before turning the cell over to commercial service.

5.2.12 High Processor Occupancy rates

Concept/Reasons for failure

When equipment processor occupancy levels exceed certain thresholds, it is likely that new
connections are blocked and existing connections are dropped. There are several components in
the 1xEV-DO system for which occupancy levels can be monitored including the Application
20
The Maximum Number of Users constraint may block new call attempts only. Handoffs are not blocked. So, if a
sector receives a lot of soft handoff traffic, the total number of users supported can exceed the limit, potentially
bumping against the MAC Index limit.

ALL RIGHTS RESERVED ALCATEL-LUCENT 2007 78


1 X EV-DO RF P ER FORM ANCE A NALYS IS & T ROUB LESHOOT ING G U IDE

Processor (AP), Traffic Processor (TP), Line Interface Unit (LIU), Forward Link Modem (FLM)
and Reverse Link Modem (RLM).

Failure Symptoms and Identification strategies

With R23, there are several service measurement counts that have been introduced to monitor the
occupancy levels. High values for these counts over several days could potentially result in
blocked and dropped calls. These counts include:

AP Peak Processor Occupancy (AP_PEAK_PROC_OCCUPANCY)

AP Average Processor Occupancy (AP_AVG_PROC_OCCUPANCY)

LIU Peak Processor Occupancy (LIU_PEAK_PROC_OCCUPANCY)

LIU Average Processor Occupancy (LIU_AVG_PROC_OCCUPANCY)

FLM Peak Processor Occupancy (FLM_PEAK_PROC_OCCUPANCY)

FLM Average Processor Occupancy (FLM_AVG_PROC_OCCUPANCY)

RLM Peak Processor Occupancy (RLM_PEAK_PROC_OCCUPANCY)

RLM Average Processor Occupancy (RLM_AVG_PROC_OCCUPANCY)

With R25, there are additional peg counts in support of the 1xEV-DO cell processor overload
control feature. High values for the following counts could result in blocked and dropped calls:

LIU_Overload_Num

LIU_Overload_Duration

FLM_Overload_Num

FLM_Overload_Duration

RLM_Overload_Num

RLM_Overload_Duration

The Peak_TP_Utilization peg count in R25 provides the maximum TP processor occupancy
rate.

Suggested Mitigation Techniques

If the AP or TP occupancy levels are high and with high traffic volume, consider re-homing some
cell sites to another AP with lower occupancy levels. If LIU, FLM and/or RLM occupancy levels
are high, consider off-loading traffic from the cell in question to surrounding cell sites.

ALL RIGHTS RESERVED ALCATEL-LUCENT 2007 79


1 X EV-DO RF P ER FORM ANCE A NALYS IS & T ROUB LESHOOT ING G U IDE

5.2.13 Handoff failures

Concept/Reasons for failure

When the AN directs the AT to perform a handoff, it expects the AT to respond back indicating a
successful handoff. If there is no response from the AT, the handoff is considered unsuccessful
and the connection is dropped, resulting in increased Dropped Connection Rate.

Failure Symptoms and Identification strategies

The Soft/Softer handoff failures due to no AT response peg count measures the number of
failed soft/softer handoffs at the source sector that resulted in drop calls. With R24, there are new
counts that measure the soft/softer handoff failure at the target sector. The Soft/softer handoff
failure due to no AT response at target sector peg count can be used in conjunction with the
Soft/Softer handoff attempt at target sector to compute the handoff failure rate at the target
sector that contributes to drop calls. These metrics are available as Soft/softer handoff failure rate
and Soft/softer handoff failure rate at target sector in SPO.

In addition, analysis of call processing failures and associated error codes provides additional
details for the reason for handoff failures. SPO call processing failure analysis report provides
per sector level summary of all failures along with unit information and error codes. Refer to [20]
for a look-up table that translates the error codes to detailed failure cause.

Neighbor list analysis often reveals PN ambiguities that could result in handoff failures. NL Alert
analysis in SPO provides a comprehensive report of such neighbor list inconsistencies. In
addition, if there are too many strong pilots vying for handoffs, handing off to one target sector
with interference from other strong neighbor pilots could also contribute to failed handoffs. Pilot
pollution problems are dealt with in more detail in section 5.2.3.

With 1xEV-DO R25, Handoff Matrix (HOM) and Undeclared Neighbor List (UNL) data is
available. In addition to NL Alert analysis, the Handoff Matrix analysis and Undeclared
Neighbor List analysis reports and maps in SPO can be used to add required neighbors and
remove unneeded neighbors.

As of Release 26, new peg counts and metrics became available to measure inter frequency (hard)
handoff failure rates. Some of the available hard handoff attempt failure counts include the Hard
Handoff Failures Lack of Resources - Requesting Sector or Hard Handoff Failures Lack of
Resources - Target Sector. These counts provide the number of attempted inter frequency
handoffs that failed due to no resource availability at the target or requesting sector. Other counts
include the Hard Handoff Failures No AT Response - Requesting Sector and Hard Handoff
Failures No AT Response - Target Sector, which provide a measure of failed hard handoff
attempts due to no AT response.

Suggested Mitigation Techniques

If there are neighbor list ambiguities that are contributing to increased handoff failures,
appropriate steps need to be taken to resolve such ambiguities (section 5.2.4 discusses this in
detail). Typical RF optimization techniques to mitigate pilot pollution include adjusting cell site

ALL RIGHTS RESERVED ALCATEL-LUCENT 2007 80


1 X EV-DO RF P ER FORM ANCE A NALYS IS & T ROUB LESHOOT ING G U IDE

power and/or antenna down-tilt of one or more neighboring cells to create a dominant pilot or
fewer visible pilots (section 5.2.3 discusses this in detail).

With adequate traffic in the network, an add/delete neighbor analysis can be performed using the
Handoff Matrix analysis reports and maps. In addition, UNL reports and maps can be used to
study strong remaining set pilots that should either be considered to be added into the neighbor
list or considered for reduced coverage.

Regarding hard handoff failures, RF optimization techniques apply as well. A first step is to
verify all hard handoff parameter settings are properly tuned, and that there is enough RF
coverage to support the inter frequency handoff. Additionally, all inter frequency neighbor list
entries must be correct, and all inter frequency handoff trigger thresholds must be set to values
that actually support successful inter frequency handoffs, see [27] for more information.

5.2.14 Reverse link lost due to hybrid tuneaways

Concept/Reasons for failure

Hybrid mode operation affords AT the flexibility to receive voice calls on a 2G/3G1x system
while on a 1xEV-DO Data session. The downside is that the AT has to periodically tune to
2G/3G1x system to look for pages.

A hybrid 1xEV-DO/3G-1X AT periodically tunes away from an active 1xEV-DO connection to


check for paging messages on the underlay 3G-1X system and to measure the current RF
conditions to assist with triggering the 1xEV-DO to 3G-1X hybrid handoffs at the edge of 1xEV-
DO coverage. These tuneaways can cause erroneous pegging of the RF Link Lost (RLL) SM
count.

Failure Symptoms and Identification strategies

By distinguishing between actual RLL (occurring because of bad RF conditions) and tuneaways,
this feature limits erroneous pegging of the preexisting CONN_REL_RLL count. The new count,
CONN_REL_AT_TUNAWAY, is pegged when the system detects that the AT has tuned away
to 3G1X.

Note that, there is no impact to the existing handoff failure SM counts. Prior to R26 (or with R26
and the feature disabled), if a handoff failure occurred due to RLL then both the
CONN_REL_RLL and the appropriate handoff failure counts are pegged. With the feature
enabled on R26, the appropriate handoff failure count still pegs along with either the
CONN_REL_RLL or CONN_REL_AT_TUNAWAY_RLL count, depending on the scenario.

Four new ROP messages were added to account for RLL due to hybrid mode tuneaway.
Depending on the connection or session state of the AT, the appropriate ROP message will result.

Suggested Mitigation Techniques

As mentioned earlier, if hybrid mode is enabled, work with the service provider to identify any
issues with hybrid mode performance of most prevalent terminals in the market.

ALL RIGHTS RESERVED ALCATEL-LUCENT 2007 81


1 X EV-DO RF P ER FORM ANCE A NALYS IS & T ROUB LESHOOT ING G U IDE

5.3 Data Rate

Data Rate is defined as the amount of data transferred per unit time. It can be measured at various
layers in the protocol stack, see Figure 3 for more information. Unlike 3G1x, in 1xEV-DO Rel 0
systems, the peak forward and reverse data rates are different, with the peak forward data rate
being 2.4Mbps and the peak reverse data rate being 153Kbps. Additionally, for 1xEV-DO Rev A,
the peak forward and reverse data rates are even higher at 3.07Mbps and 1.84Mbps respectively.

We can define the Data Rate at the RLP layer for EVM boards as follows for Release 24 and
later:

RLP Forward Data Rate


Number of RLP Octets Transmitte d On Forward Traffic Channels 8 / 1000
(% Busy Slots / 10 ^8) * 2160000 * 0.001667

This metric present several inaccuracies, so its use should be estimating data rates are the RLP layer, see
[25] for more information. For Release 28, the above metric will report a value of 0 for sites equipped with
single board EVMs (SBEVM) only, thus a new metric for sectors equipped with SBEVMs was introduced.
This metric is:

Sum of ( EVM RLP Octets Transmitte d On Forward Traffic Channels


/ 1000
[ BE CPS CV CS SMC ]) 8
RLP Forward Data Rate [ SBEVM ]
( Sum of Forward Traffic Channel Slots for different Rates ) 0 .001667

The numerator in this metric is the sum of the EVM RLP Octets transmitted on the forward traffic channel
for the different types of traffic flows: Best Effort, Push-to-talk, Conversational Video, Conversational
Speech and SMC octets.

These two formulas may be combined to produce a composite RLP Forward Link Data Rate metric by
summing both numerators and dividing this sum by the RLP Forward Data Rate metric. This composite
formula may be used for obtaining an RLP data rate that takes into account both types of modems, see [25]
for more information.

For the reverse link, two formulas are also available, one for non-SBEVM sites and one for sites equipped
with SBEVMs. The formulas, for Release 28 and later are as follows:

RLP Reverse Data Rate


Number of RLP Octets Transmitte d On Reverse Traffic Channels 8 / 1000
( Sum of Reverse OL Power Control Frames per Data Rate ) 0 .0266

And

RLP Reverse Data Rate [ SBEVM ]


Number of RLP Octets Transmitte d On Reverse Traffic Channels 8 / 1000
( Sum of Reverse Link Subpackets per Data Rate ) 0.00667

ALL RIGHTS RESERVED ALCATEL-LUCENT 2007 82


1 X EV-DO RF P ER FORM ANCE A NALYS IS & T ROUB LESHOOT ING G U IDE

As in the forward RLP data rate metrics, a composite metric may also be defined by combining both
equations. This composite RLP Reverse Data Rate may be used independent of the type of EVMs
equipped.

The unit for these metrics is in kilobits per sec (kbps). These are direct SM counts pegged in the
TP on the DRC pointed sector. It is important to use only the original RLP data, not include the
re-transmitted data, in order to obtain a more accurate view of the end-user experience. The
constants 0.001667, 0.02667 and 0.00667 in the data rate equations represent the frame
size in the forward and reverse link respectively.

The forward data rate related metrics are available as Avg Fwd link MAC data rate, Avg Fwd
link Physical Layer data rate and Avg Fwd link RLP layer data rate in SPO. The reverse data
rate related metrics are available as Average Rev link burst rate, Avg Rev link Physical layer
data rate and Rev link RLP data throughput in SPO.

Additionally, per user data rate metrics are available for the RLP layer. This per-user metric uses
in the denominator a count that measures the number of users who have data to send for any flow
(BE, or EF), either waiting to be scheduled or in the process of transmission. This count is
reported as the summation of all such users across all slots over the entire SM collection interval.

Also note that on the forward link, it is possible that some of the data never makes it to the end-
user even after a retry; there is no way to obtain that view from service measurements normally
it represents a small percentage (of the order of 0.02%) and the above is a good approximation.
data rates defined above can be obtained on a per sector basis.

On the forward link, the data rate is impacted either because the mobile is requesting lower DRC
rates (poor RF conditions, interference, bad cell site hardware corrupting the transmit Signal to
Noise Ratio, etc.), or the mobile is not getting assigned enough time slots as the scheduler is
serving other user(s) on the cell. Note that there is no power control on the forward link. Cell
always transmits at full power. It is up to the mobile to adjust the requested rates based on the
short-term estimates of SNR (similar to Ec/Io) to meet 1% packet error rate. The mobiles measure
SNR for small portions in each time slot (that is, when all sectors are transmitting Pilot channel).
The received SNR is a function of RF conditions at the mobile (fading, coverage, interference
from other Pilots) and the transmitted SNR (in case of cell site hardware failure). It is interesting
to note that since 1xEV-DO has a time multiplexed forward link, the received SNR at the AT is
not a function of other active users on the cell. This is in contrast to 2G/3G1x systems, where RF
loading from other users impacts pilot the Ec/Io at the mobile.

On the reverse link, the data rate is impacted when the reverse frame error rate is not being met
(due to poor RF conditions or interference), or the mobile is transmitting at lower rates (either
because it does not have adequate power to transmit at higher rates such as when the pathloss is
high, or cell is instructing to do so as part of reverse overload control due to high traffic demand).

In subsequent sections, we discuss the constituent causes for low data rates in more detail. One
area that is not treated in any of the discussions in the document is low data rates due to
configuration problems in certain network elements (such as PDSN, router between PCF/PDSN,
etc.). Such problems are likely to be systemic (a large number of calls in the system will be
impacted) and should have been resolved either prior to commercial service operation or soon
thereafter. The user is encouraged to refer to [12] and [13]. Reference document [12] has

ALL RIGHTS RESERVED ALCATEL-LUCENT 2007 83


1 X EV-DO RF P ER FORM ANCE A NALYS IS & T ROUB LESHOOT ING G U IDE

information on network configuration troubleshooting (such as using test call traces). Reference
document [13] contains information on data rate expectations in a variety of environments.

The data rate is impacted negatively for MSM5500-based ATs due to the limitations in its mobile
IP (MIP) implementation [13]. In MIP, the MSM5500-based AT experiences additional latency
resulting in a maximum TCP data rate of 1.46Mbps. It has been observed that the data rate
performance for MSM6500-based ATs is significantly better than the MSM5500-based ATs.
ATs with MSM6500 chipsets are also expected to have a more efficient ramp-up algorithm
following a tune-away, which improves reverse link data rate performance. In addition, these
ATs also tend to complete a 1xEV-DO to 3G1X handoff faster than ATs with the older chipsets.

In hybrid mode, there will be a perceivable impact on data rate based on Slot Cycle Index settings
[13]. The lower the slot cycle index setting the higher the impact on both the forward and reverse
link data rates. It has been estimated that with a SCI setting of 0 and in SIP mode, there could be
data rate degradation of up to 20% in the forward link and 30% in the reverse link compared to an
EV-DO only mobile.

5.3.1 Lack of RF coverage

Concept/Reason for failure

AT in weak coverage areas (such as AT receive power less than 105 dBm and best pilot SNR < -
10dB, mobile transmit power > 20 dBm), by definition, suffers from excess pathloss affecting
SNR at the receiver on either link.

Lower received SNR can not only reduce the transmitted data rates, but can also trigger re-
transmission due to potential for high packet error rates. In either case, achieved data rate will
suffer in the weak coverage area.

Typically, the reverse link runs out of coverage first given that the 1xEV-DO forward link has
roughly 10 dB advantage (Pilot channel transmitted at full power). Areas suffering from heavy
shadowing such as in-building locations or areas blocked by buildings, trees, etc can likely suffer
from low data rates on the either link.

Failure Symptoms and Identification strategies

Typically, cells suffering from low data rate due to ATs in weak coverage area will also tend to
exhibit lower packet transmission rates. There are separate SM counts to measure this on either
link. Additional SM derived metrics that should also tend to show correlated degradation
include:

Relatively larger number of DRC Rate Requests at lower forward link rates (counts pegged
in the Modem)

High RLP re-transmission rate (ratio of RLP Octets Re-transmitted to RLP Octets
Transmitted on the forward link; ratio of Missing RLP Octets Requested to RLP Octets
Received on the reverse link all these counts pegged in the TP)

ALL RIGHTS RESERVED ALCATEL-LUCENT 2007 84


1 X EV-DO RF P ER FORM ANCE A NALYS IS & T ROUB LESHOOT ING G U IDE

High Physical layer NACK rate (ratio of Total Number of Physical Layer ACKs Received
to Total Number of Physical Layer NACKs Received on the forward link only both
counts pegged in the modem)

High Reverse Frame error rate (ratio of Reverse Link Frame Error Count to Total Reverse
Link Frame Count both counts pegged in the TP)

Ascertaining degradation using the above metrics does not lead to confirming weak coverage area
as the root cause for low data rate. Such problems are best investigated further via drive tests.
The drive test area can be narrowed to within the coverage of a few cells by identifying cells via
SM, which tend to exhibit higher degradation in data rate compared to average.

Suggested Mitigation Techniques

Having identified select areas, drive test based RF optimization techniques can be employed as
detailed in [8]. The process typically entails analyzing drive test data to arrive at a suitable
conclusion, such as, increasing cell power or adjusting antenna azimuth, or in the worst case
adding repeater (especially in in-building environments) or cell to fill in low signal strength areas.

Improving the coverage may also result in other benefits such as improved access and dropped
call rate in the region.

5.3.2 Excessive pilots/Non-dominant pilots

Concept/Reason for failure

Presence of too many pilots in an area often impacts forward link performance due to interference
it creates to the pilot(s) being actively tracked by the AT. Typically, there is no dominant pilot in
such a problem area, that is, there are several pilots with weak comparable strengths, often
accompanied by constant Pilot thrashing (dominant sector changing quickly). Another scenario is
where a distant sector overshoots to create/contribute to a localized non-dominant pilot region.

Since AT decodes traffic on only one pilot, reduced SNR due to interference from other pilots
results in lower DRC rate requests. As a result data rate will decrease on the forward link. Low
data rate may also occur due to gaps in transmission during virtual soft handoffs, as the pilots
thrash back and forth.

Failure Symptoms and Identification strategies

There are several ways to attain a pointer to the potential existence of this problem.

On a relative basis, cells exhibiting higher than normal rate of connection requests with 3 or more
Pilots provide one such indication. There are also separate SM counts to flag Connection
Requests with 2 Pilots and Connection Requests with 3 Pilots events. All these counts are
reported in the AP for the sector that received the Connection Request. [Sum of AT Initiated
Connection Requests and AN Initiated Connection Requests] minus [Sum of these multiple
pilot counts] reflect the number of Connection Requests with single pilot. The following four
metrics in SPO can verify the existence of the pilot pollution problem:

ALL RIGHTS RESERVED ALCATEL-LUCENT 2007 85


1 X EV-DO RF P ER FORM ANCE A NALYS IS & T ROUB LESHOOT ING G U IDE

Connection Request with One Pilot (%)


Connection Request with Two Pilots (%)
Connection Request with Three Pilots (%)
Connection Request with Four or more Pilots (%)
Guidelines similar to the one used to detect pilot pollution in 2G/3G1x networks, based on SM
counts for the number of Tadd pilots reported on the PSMM, can be used. Higher than normal
rate of Soft/Softer HO fail due to maximum number of legs reached (ratio of
SO_SOR_HOFF_FAIL_MAX_NO_LEGS_REACHED and SO_SOR_HOF_ATTMPT)
when the Maximum Legs in Handoff configuration parameter is set to a value higher than 3 can
also indicate pilot pollution problems.

Another pointer, though a weak one21, is excessive activity for Soft and/or Softer handoff
attempts. It is likely that other KPIs such as Dropped call and Established Connection Rates may
also exhibit impairment in these areas. Other derived SM metrics (re-transmission rate, NACK
rate) discussed in the prior section may also show degradation on the forward link.

For one-to-one overlay over Alcatel-Lucent 2G or 3G1x network, additional data and tools from
the 2G/3G1x infrastructure may also be employed for identification of excess Pilot areas.

2G/3G1x cells showing high secondary CE usage relative to Primary (or Total) CE usage,
or Total Walsh Code relative to Primary Erlangs, are potential indicators of excess Pilot
regions.

Handoff Matrix and Undeclared Neighbor List analysis tools in SPO can be used on the
underlying 2G/3G1x network to flag cells overshooting from a distant region. Every
effort must be made to contain their coverage/reduce their interference in order to not only
benefit call set-up rate, but also to improve overall performance (Dropped Connection
Rate, Data Rates, etc).

Suggested Mitigation Techniques

Similar to the Lack of RF coverage reason, drive tests can be employed to analyze and resolve
such problem areas. Typical RF optimization techniques include adjusting cell site power and/or
antenna downtilt of one or more neighboring cells to create a dominant Pilot or fewer visible
Pilots. Antenna downtilt usually are more effective in controlling coverage of overshooting
sectors. Refer to [8] for more details.

When improving pilot dominance, care should be taken not to pull back coverage to the extent
that the reverse link performance begins to suffer. Higher handoff diversity is usually beneficial
to the reverse link as it mitigates AT transmit power. Reverse link performance should be
verified under load to ensure the impact on mobile transmit power requirements is minimal.

21
Soft/softer handoff activity is more symbolic of mobility, but along with several other indicators tends to suggest a
potential multi-pilot region.

ALL RIGHTS RESERVED ALCATEL-LUCENT 2007 86


1 X EV-DO RF P ER FORM ANCE A NALYS IS & T ROUB LESHOOT ING G U IDE

5.3.3 Poorly constructed neighbor lists

Concept/Reason for failure

Not having a fine tuned neighbor list is a common source of data rate performance degradation.
In essence, it prevents AT from pointing its DRC to the best Pilot. As it continues to tune to a
Pilot with weaker SNR, it requests lower DRC rates, thereby, impacting forward link data rate.
The reverse link data rate may also suffer due to call drag resulting in higher effective path loss.
The call drag in general will cause higher interference levels in the area due to higher mobile
transmit power, impacting reverse link data rate for other mobiles.

A poor neighbor list may typically be missing a nearby pilot to which AT will otherwise handoff
to, or may have integrity issues, such as, lack of reciprocity or suffer from PN ambiguity,
discussed below.

Rather than the outright neighbor list omissions whose impact is clearly evident in terms of
dropped call and access failures, the marginal pilots require more due diligence to detect since the
data rate impact will be a more gradual or wider spread phenomenon rather than a single event.
Nevertheless, it is important to detect and add such pilots to improve the achieved data rates,
especially on the forward link. The reverse link improvement may be subtler, since it may accrue
only as a side benefit in terms of reduced interference (AT transmit power is mitigated due to
reduced call drags).

Failure Symptoms and Identification strategies

The first check for cells with low data rate should be to review the neighbor list entries and ensure
they follow basic rules, such as other sectors of the same cell as well inward facing sectors of the
surrounding first tier cells are included as neighbors and the correct AP IP addresses are
populated for the neighbor sectors.

If the Soft Handoff Attempt to Assign rate metric is high, it typically indicates that some of the
neighbor sectors have incorrect AP IP addresses listed in the NeighborSector form.

The Neighbor List Alert (NL Alert) feature in the SPO tool can be employed effectively to fine
tune neighbor lists. The integrity can be verified in terms of:

Reciprocity - if cell X is on neighbor list of cell Y, then so should be Y on the neighbor list of
X.

No PN ambiguity - cell X should not have two cells with the same PN on its neighbor list22,
or if cell X and Y are in handoff, where X sponsors neighbor cell A and cell Y sponsors cell
B, A and B should not be configured with the same PN23. Otherwise, it is possible that AT
ends up establishing a traffic link on the weaker of the two cells with the same PN, thereby,
impacting access performance due to noise component brought by the stronger non-traffic

22
Commonly known as one-way PN ambiguity.

23
Commonly known as two-way PN ambiguity.

ALL RIGHTS RESERVED ALCATEL-LUCENT 2007 87


1 X EV-DO RF P ER FORM ANCE A NALYS IS & T ROUB LESHOOT ING G U IDE

bearing PN at the AT demodulator. In some cases, AT may have trouble decoding control
channel contents if

Cross-face omission It is a good idea to include other sectors in the cell as high priority
neighbors.

AP IP alert The IP addresses of the neighbor sectors APs have to be defined correctly in
order for handoffs to occur.

With 1xEV-DO R25, Handoff Matrix (HOM) and Undeclared Neighbor List (UNL) data is
available. In addition to NL Alert analysis, the Handoff Matrix analysis and Undeclared
Neighbor List analysis reports and maps in SPO can be used to add required neighbors and
remove unneeded neighbors.

Suggested Mitigation Techniques

Fix any reciprocity issues and resolve PN ambiguity by changing PN offset of one of the two
cells with common PN. Ensure that the neighbor sectors have the correct AP IP addresses
populated.

If it is a recent deployment without a 2G/3G1x overlay or during initial optimization, it is best to


shape neighbor lists with drive tests. Neighbor list omissions can be detected for example, where
a call drops and shows up on a strong pilot that was not in the prior Active set or the Neighbor
List message sent to the mobile. Opening the Remaining set can also help identify missing
neighbors, if they are strong enough for the AT to detect and report them, either via Route Update
or Searcher info message. Refer to [8] for more details on drive test based neighbor list
refinement techniques.

With adequate traffic in the network, an add/delete neighbor analysis can be performed using the
Handoff Matrix analysis reports and maps. In addition, UNL reports and maps can be used to
study strong remaining set pilots that should either be considered to be added into the neighbor
list or considered for reduced coverage.

5.3.4 Improper PN planning

Concept/Reason for Failure

Improper PN planning could result in PN assignments such that AT cannot resolve signal from 2
different cells. There are two common scenarios in which this problem manifests.

PN offsets are time-shifted versions of the same PN sequence. Each PN offset is 64 chips
apart. Large propagation delays, as in the case of signal from a distant overshooting cell,
can alter the identity of a PN at the AT, which in the worst case could make it
indistinguishable from a nearby cell. When AT sends Route Update message, it will
include this common PN it sees. The Active set will include only the close-in cell, not on
the far-away cell since it is actually transmitting on a different PN. As a result, only the
close-in cell will key up traffic. When AT attempts to demodulate traffic channel, it will

ALL RIGHTS RESERVED ALCATEL-LUCENT 2007 88


1 X EV-DO RF P ER FORM ANCE A NALYS IS & T ROUB LESHOOT ING G U IDE

not only include valid traffic energy from the close-in cell, but also weigh in noise
component from the distant cell. Consequently, the traffic signal-to-noise ratio will nose-
dive, impacting data rate performance.

Another scenario is where two cells are assigned the same PN, but are not physically
separated enough such that at the AT the two cells becomes indistinguishable. This
triggers the same performance impacting phenomenon as described above in the case of
two far-way cells with different PNs.

In both cases, the problem appears because there is not enough separation in the PN offset of the
two cells. Depending on the severity of interference generated on the forward link, there could be
more series consequences such as access and dropped call failures. If that is not the case, the
small amount of interference can create pocket areas of low data rate on the forward link,
potentially escaping visibility from the SM.

Prudent PN planning can help minimize the problem. As mentioned above, if the interference
effect is not severe, the small but pervasive data rate degradation may evade detection via SM, or
worse, become difficult to detect even via drive tests. This makes proper PN design even more
critical.

The first scenario is mitigated by using adequately large Pilot_increment (available as Pilot PN
Sequence Offset Index Increment in the Service Node translation form) so that AT can still
resolve signals from cells received with large relative difference. Special attention should be
heeded for situations where signal is expected to propagate much farther than typical range, for
example, due to low loss experienced over water bodies, signal from elevated cell sites, etc. The
second scenario requires proper PN re-use margin.

Failure Symptoms and Identification strategies

The NL Alert feature in SPO is designed to uncover PN ambiguity issues and can potentially help
identify the problem with inadequate PN reuse margin. See the previous section for more details.
However, this approach cannot detect all problem configurations. In such cases, drive tests are
often effective, though not simplistic, in isolating potential problem(s).

A common symptom to look for is where FER on the forward link appears poor while the mobile
receive power and Pilot Ec/Io remain at decent levels. This follows from the discussion earlier
that traffic signal to noise ratio weakens since the AT ends up demodulating noise from the cell
not in the Active set. Usually, this is a very telling indicator of two or more cells being received
with same PN (either of the two problem scenarios) assuming there are no other issues (such as
cell software/hardware defects). To confirm the problem, the cell suspected to be facing PN
interference should be turned off to verify if the Ec/Io associated with this PN still remains
strong. If it does, the next step will be to identify cells in the vicinity configured with same PN or
explore the possibility of a distant overshooting cell. Latter may require meticulous investigation
of terrain/morphology/cell elevation to identify the potential for a benign propagation
environment in the area.

Suggested Mitigation Techniques

Once the failure scenario is identified, it is relatively easy to resolve the problem. For the first
scenario (overshooting cell), standard RF optimization techniques, such as antenna downtilt

ALL RIGHTS RESERVED ALCATEL-LUCENT 2007 89


1 X EV-DO RF P ER FORM ANCE A NALYS IS & T ROUB LESHOOT ING G U IDE

and/or azimuth adjustments can be attempted to contain its coverage. The other option is PN re-
assignment, but care should be exercised to ensure it does not become a problem elsewhere.
Using a large PN increment will also mitigate the problem, though will require quite a bit of PN
planning re-work.

The second problem scenario can be resolved by maintaining sufficient number of cells between
PN re-use. A judicious design at the outset will go a long way in minimizing a potentially large
amount of effort spent resolving it later.

5.3.5 External interference

Concept/Reason for Failure

Presence of interference raises the noise floor, which raises the amount of power needed to
overcome it, further elevating the noise levels. As the cell or the AT runs out of power needed to
close the link, DRC rate request on the forward link and transmitted rate (on either link) decreases
impacting data rate performance. The level of impact, and hence, the ability to detect/resolve,
may depend on the amount of interference. An intermittent/weak transmitter may often be very
difficult to identify.

There are several flavors of External Interference that could impact performance of a network. It
can exist on the forward or the reverse link.

An unlicensed transmission within the CDMA band is the most common cause of
interference. Usually spectral clearance tests are performed in a new deployment to identify
and rectify such sources. If the tests were not executed, such as, when additional spectrum is
allocated to an existing operator, continued operation of interference sources could cause
significant performance issues as the commercial service is activated on the carrier.

Interference is also caused by spill over of a legitimate transmission from the adjacent
channel due to narrow spectral spacing, and/or, inadequate filter roll-offs implemented by
the transmitter or the receiver.

Another flavor of interference is where inter-modulation (IM) products generated at the


receiver amplifier due to a strong signal on neighboring channel (although within the
amplifier band) causes significant performance degradation.

Failure Symptoms and Identification strategies

Detecting presence of interference is not straightforward and locating the exact source may be
even more difficult. It often requires shutting down the desired transmission (that is, turn off
service) to pinpoint the root cause. Thus, it is best to isolate external interference upfront before
pressing system into commercial system.

The most telling indicator on the reverse link is much higher levels of RSSI rise, Eb/No setpoint
and RFER, all happening as a result of inflated noise floor. There are SM counts available to
inspect these per sector performance metrics RSSI (Short and Long Term Average RSSI
Rise pegged in the MCC), reverse Eb/No setpoint (Total Setpoint for Reverse Outer Loop
Power Control separate count for each reverse frame rate ranging from 0 to 153.6kbps, pegged

ALL RIGHTS RESERVED ALCATEL-LUCENT 2007 90


1 X EV-DO RF P ER FORM ANCE A NALYS IS & T ROUB LESHOOT ING G U IDE

in the TP), and RFER (Reverse Link Frame Error and Total Reverse Link Frame counts,
pegged in the TP). Corresponding performance metrics for these peg counts available in SPO
include: RSSI (One Second Average RSSI Rise, One Second Peak RSSI Rise, Ten Second
Average RSSI Rise, and Ten Second Peak RSSI Rise), and RFER (Reverse Frame Error rate,
Reverse Retransmission Failure rate and EVM DRC Erasure rate).

From prior field experience with such type of interference in 2G/3G1x networks, several sectors
facing the interference source will exhibit a simultaneous increase in the RSSI, Eb/No setpoint
and RFER. This is one of the surest indicators of external interference. To pinpoint the problem,
more expensive/elaborate means such as use of spectrum analyzer at/around the cell site may be
called for.

On the forward link, drive testing may help provide initial pointers to the presence of external
interference. Basically, the higher interference will result in very high mobile receive power, but
poor Ec/Io (Io goes up due to interference) and poor fwd FER. This, however, may also be caused
to due to missing a neighbor or an inadequate search window size; hence, these have to be
eliminated first along with any potential hardware/software issues. Turning off some cells or
service in the affected area may be required to allow for thorough spectral clearance tests.

On either link, one way to isolate problem related to IM interference is by using an in-line
attenuation to the mobile or the cell. If the measured signal power at either end (mobile receive
power on the forward link, or cell RSSI on the reverse link which is supposedly much higher than
the noise floor) increases at a much faster rate than the additional attenuation value, it suggests
the existence of IM products. Basically, the IM harmonics created due to amplifier non-linearity
de-emphasize at a larger rate than the additional attenuation value while the desired inband signal
attenuates at the nominal rate; therefore, the composite signal still attenuates at a larger rate.

Suggested Mitigation Techniques

As above, identifying the precise location/source of the interference is the most difficult first step.
Once the source is identified, the remedy is simply to turn it off.

5.3.6 Insufficient search window size

Concept/Reason for Failure

There are two main search windows whose sizes are of interest Active and Neighbor sets.
When searching the PN space to track/detect Pilots in its various sets, the AT utilizes a finite
window width in order to keep the search time small. The size is optimized to suit local terrain
and cell topology in order to maximize the likelihood of AT finding a pilot and its multipath
components in the smallest possible time.

Different considerations go into determining Active and Neighbor set search window size.
Accordingly, there are two failure modes.

When the Active set search window size is not configured large enough to allow AT to
receive/track a strong multipath component. Typically, the default search window size is
good enough to cover most of the significant components. In some rare cases, especially in

ALL RIGHTS RESERVED ALCATEL-LUCENT 2007 91


1 X EV-DO RF P ER FORM ANCE A NALYS IS & T ROUB LESHOOT ING G U IDE

large delay spread environments, this problem may surface. AT cannot combine/demodulate
the multipath since it falls outside the search window, rendering it as an interferer.

When the AT fails to detect a neighbor due to an inadequate Neighbor search window size.
More than the presence of multipath, it is often the relative path difference at the AT
between the reference and the neighbor cells that governs this window width. Since PN
offset transmitted by each cell is a time-shifted version of a single PN sequence, the
difference in propagation delays to the AT requires the Neighbor search window to be wide
enough to span the largest path differential. Otherwise, a particular neighbor may fall outside
the search window, causing it to interfere with the Active set.

Both cases of interference impact Data performance on the forward link as the AT requests lower
DRC rates amidst reduced SNR.

On the reverse link, not adding a strong Neighbor to the Active set may cause the call to drag,
potentially lowering the achieved data rate as well as increasing interference to other users.

When changing Active set search window size, it is usually a good idea to make the one used at
the cell site for reverse link demodulation, called Reverse Traffic Window Size, as well as for
acquiring soft handoff leg, called Data Window size, comparable. Both translation parameters on
BTSs section of EMS GUI.

Failure Symptoms and Identification strategies

SM counts do not provide a clear clue as to the existence of a search window problem.

For an overlay network, a basic sanity check is recommended to ensure Active and Neighbor
search window sizes are comparable to the 2G/3G1x underlay.

Drive test is often the most productive method to identify search window issues. LDAT can be
used to alert and recommend proper neighbor search window sizes. Based on the mobile logs, it
employs the phase-offset information for various Pilots reported on the Route Update Message
for an active call to identify potential for larger neighbor search windows. However, it cannot
identify situations where the existing neighbor search window is small enough to let AT even
detect the neighbor Pilot. Towards that end, another tool that could be of particular significance,
especially in tough to resolve problem areas, is a Pilot scanner, which employs independent GPS
time reference to provide an accurate view of multipath components from each cell.

Suggested Mitigation Techniques

Once the need for opening the search window for an Active or Neighbor set is identified, it can be
increased via translations. If a proper size is not known, it can be expanded in smallest possible
increments until performance is improved and/or mobile logs confirm proper detection at the
problem area.

ALL RIGHTS RESERVED ALCATEL-LUCENT 2007 92


1 X EV-DO RF P ER FORM ANCE A NALYS IS & T ROUB LESHOOT ING G U IDE

5.3.7 Failures due to faulty hardware

Concept/Reason for failure

Presence of faulty or mis-configured hardware can influence the performance on the affected cell.
Hardware failures can occur in various forms and intensity. For example, a CRC or a FLM/RLM
or a CBR board entering into faulty operation registers sudden degradation in data rate while
other indicators such as access and dropped calls may remain sane. In some cases, the affected
board impacts the transmit Signal to Noise Ratio to an extent that the mobile measuring decent
absolute receive power is not able to demodulate Pilot, thereby, requesting low forward link rates.
A different flavor of a hardware problem could be a faulty RF antenna cable with high return loss
likely resulting in smaller than desired cell footprint. In many cases, the hardware simply goes
into a faulty state, the hardware itself is fine software restore or stable clear fixes the problem.

It is also possible to witness intermittent hardware outage instead of an outright malfunction, such
as loss of T1/E1 during certain hours.

Failure Symptoms and Identification strategies

The first check should involve careful review of alarms generated and flagged by fault
management module of the network. Certain issues associated with receive path (such as RSSI
imbalance across main and diversity paths), malfunctioning hardware devices, backhaul outages,
etc. can easily be identified through the alarms.

The ROP analysis report in SPO provides information on these alarms at the AP, TP and BTS
levels. A pareto analysis of call set-up failures by specific hardware type based on ROP data can
offer a tell-tale sign of issue with a specific hardware/cell based on the over-abundant
concentration of the degraded data rate activity.

Suggested Mitigation Techniques

Once the suspected malfunctioning hardware is identified, first step is to attempt to rectify
through software restore or download/restore remotely. If the performance continues to suffer, it
may require cell visit to reseat the hardware and in some cases, to ensure tight cable connection.
If all on-site diagnostics fail, last step will be to replace the hardware.

5.3.8 Heavy reverse link traffic


Concepts/Reasons for failure

As the reverse link load increases, either due to higher number of active users or increased data
upload activity, it elevates the overall interference floor seen by each cell. In order to overcome
the raised floor, AT has to transmit at much higher power levels. An AT already at the edge can
no longer close the reverse link as its signal arrives with a weak signal-to-noise ratio at the cell.
This would sacrifice data rate achieved on the reverse link, especially for the users at the edge of
the coverage.

ALL RIGHTS RESERVED ALCATEL-LUCENT 2007 93


1 X EV-DO RF P ER FORM ANCE A NALYS IS & T ROUB LESHOOT ING G U IDE

A closely associated, but an extreme form of revere link loading is when it pushes the cell into
reverse overload. Reference [5] provides an overview of the Reverse Overload Control
algorithm. Some of the highlights are as follows.

Reverse overload control

Alcatel-Lucent 1xEV-DO network employs a proprietary algorithm to manage integrity of the


reverse link. Key objectives are:

To protect coverage and end-user Data performance on the reverse link

To manage reverse link loading within tolerable limits to avoid instability

To preserve integrity of DRC and ACK channels critical for forward link Data
performance

Current implementation offers a choice between two control mechanisms:

Rate limit message - reduces the reverse Data transmission rate used by all users in a sector
based purely on the number of active users.

Reverse Activity Bit (or RAB) - controls rate of all users based on RAB bit coupled with a
standards-defined probability decision matrix for rate transition. This is a dynamic
mechanism based on estimated reverse link loading and RSSI.

Either option constrains the transmitted rate to protect the reverse link, reducing data rate when
overload conditions are met.

Failure Symptoms and Identification strategies

The indication of high reverse loading condition can be inferred by increase in several SM counts,
relative to normal levels observed on other cells:

High values for peg counts introduced in R23 to measure Reverse Link Overload Control
including Short Term Average Fast Control Low Load count
(SM_SHTM_AVGLLC_FASTCTRL), Short Term Average Fast Control Medium
Load count (SM_SHTM_AVGMLC_FASTCTRL) and Short Term Average Fast
Control High Load count (SM_SHTM_AVGHLC_FASTCTRL). These peg counts
are available on a per sector basis.

High Reverse Frame Error rate (ratio of Reverse Frame Error and Total Reverse
Frame counts, pegged in the TP)

High average setpoint for various rates (ratio of Total Setpoint for Reverse Outer Loop
Control and Total Number of Frames for Reverse Outer Loop Control at each rate,
pegged in the TP)

ALL RIGHTS RESERVED ALCATEL-LUCENT 2007 94


1 X EV-DO RF P ER FORM ANCE A NALYS IS & T ROUB LESHOOT ING G U IDE

Increase in Number of Connections Force Released and Abnormal Release rate,


pegged in the MCC

Higher values for One Second and Ten Second Average/Peak RSSI Rise, pegged in
the MCC

Large number of AT/AN Initiated Connection Requests per hour, pegged in the AP

Regardless of whether data rate is degraded by heavy reverse activity or overload, some of these
indicators may also be influenced by reasons other than 1xEV-DO traffic, such as:

External interference resulting in elevated RSSI levels, high RFER and high average set
points.

A malfunctioning device in the RF receive path, such as CBR/UCR or low noise


amplifier, causing consistent or intermittent RSSI spikes, thereby, reducing the coverage.

A component breakdown in the RF receive path, or improper antenna cable connection,


incapacitating one of the two diversity branches. This would tend to increase the Eb/No
requirement as well as the power control variation. Not only will this require increase in
power transmitted by the mobile, but the variation will also push the cell further into
instability.

Hence, careful investigation of all possible causes may be required before a particular one can be
confirmed. Though not always necessary, one distinction may be that there are large number of
users consistently on the cell, as can be judged from AT/AN Initiated Connection Requests. It
is less likely that just a few users consistently load up the cell, usually a pointer, to external
interference or faulty hardware.

Suggested Mitigation Techniques

It is always a sound practice to track/trend various performance indicators, especially, Connection


Requests and RSSI rise to allow timely corrective action instead of waiting till performance
becomes unacceptable.

Once the cell is identified as a heavy traffic-bearing cell impacting data rate performance due to
reverse link loading or overload, there are a few options available to offload the traffic:

Increase the overload control thresholds to allow higher reverse transmission rates. With
R23, the HDR Reverse Link Overload Control (HROC) tunable parameters have become
translation parameters. The Alcatel-Lucent Reverse Overload Control mechanism utilizes
these tunable thresholds to manage the integrity of reverse link performance. Refer to [5] for
an overview of operation and associated translations. The ones of interest are Slow and Fast
Thresholds - 1, 2 and 3. Raising these thresholds will increase the probability that AT will
transmit at a higher rate, under load. However, this may come at the expense of elevated
reverse interference levels, potentially impacting access, drop call and reverse FER

ALL RIGHTS RESERVED ALCATEL-LUCENT 2007 95


1 X EV-DO RF P ER FORM ANCE A NALYS IS & T ROUB LESHOOT ING G U IDE

performance. Improvement in data rate due to higher transmission rates will to some extent
be capped by higher reverse FER.

This is not a preferred solution in the long-term. It may, however, be acceptable as a stopgap
measure until longer-term measures, stated next, are pursued. However, customer must be
made aware of the trade-offs before attempting any threshold adjustments.

One solution is to increase pilot overlap such that other links get into the Active set of the
call. This will tend to mitigate the traffic channel power requirement and hence, the overall
interference. This would enable ATs to sustain higher transmission rates at acceptable frame
rate, improving the reverse link data rate. However, careful consideration is warranted to
characterize potential impact on the forward link performance (more pilot overlap could hurt
pilot dominance) as well as increase in cell resource utilization (e.g., channel element
hardware).

Add a 1xEv-DO carrier.

Add a cell to serve the heavy traffic area. If adding a carrier is not a possibility due to lack of
spectrum, adding a cell (geographically separated from the high traffic cell(s)) is often an
effective, although a long-lead, solution to improve coverage and hence, permit higher levels
of offered traffic per unit area. Care must be taken to ensure proper RF optimization is
performed to minimize excessive Pilot with cells in the vicinity otherwise it may impact
forward link performance.

5.3.9 High forward link traffic

Concept/Reasons for failure

There is an inherent scheduler gain on the 1xEV-DO forward link that tends to increase overall
(or aggregate) data rate supported by a sector as the number of active users increases, to a certain
point. The proportionally fair scheduler rides along the SNR peaks experienced by individual
users at different times. In essence, each user is served when it experiences decent SNR (thereby,
requesting high DRC rate), and will be allowed to starve when other users are experiencing much
better SNR levels.

However, as with any wireless system, the shared air-interface bandwidth is limited; as a result,
the individual data rate will begin to suffer eventually in a 1xEV-DO system as the usage
increases. It can happen with fewer users if their applications are data intensive (such as, file
transfer, email attachments, video streaming, etc). On the other hand, a relatively larger number
of users can be supported per 1xEV-DO cell if the usage is primarily based on low demand data
applications (such as web browsing). In any event, action is warranted to offload traffic and
improve individual user data rate beyond certain traffic levels.

Failure Symptoms and Identification strategies

Growing disparity between the requested and allocated rates is a good measure of data starvation
at the users. There are two SM counts whose comparison may offer an approximation for this
metric - DRC Rate Requests and Total Packets Transmitted on the Forward link each
provided for various rates and pegged in the Modem. However, there are couple issues with this

ALL RIGHTS RESERVED ALCATEL-LUCENT 2007 96


1 X EV-DO RF P ER FORM ANCE A NALYS IS & T ROUB LESHOOT ING G U IDE

type of analysis, which will require experimenting with commercial data in order to confirm
validity and calibrate expectations. One issue is that the DRC Rate Request does not reflect the
amount of the data backlog at the AN, necessary to obtain truer view of the starvation. Second,
each packet may span over several DRC requests, so there may not be one-to-one correspondence
between the two quantities.

Another metric to trend is the total data transmitted (RLP Octets Transmitted on the Forward
link, pegged in the TP) per established call (numerator of the metric for Established Connection
Rate in section 5.1). Reduction in the metric will in general point to increased end-user
starvation.

Suggested Mitigation Techniques


As with combating any traffic related bottlenecks, it is always a good practice to track/trend
various performance indicators, such as ones mentioned above, to be able to anticipate and take a
timely corrective action.

Once the cell is identified as one suffering from degraded data rate performance due to user
contention on the forward link, there are a few options available to offload the traffic:

Investigate potential for RF optimization to fine-tune performance. It will be a good first step
to audit the cell for any issues related to coverage, pilot dominance, neighbor list omissions,
search windows size, external interference, etc. There may not be any systemic or blatant
issues at this stage of commercial operation, but the presence or development of several
marginal issues can come together to degrade performance over time. For example, recently
added cells in the area not well optimized, could cause overshoot/non-dominance in the
coverage of the cell under investigation.

Add a 1xEv-DO carrier. As scoped earlier, this document focuses on single carrier 1xEV-DO
deployments, so no further discussion is included here. It is mentioned here for future
considerations.

Add a cell to serve the heavy traffic area. If adding a carrier is not a possibility due to lack of
spectrum, adding a cell (geographically separated from the high traffic cell(s)) is often an
effective, although a long-lead, solution to improve coverage and hence, permit higher levels
of offered traffic per unit area. Care must be taken to ensure proper RF optimization is
performed to minimize excessive Pilot with cells in the vicinity otherwise it may impact
forward link performance.

5.3.10 Hybrid mode

Concept/Reasons for failure

Hybrid mode operation affords AT the flexibility to receive voice calls on a 2G/3G1x system
while on a 1xEV-DO Data session. The downside is that the AT has to periodically tune to
2G/3G1x system to look for pages. The time spent away from the 1xEV-DO system will
interrupt data transfer, lowering data rate. How often it searches the underlying system (that is,

ALL RIGHTS RESERVED ALCATEL-LUCENT 2007 97


1 X EV-DO RF P ER FORM ANCE A NALYS IS & T ROUB LESHOOT ING G U IDE

stays out of 1xEV-DO system) depends on the Slot Cycle Index (SCI)24 setting. Larger values
mean mobile can stay longer on the 1xEV-DO system and hence, smaller impact to data rate. It
has been estimated that with a SCI setting of 0 and in SIP mode, there could be data rate
degradation of up to 20% in the forward link and 30% in the reverse link compared to an EV-DO
only mobile.

It is possible that the AT makes conservative rate requests (transmit on the reverse link)
immediately upon return from the underlying system. This will have further impact on data rate
compared to non-hybrid mode operation. This type of logic will likely be AT implementation
specific.

Failure Symptoms and Identification strategies

There is no SM count to reflect hybrid mode operation. Check the AT configuration to see if this
operation is allowed. Work with the service provider to obtain information on data rate
performance of various terminal types in hybrid mode. The goal will be to verify that a particular
AT model/make performs within the average range of other well-characterized terminals. This
step will be important during initial 1xEV-DO deployments where behavior of different vendor
terminals in hybrid mode is not well understood.

It would also be useful to analyze the percentage of AT models/makes that use the older
MSM5500 chipset versus the ones that use the newer MSM6500 chipsets. ATs with MSM6500
chipsets are expected to have a more efficient ramp-up algorithm following a tune-away, which
improves reverse link data rate performance. In addition, these ATs also tend to complete a
1xEV-DO to 3G1X handoff faster than ATs with the older chipsets.

Suggested Mitigation Techniques

As mentioned earlier, if hybrid mode is enabled, work with the service provider to identify any
issues with hybrid mode performance of most prevalent terminals in the market. Another thing to
discuss with the customer is raising SCI. The current recommendation is to set the SCI value as 2
for 1xEV-DO PCMCIA card based ATs. As mentioned earlier, this will allow AT to tune away
from the 1xEV-DO carrier less frequently causing fewer gaps in data transfer. It must be noted
that the SCI can also be programmed at the mobile. The mobile will use the smaller of the SCI
value sent by the system and its programmed value. For example, if desired the service providers
have the option of setting the system SCI value to 2, program EV-DO hybrid mode AT to 2 and
program 3G1X mobiles to 1.

5.3.11 Backhaul restrictions

Concept/Reasons for failure

To realize full data rate capabilities of the 1xEV-DO air-interface, it is important to minimize
system bottlenecks elsewhere. A system can only deliver as good a Data experience as its

24
The effective SCI for the AT is the smaller of the one programmed in the AT and the maximum SCI, a cell site
parameter.

ALL RIGHTS RESERVED ALCATEL-LUCENT 2007 98


1 X EV-DO RF P ER FORM ANCE A NALYS IS & T ROUB LESHOOT ING G U IDE

weakest link can offer. One such link, which requires important considerations, is the T1/E1
backhaul.

Alcatel-Lucent 1xEV-DO system utilizes unchannelized T1/E1s for backhaul transport between
the cell and the FMS. (Note: Alcatel-Lucent 2G/3G1x infrastructure, in contrast, uses packet
pipes, which can be viewed as logical slices of T1/E1. Each packet pipe consists of multiple
DS0s).

To mitigate T1/E1 related data rate bottlenecks, Alcatel-Lucent currently recommends a


minimum of 2 T1/E1s per 3-sector single-carrier cell25. This is based on certain assumptions
from Systems Engineering on RLP layer data rate to be supported per sector as well as T1/E1
capacity. Reference [14] shows a range of supported data rates based on the number of T1/E1s.
One such scenario is as follows.

Assuming uniform traffic across sectors and the usable T1/E1 capacity of ~1100 and ~1400 kbps,
2 T1s/E1s per cell will be adequate to support 680 kbps/760 kbps RLP layer data rate per sector.
This is roughly 80%/92% of the 850kbps data rate that a typical 1xEV-DO sector may need to
offer when serving all dual-antenna ATs, with minimal backhaul bottlenecks. In reality, sector
loading is not uniform and hence, sectors can individually deliver peak data rate of 850kbps using
2 T1/E1s per cell.

In some situations, service providers may not be able to provision backhaul per Alcatel-Lucent
recommended guidelines due to cost issues or during the interim period as system is evolved over
time. One scenario likely to be encountered especially in some of the initial 1xEV-DO
deployments is use of 1 T1/E1 on each link. This will limit aggregate sector data rates achieved
on the cell on the forward link.

Failure Symptoms and Identification strategies

Evaluate the T1/E1 service measurement counts DL_UPLINK_AVG_THRUPUT,


DL_UPLINK_PEAK_THRUPUT, DL_DOWNLINK_AVG_THRUPUT and
DL_DOWNLINK_PEAK_THRUPUT

Anytime maximum achieved data rate on a sector is suspected to be much lower than the average
seen on a cell with adequately provisioned backhaul, it is a good idea to check number of T1/E1s
configured on the cell. If they are not configured per Alcatel-Lucent recommendations, the
service provider should be informed.

If the provisioning is adequate, look for issues related to T1/E1 outage, or defective T1/E1
terminating hardware at the cell. Hardware outage/defects are more likely to cause intermittent or
sudden impact as opposed to under-provisioning, which will result in consistently reduced levels
of data rate since the rollout of 1xEV-DO carrier.

Suggested Mitigation Techniques

25
Two T1s are necessary to achieve a decent data rate performance on the forward link. Only one is needed on the
reverse link. The difference stems from the asymmetry in maximum air-interface transmit rates as defined in IS856.
However, since T1s are bi-directional, Alcatel-Lucent recommends minimum of two.

ALL RIGHTS RESERVED ALCATEL-LUCENT 2007 99


1 X EV-DO RF P ER FORM ANCE A NALYS IS & T ROUB LESHOOT ING G U IDE

After confirming that the degraded data rate performance is on account of under-provisioned
T1/E1s, alert it to the operator and suggest appropriate backhaul adds.

5.3.12 Handoff rejects

Concept/Reasons for failure

This category of handoff failures is defined as those that do not result in Traffic Channel
Assignment message sent to include a candidate Pilot.

Handoff attempts may be denied if a candidate cell is unable to assign resource due to shortage,
or if the current Active set size is already at the maximum allowed translation value. In some
situations, errors such as traffic channel activation failure on the candidate cell can also block
handoff requests. Another reason would be the candidate cell losing GPS synchronization, also
known as Island cell problem.

In any case, the SNR on one or both the links will degrade due to growing pathloss as the AT
approaches the cell it was denied connection with.

The poor SNR levels due to call drag will lower the transmitted rates, thereby, impacting Data
data rate. On the reverse link, higher AT transmit power levels will also create more interference,
further degrading data rate performance for all users.

In many cases, the failed handoffs will also result in dropped calls.

Failure Symptoms and Identification strategies

There are separate SM counts pegged in the AP to facilitate investigation of handoff rejects due to
resource shortages. These are:

Soft and softer handoff failures lack of resources in the candidate sector

Soft and softer handoff failures max number of legs reached

Miscellaneous handoff rejects (such as, due to traffic channel activation failure) will be pegged
under Soft and softer handoff failures Other Reasons in the AP.

As the cell begins to hit any of the above limits, other performance indicators, mainly dropped
call rate, will also suffer.

Any lack of resources will typically be due to the cell running out of channel elements. Note that
with one FLM/RLM, there are 93 CEs available on a 3-sector cell. In rare cases, where a sector
takes on a large share of traffic, it could run out of MAC Indices (max 59 MAC Indices available

ALL RIGHTS RESERVED ALCATEL-LUCENT 2007 100


1 X EV-DO RF P ER FORM ANCE A NALYS IS & T ROUB LESHOOT ING G U IDE

per sector) before all CEs on the cell are used or the other limitation of 48 Max Number of
Users26 per sector limit is reached.

The handoff rejects due to Max Number of Legs reached is a straightforward one to identify
directly based on the SM count mentioned above.

Miscellaneous handoff reject causes may be more difficult to identify. If they occur at an
unacceptable rate, development support may be needed for more detailed tracing in order to get to
the root cause.

Suggested Mitigation Techniques

The workarounds depend on the root cause.

Lack Of Resources In The Candidate Sector:

One solution is to reduce the coverage of the cell in an attempt to offload traffic to the
neighboring cells/sectors. This, however, assumes the neighboring cells have the room to handle
more traffic, which may not be the case all the time.

An alternative is to add multiple carriers. This is a more graceful solution than adjusting
coverage, which may not always yield desired traffic offloading. At this point, only single carrier
deployments are considered.

Where there are spectrum availability issues, adding a cell may be the last resort solution. This, of
course, requires longer lead times in order to accommodate zoning, cell installation, RF
calibration and optimization stages before turning the cell over to commercial service.

Max Number Of Legs Reached:

One method to mitigate handoff failures due to full active set is to allow more pilots in the Active
set. That is, the Maximum Legs in Handoff translation can be increased to 6.

Note, however, that there could be potential downsides to expanding the Active set (e.g., use up
more MAC Indices on the forward link and increase hardware utilization on the reverse link). It
may still be a good interim step, however, especially if the problem areas are localized and not
severe in nature.

A better/longer-term solution would be to resort to traditional drive test based RF optimization


techniques and attempt to improve dominant coverage in the region (see section 5.2.2). This will
limit the likelihood of encountering a full active set when a strong candidate pilot needs to be
added. Another option after exhausting efforts to improve dominant coverage in the area is to
experiment with higher values of Pilot Detection Threshold and Pilot Drop Threshold. These
translation parameters are in the Sectors section of EMS GUI and the generally recommended
values are 9 and 11 dB respectively. Higher values will tend to keep Active set smaller. In

26
The Maximum Number of Users constraint may block new call attempts only. Handoffs are not blocked. So, if a
sector receives a lot of soft handoff traffic, the total number of users supported can exceed the limit, potentially
bumping against the MAC Index limit.

ALL RIGHTS RESERVED ALCATEL-LUCENT 2007 101


1 X EV-DO RF P ER FORM ANCE A NALYS IS & T ROUB LESHOOT ING G U IDE

future, the availability of dynamic add/drop thresholds (similar to IS95B handoffs enhancements)
will also help reduce Active set size.

Miscellaneous Reasons:

As mentioned earlier, this will require more detailed tracing to identify the root cause and fix
accordingly. Simple steps such as restoring the cell, reseating/replacing the FLM/RLM,
identifying issues with reference drifts might first be attempted to mitigate/eliminate handoff
failures due to other reasons.

5.3.13 Core IP network issues

Concept/Reasons

Performance issues in the core IP network will directly affect the data rate. These issues in the IP
network could include latency, packet drop rate and degraded data rate. Data to and from the user
flows through the PCF, PDSN and several routers. Sub-optimal performance of any of these
network elements or the interface between these elements could result in degraded data rates.

Failure Symptoms and Identification strategies

The general failure symptom is lower forward and reverse data rate rates observed from service
measurements. In addition, there are some secondary metrics that indicate the performance of the
link between the PCF and the PDSN.

Higher than normal rate of PCF-PDSN failure rate (ratio of Errors in A10 interface and RLP
octets sent to PCF) indicates that there are performance issues in the A10 interface between the
PCF and the PDSN.

Call processing failure and alarm analysis in SPO also provides some information on failures on
the PCF and the link between the PCF and the PDSN. Error codes and reject codes on the call
processing failures provides additional information on the failure [20].

Suggested Mitigation Techniques

One way to clearly identify and mitigate the problem in the core IP network is to use the Packet
Data Monitoring (also known as Data Quality of Service, DQoS) tool. The PDM tool collects
and analyzes data from different nodes in the network. Correlated analysis of data from different
monitoring points provides information about which network node and/or link is performing non-
optimally. However, deploying this tool requires physically hooking up monitoring points at
different points, see Appendix C for more information.

ALL RIGHTS RESERVED ALCATEL-LUCENT 2007 102


1 X EV-DO RF P ER FORM ANCE A NALYS IS & T ROUB LESHOOT ING G U IDE

APPENDIX A METRIC CROSS-REFERENCE

Table A-1 Established Connection Rate Metric Cross Reference

KPI Metric Name Sections


Established Connection Rate Abnormal Release rate 5.1.5
AN initiated RF Failure rate 5.1.2
AT initiated RF Failure rate 5.1.2
Inter Subnet Idle Transfer Failure Rate 5.1.1.2, 5.1.2.8
Connection Request with Four or More Pilots (%) 5.1.2.2
Connection Request with One Pilot (%) 5.1.2.2
Connection Request with Three Pilots (%) 5.1.2.2
Connection Request with Two Pilots (%) 5.1.2.2
Established Connection Rate 5.1
Established Connection Rate [Peg] 5.1
Established Connection Rate [User] 5.1
Paging Effectiveness [BE & other profiles] 5.1.1.2
EVM DRC Erasure Rate 5.1.2.5
Fast Connect Success Rate 5.1.1.3
Fast Connect Success Rate [Peg] 5.1.1.3
RF Failure Rate 5.1.2
RF Failure Rate [Peg] 5.1.2
Connection Blocking Rate 5.1.3
Connection Blocking Rate User 5.1.3
Number of times Maximum Connections Reached 5.1.3.2
Number of Times TP is in Overload 5.1.3.1
One Second Average RSSI Rise 5.1.2.5, 5.1.5
One Second Peak RSSI Rise 5.1.2.5, 5.1.5
Page Failure rate 5.1.1.2, 5.1.2.8
Paging Effectiveness 5.1.1.2, 5.1.2.8
RAN Authentication Failure rate 5.1.7
RATI Session Setup Requests 5.1.2.7
Reverse Frame Error rate 5.1.2.5, 5.1.5
Reverse Retransmission Failure rate 5.1.2.5
Sessions Denied due to TP Overload 5.1.3.1
Soft Handoff Attempt to Assign rate 5.1.2.3
Ten Second Average RSSI Rise 5.1.2.5, 5.1.5
Ten Second Peak RSSI Rise 5.1.2.5, 5.1.5
Total Connection Requests 5.1.5
TP Overload Duration 5.1.3.1
TP Processor Usage 5.1.3.1

ALL RIGHTS RESERVED ALCATEL-LUCENT 2007 103


1 X EV-DO RF P ER FORM ANCE A NALYS IS & T ROUB LESHOOT ING G U IDE

Table A-2 Dropped Connection Rate Metric Cross Reference

KPI Metric Name Sections


Dropped Connection Rate Abnormal Release rate 5.2.9
Connection Release rate Other Reasons 5.2.8
Connection Request with Four or More Pilots (%) 5.2.3
Connection Request with One Pilot (%) 5.2.3
Connection Request with Three Pilots (%) 5.2.3
Connection Request with Two Pilots (%) 5.2.3
Dropped Connection Rate 5.2
Dropped Call rate due to HO Failures 5.2
Established Connection Rate 5.2.2
EVM DRC Erasure Rate 5.2.6
One Second Average RSSI Rise 5.2.6, 5.2.9
One Second Peak RSSI Rise 5.2.6, 5.2.9
Page Failure rate 5.2.2
Reverse Frame Error rate 5.2.2, 5.2.6, 5.2.9
Reverse Retransmission Failure rate 5.2.6
Soft Handoff Attempt to Assign rate 5.2.4
Soft Handoff Failure rate 5.2.13
Soft Handoff Failure rate at target sector 5.2.13
Ten Second Average RSSI Rise 5.2.6, 5.2.9
Ten Second Peak RSSI Rise 5.2.6, 5.2.9
Total Connection Requests 5.2.9

Table A-3 Data Rate Metric Cross Reference

KPI Metric Name Sections


Data Rate Abnormal Release rate 5.3.8
Average Forward Data Burst rate [Kbps] 5.3
Average FWD Link MAC Data Rate [EVM] [kbps] 5.3
Average FWD Link Physical layer Data Rate
5.3
[SBEVM] [kbps]
FWD Link Physical Layer Data Rate per User
5.3
[SBEVM] [kbps]
Average FWD Link RLP Data Rate [EVM] [kbps] 5.3
Average FWD Link RLP Data Rate [SBEVM]
5.3
[kbps]
Average FWD Link RLP Data Rate per User
5.3
[SBEVM] [kbps]
Average Reverse Outer Loop Frames rate [Kbps] 5.3
Average REV Link Physical Layer Data Rate [EVM] 5.3

ALL RIGHTS RESERVED ALCATEL-LUCENT 2007 104


1 X EV-DO RF P ER FORM ANCE A NALYS IS & T ROUB LESHOOT ING G U IDE

[kbps]
Average REV Link Physical Layer Data Rate
5.3
[SBEVM] [kbps]
Connection Request with Four or More Pilots (%) 5.3.2
Connection Request with One Pilot (%) 5.3.2
Connection Request with Three Pilots (%) 5.3.2
Connection Request with Two Pilots (%) 5.3.2
EVM DRC Erasure rate 5.3.5
Forward Transmit Data rate 5.3
One Second Average RSSI Rise 5.3.5, 5.3.8
One Second Peak RSSI Rise 5.3.5, 5.3.8
PCF PDSN Failure rate 5.3.13
Reverse Frame Error rate 5.3.1, 5.3.5, 5.3.8
REV Link RLP Data Throughput [EVM] 5.3
REV Link RLP Data Throughput [SBEVM] 5.3
Reverse Retransmission Failure rate 5.3.5
Soft Handoff Attempt to Assign rate 5.3.3
Soft Handoff Failure due to other reasons 5.3.12
Ten Second Average RSSI Rise 5.3.5, 5.3.8
Ten Second Peak RSSI Rise 5.3.5, 5.3.8

ALL RIGHTS RESERVED ALCATEL-LUCENT 2007 105


APPENDIX B O V E RV I E W O F S P O T O O L

System Performance Optimization Tool (next generation SPAT3G) is a PC-based tool that is
designed for rapid troubleshooting, system performance analysis and RF optimization of CDMA,
and 1xEV-DO networks. SPO for 1xEV-DO has the ability to analyze Service Measurements
(SM), ROP, Neighbor lists and Configuration data, HOM/UNL and PCMD data simultaneously.
SPO supports 1xEV-DO releases 26 and beyond.

SPO structure

There are three components in SPO the OMP scripts, known as the DFDP (Data Filtering and
Data Packing), the DSM (Dedicated SPO Module) and the SPO PC GUI tool. The DFDP scripts
have to be installed on the OMP to collect SM, ROP, Configuration, HOM/UNL and PCMD data
on a daily basis. A cron job can be set up on the OMP to run these scripts a few minutes past
midnight every day to collect the relevant data. At the completion of the cron job, a single zipped
file is generated that is an archive of the SM, ROP, Configuration, HOM/UNL and PCMD data
that was collected for that day. The naming convention used for the zipped file is SN-
Number_yyyymmdd.spr (e.g. 1_20040901.spr). Users may specify a different date, release, range
of hours, System ID, or SN number using command line options.

The spr file needs to be downloaded to the PC from the OMP everyday for further analysis. The
DSM portion of the SPO tool processes the spr file(s) along with the cells information (e.g. the
LCAT file) and creates an SP file. The SP file is read in by the SPO PC Tool to create datasets.
Datasets are used in displaying the SM performance metrics, ROP analysis, Configuration and
HOM/UNL summaries, NL Alert analysis and PCMD metrics and analysis. The cell information
and street maps can be combined into a Geoscheme in SPO which can be re-used to create
multiple datasets for the same market (e.g. one dataset for each month of the year). The next few
sections explain the different types of 1xEV-DO-specific analyses that are available with SPO
once the data has been processed.

Service Measurements analysis

The sp files contain raw service measurement peg counts that are used within the PC portion of
SPO to compute performance metrics based on hard-coded equations (e.g. Connection Failure
rate, Forward Data rates etc.). These performance metrics are available on a per SN, per
RNC/FMS, per AP, per OHM, per TP, per cell and per sector depending on the peg counts used
for calculations.

Appendix Figure 1 shows the executive summary of SPO. The executive summary shows key
performance metrics on a per system level for the whole day (24-hour), for the busy hour for each
day for which SM data is available, or for the individual hours of the day. Each metric is color-
coded red when it exceeds a certain threshold. This enables the user to rapidly identify which
metric is under-performing in a system. All the performance metrics are classified under multiple
categories available as individual tabs in SPO. For example, Dropped Call rate and Connection
Failure rate metrics are available under the Failures tab in SPO.

ALL RIGHTS RESERVED ALCATEL-LUCENT 2007 FOR INTERNAL USE ONLY


USE PURSUANT TO COMPANY INSTRUCTIONS
1 X EV-DO RF P ER FORM ANCE A NALYS IS & T ROUB LESHOOT ING G U IDE

APPENDIX FIGURE 1 EXECUTIVE SUMMARY

APPENDIX FIGURE 2 SM SUMMARY WINDOW AND PEG COUNT DESCRIPTION

ALL RIGHTS RESERVED ALCATEL-LUCENT 2007 107


1 X EV-DO RF P ER FORM ANCE A NALYS IS & T ROUB LESHOOT ING G U IDE

Appendix Figure 2 shows the per cell level view of a certain metric. Double-clicking on any
particular metric or peg count in an SM Table opens up a definition window for the metric or the
peg count. The formula used for the calculation of the metric is available as part of the display
window along with the threshold that is used to color-code the metric. SPO metrics are also
available in map view format. Appendix Figure 3 shows one such example of a performance
metric on a map. Each cell/sector is color-coded according to the value of the metric that is being
displayed.

SM metrics and peg counts can also trended if data from multiple days is available. Appendix
Figure 4 shows an hourly trend of a metric for several days worth of data.

APPENDIX FIGURE 3 SM MAP

ALL RIGHTS RESERVED ALCATEL-LUCENT 2007 108


1 X EV-DO RF P ER FORM ANCE A NALYS IS & T ROUB LESHOOT ING G U IDE

APPENDIX FIGURE 4 SM TREND

ROP Analysis

SPO analysis of 1xEV-DO ROP data is done on a per AP, per TP and per cell level. In addition
to the 1xEV-DO alarm analysis, the 1xEV-DO ROP analysis feature in SPO has been updated to
include call processing failure analysis. Appendix Figure 5 shows the system level view of
1xEV-DO ROP Reports for AP, TP and cell CP Fails and Alarms. Appendix Figure 6 shows the
ROP Tables for System Level view of call processing failures for several hours.

ALL RIGHTS RESERVED ALCATEL-LUCENT 2007 109


1 X EV-DO RF P ER FORM ANCE A NALYS IS & T ROUB LESHOOT ING G U IDE

APPENDIX FIGURE 5 ROP REPORTS CP FAILS AND ALARMS ANALYSIS

APPENDIX FIGURE 6 ROP TABLES CALL PROCESSING FAILURE ANALYSIS

ALL RIGHTS RESERVED ALCATEL-LUCENT 2007 110


1 X EV-DO RF P ER FORM ANCE A NALYS IS & T ROUB LESHOOT ING G U IDE

Configuration Summary

The existing translations settings for a system can be viewed using SPO. As part of the OMP
scripts, SPO picks up the XML file that is generated on the OMP using the cmexportrun utility.
The XML file contains the translations/configuration settings from all the database forms and for
all cell/sectors in the system.

SPO color-codes any translation setting that is different from Alcatel-Lucent recommended
values. Comparison of this setting can also be made against specific user-set values also and this
is color-coded differently.

A parameter definition window pops up when the user double-clicks on a particular translation
grid. The translations summary can also viewed for all the cells/sectors at the same time or for
one cell/sector at a time. The difference between translations settings for two different cells or
two different days can also be done in SPO. Appendix Figure 7 shows a sample configuration
summary window with a parameter definition window.

APPENDIX FIGURE 7 CONFIGURATION SUMMARY WINDOW

ALL RIGHTS RESERVED ALCATEL-LUCENT 2007 111


1 X EV-DO RF P ER FORM ANCE A NALYS IS & T ROUB LESHOOT ING G U IDE

NL Alert Analysis

NL Alert feature enables the optimization of 1xEV-DO neighbor lists. The different types of
warnings that are generated include Reciprocity alerts, One-way and Two-way PN ambiguity
alerts, AP IP alerts and reciprocal search window alert, among others

In addition, the NL Alert tabular report, a map view is available for the current neighbor list and
for all the individual alerts. This enables the user to visualize where the neighbor list problem is
located.

SPO also displays distance between source and target sectors for all NL Alerts if cell location
information is available as a Geoscheme. This data may be used to prioritize NL Alert fixes.
Appendix Figure 8 shows a sample NL Alert summary window with a map indicating the
locations of the cells involved in the alert.

APPENDIX FIGURE 8 NL ALERT MAP & SUMMARY

Handoff Matrix and Undeclared Neighbor List analysis

ALL RIGHTS RESERVED ALCATEL-LUCENT 2007 112


1 X EV-DO RF P ER FORM ANCE A NALYS IS & T ROUB LESHOOT ING G U IDE

Handoff Matrix and Undeclared Neighbor List analysis features help optimize 1xEV-DO
neighbor lists. Handoff Matrix data summarizes all the handoffs attempted and completed
between a source sector and target sectors. Handoff Matrix analysis in SPO reports the Handoff
Add and Drop failure rates and utilizes the Handoff Add Attempts to recommend adding or
deleting neighbors. Appendix Figure 9 shows a sample Handoff Matrix summary report and the
add/delete neighbor map.

On the other hand, UNL data provides information about pilots that are strong enough to be
considered for handoffs but are not in the neighbor list. UNL analysis in SPO reports all the
remaining set pilots and the corresponding number of undeclared neighbor events.

APPENDIX FIGURE 9 HANDOFF MATRIX ANALYSIS SUMMARY

PCMD Analysis

With R26, PCMD data is available on the OMP for 1xEV-DO. SPO supports 1xEV-DO PCMD
analysis with the following reports: PCMD Metrics, Latency Histograms, Drop Connection
Analysis and Connection Failure Analysis.

ALL RIGHTS RESERVED ALCATEL-LUCENT 2007 113


1 X EV-DO RF P ER FORM ANCE A NALYS IS & T ROUB LESHOOT ING G U IDE

The Drop Connection and Connection Failure Analysis report provides pilot set information for
active and non-active set information for dropped calls and access failures based on RUM data as
well as drop connection and connection failure reasons and possible root cause information. The
PCMD Metrics Summary analysis provides performance metrics derived from PCMD recorded
connection and session events. Appendix figure 11 highlights the output of the connection failure
analysis report in SPO.

The PCMD latency histogram feature provides histogram plots for the following metrics: Session
Setup/Transfer Request to Complete, Connection Request to Established Connection and
Configuration Negotiation Start to End. Appendix figure 12 highlights the output of the latency
histogram report in SPO.

APPENDIX FIGURE 11 PCMD CONNECTION FAILURE ANALYSIS

ALL RIGHTS RESERVED ALCATEL-LUCENT 2007 114


1 X EV-DO RF P ER FORM ANCE A NALYS IS & T ROUB LESHOOT ING G U IDE

APPENDIX FIGURE 12 PCMD LATECNY HISTOGRAM PLOTS

Other features in SPO

There are other features in SPO that have not been described in detail in the previous sections that
are useful for performance analysis. One such key feature is the ability to create a subset. A
subset could be created as a collection of a few RNCs/FMSs, cells, sectors or carriers (e.g. cells
within a cluster). The subsets performance can be compared against the performance of the
system as a whole. Another useful feature is the ability to create a superset, which allows the
users to aggregate service measurements data for a group of days.

The user defined metrics feature allows the users to create their own metrics. With this feature,
users are allowed to define a new service measurement metric or modify an existing service
measurement metric.

The user defined category feature allows the users to create a new category with a set of service
measurement peg counts and/or metrics. Once created, the user defined category will be
available as a new tab in the SM table window.

ALL RIGHTS RESERVED ALCATEL-LUCENT 2007 115


1 X EV-DO RF P ER FORM ANCE A NALYS IS & T ROUB LESHOOT ING G U IDE

Even in the absence of SM and/or ROP files, SPO can also be used to view translations
summaries and perform NL Alert analysis.

Any table that is displayed in SPO can also be exported as a comma-delimited file or as an MS
Excel spreadsheet. This file can then be used for further analysis if the need exists. SPO also has
the mechanism to report bugs or feature requests. This can be done from the Help menu
option and an email from the users computer is sent automatically to the SPO team.

User preferences can be set from the Preferences option. Some examples of preferences are
setting working directory, and defining custom ranges and labels for maps.

SPO provides a how to user guide within the tool in the form of Online Help.

SPO is now available for external customers (service providers like Alltel, USC, etc.). Service
providers can sign up for SPO on an annual service subscription basis with the Reality Services
Platform (RSP) hosting the processed data. The user in the service provider network will have
the SPO application installed on their computers and will have access to the data from within the
application through the internet.

ALL RIGHTS RESERVED ALCATEL-LUCENT 2007 116


1 X EV-DO RF P ER FORM ANCE A NALYS IS & T ROUB LESHOOT ING G U IDE

APPENDIX C O V E RV I E W O F P D M TO O L

The Packet Data Monitoring (PDM) tool is used by engineers to analyze and troubleshoot packet
data networks. Analysis is available from end-to-end, e.g. from the AT through the RAN
through the backhaul into IP networks and remote backhaul networks. Analysis can be
segmented to links within the entire packet data domain. By sniffing packet data at various
locations within the network, engineers are able to identify trouble spots and bottlenecks of
packet data traffic. This allows engineers to optimize network performance and resolve critical
network issues. The tool is also used to show compliance with established KPIs.

PDM is built upon scripts and code that have been used within Alcatel-Lucent since the late
1990s. The Voice and Data Quality Performance and Analysis group has used the sniffer (MP)
portion of PDM to troubleshoot issues in live customer networks, and for product verification
testing within in-house labs. PDM formalizes and documents these previously implemented
scripts. At the same time, PDM adds many additional capabilities such as an easy-to-install/easy-
to-use GUI, remote control of MPs, test automation, analysis of SIP messaging, and the Analyzer
Server in the Reality Service Platform (RSP), as well as many additional analysis capabilities.

The current version of PDM is geared towards 3G EVDO networks (Rev0 and RevA), and
UMTS. However, PDM can capture data packets from any network components as long as an IP
aggregate tap or port monitor can be accessed.

PDM contains troubleshooting functionality which is either non-existent or difficult to obtain


using other tools:

Analyze packet performance across RF domain and backhaul/IP network


Application layer, Transport layer, Network layer, Data Link layer, RLP, RF Physical layer
Precision latency measurements
o Best in market
o PDM provides precise data correlation across the entire network based on GPS
timestamping (0 to 20 microsecond accuracy)
o No timing synch cables needed between components; No reliance on NTP, which
is unreliable for precise latency measurements
Data security
o Encryption of control messages (AES-128) and collected data (HTTPS 128 bit
SSL)
Easy to use
o Push button control
o Automated performance testing and analysis features
Remote control numerous collection locations from one user
o While performing drive tests
o While sitting at a home office or remote location
o In the lab
o In a customers network

Generally speaking, PDM use is as follows:

ALL RIGHTS RESERVED ALCATEL-LUCENT 2007 117


1 X EV-DO RF P ER FORM ANCE A NALYS IS & T ROUB LESHOOT ING G U IDE

PDM Monitoring Points are set up at locations throughput the network. Next, packet data is
monitored in the network. The type of data that is being collected, and the source of the data,
is based on your needs and on what is available to you.

Data is collected at the MPs, with collection based on specific user-defined data filters so that
only the necessary data is captured. Each MP is also synchronized with a GPS time stamping
unit, so that all collected data throughout the network at multiple MPs can be precisely
correlated. The client laptop/PC can have a test phone plugged into it, and can optionally run
Qualcomms QXDM, to collect data for both EVDO Rev0 and RevA.

After data has been collected, it is transferred to the Analyzer Server for storage, processing,
and report presentation.

PDM is comprised of 3 components:

1. Client Application: The Client Application functions both as a Monitoring Point,


and for configuring and controlling Monitoring Points (local and remote) and
tests. One engineer can use the Client Application to remote control any number
of MPs in the network, for data collection. The Client Application contains built-
in tests, for Automated or manual testing of the network. Tests include FTP,
HTTP, PING, Traceroute, audio download, and video download.

2. Monitoring Points (MP): Installed on laptops/PCs and placed at various strategic


locations throughout the network. MPs sniff the packet traffic on the network
based on filter settings. The PDM Client Application also functions as a
Monitoring Point. The Monitoring Point can collect data generated by the Client
Application, or any other data that is passing by (through) the network tap point.
This means that PDM can collect and analyze data generated by other sources
and other applications beyond the built-in tests.

3. Analyzer Server: After data is collected at the MPs, it is transferred to the


Analyzer Server for data processing and presentation. A connection to the
Analyzer Server is then required via a browser to select data for processing, view
reports, and download analyzed data. In the Alcatel-Lucent hosted e-Service
architecture, the Analyzer Server is the hosted server on Alcatel-Lucents Reality
Services Platform (RSP).

The current version of PDM v4.11 provides the following list of analysis presentation via the
Analyzer Server:

Critical Events (errors, packet drops, etc.)


Performance Averages
Plus, layer specific output:

Layer Data
Application Layer User Perceived Throughput, Average Latency, SIP/SDP analysis
TCP Layer Timeout, Out-of-Sequence, Reset, Checksum Error, Sequence Number, Outstanding Window
UDP Layer Throughput, latency, dropped packets, port analysis

ALL RIGHTS RESERVED ALCATEL-LUCENT 2007 118


1 X EV-DO RF P ER FORM ANCE A NALYS IS & T ROUB LESHOOT ING G U IDE

IP Layer Throughput, Latency, Packet Size, and Packet Drops, DiffServ Code Point packet analysis
PPP Layer PPP Events, Throughput, Latency, Packet Size, Packet Drops
RLP Layer Throughput, Retransmit PDU, NAKd PDU, Reset and CRC Error, BLER, Ec/Io
RF Physical Layer SIR/SINR, DRC Request, Throughput

For EVDO systems, QXDM is required for gathering statistics for the RLP and RF Physical
Layer.

The following diagram shows one example of the use of PDM in an EVDO network:

Important notes about this diagram:

This is only one permutation example of PDM in a live customers network. Many permutations
are possible:

ALL RIGHTS RESERVED ALCATEL-LUCENT 2007 119


1 X EV-DO RF P ER FORM ANCE A NALYS IS & T ROUB LESHOOT ING G U IDE

Less or more MPs


Test traffic can be introduced by the Client Application
A Remote Phone is not required
Tests can be run via Ethernet, i.e. without going over the air
Tests can be run using commercial ATs or wireless cards
The engineer will work at the Client Application, in order to remote control all the remaining
components of the system, including the Remote Phone.
The Remote Phone is a special purpose MP, which has a phone installed, and is remote
controlled from the Client Application by PDM.
Both Client and Remote MPs have QXDM installed. This is an optional component. PDM
can analyze packets from Client Application to Remote Phone without QXDM installed.
Client MP controls tests
o Starts and stops tests
o Remote controls each MP to collect data and transfer data to the AS
o Remote controls QXDM at both itself and the Remote MP
For internal ALU use, the Analyzer Server can be installed directly on the engineers Client
Application laptop and data can be transferred back to the AS via the intranet.

PDM Monitoring Point hardware:

PDM uses standard Windows laptops, with specialized decoder software and GPS timestamping
hardware. The GPS hardware is relatively cheap in price. The following picture gives an
example of a Client Application Monitoring Point kit. PDM MPs also require some means to tap
into the network, e.g. via an IP aggregate tap.

ALL RIGHTS RESERVED ALCATEL-LUCENT 2007 120


APPENDIX D R E C E N T P E R F O R M A N C E A F F E C T I N G A L U A L E RT S

ALU Alert Title Product

07-0809 1xEV-DO - Modem related Service Measurement counts for Mobility Products
Sectors 4, 5 and 6 in the HCS SM files do not contain valid data.
Flexent CDMA 1xEV-DO

07-0790 EVDO RNC - After loading R28 SU3 Call Blocking rate is higher Mobility Products
due to stale connections at Traffic Processors (TPs)
Flexent CDMA 1xEV-DO

07-0789 EVDO RNC R28 SU3 CFT E -High Failed Call Attempts (FCA) Mobility Products
for BTSs executing routine diagnostics in the maintenance
window Flexent CDMA 1xEV-DO

07-0779 1xEV-DO Service Meas. files may have zero counts & PCMD Mobility Products
records may not be getting generated, call processing is normal.
Flexent CDMA 1xEV-DO

07-0725 1xEV-DO TPs go into and stay in the MAJOR alarm state and are Mobility Products
not selected to service any new calls
Flexent CDMA 1xEV-DO

07-0710 1x-EVDO -Increase in Dropped Call rate on some RNCs after Mobility Products
loading RNC Software Release R28.0 SU3 CFT C
Flexent CDMA 1xEV-DO

07-0692 1x-EVDO RNC release R28.0 - Increased soft handoff failures on Mobility Products
six sector MLPPP cells.
Flexent CDMA 1xEV-DO

07-0659 Idle Session increased after retrofitted to R28 Mobility Products

Flexent CDMA 1xEV-DO

07-0555 1xEVDO Autonomous SBEVM Resets observed with Cell Mobility Products
R28.00
Flexent CDMA 1xEV-DO

07-0510 AP high PO caused by AMWExternalController Mobility Products

Flexent CDMA 1xEV-DO

07-0537 EVDO RNC: The heartbeat loss alarm between the RNC and the Mobility Products
BTS may not get cleared.
Flexent CDMA 1xEV-DO

07-0474 Preventive Published and Distributed Mixed DBEVM/SBEVM Mobility Products


Configurations on the Same Cell with Multiple Carriers
Flexent CDMA 1xEV-DO

07-0473 Recommended Settings for EV-DO Cell Access Channel Mobility Products
Parameters for DB-EVM carriers
Flexent CDMA 1xEV-DO

07-0393b Flexent Modcell Extended Heartbeat for Improved Resistance to Mobility Products
DS1 Outages
Flexent Modular Cell 2.0

Flexent Modular Cell 3.0

ALL RIGHTS RESERVED ALCATEL-LUCENT 2007 FOR INTERNAL USE ONLY


USE PURSUANT TO COMPANY INSTRUCTIONS
1 X EV-DO RF P ER FORM ANCE A NALYS IS & T ROUB LESHOOT ING G U IDE

Flexent Modular Cell 4.0

Flexent Modular Cell 4.0B

07-0309 1xEV-DO Modcells on Release 26.03 or 26.04 with SBCBRs Mobility Products
Cannot Process Calls on Beta Sector after EVM or URC Restore
Flexent CDMA Base Station

AutoplexSII CDMA Base


Station

07-0305 1xEV-DO Carrier Change Mobility Products

Flexent CDMA 1xEV-DO

07-0290 1xEV-DO Service Measurements not reported from some Traffic Mobility Products
Processors (TP) after a 2GB RAM AP upgrade.
Flexent CDMA 1xEV-DO

07-0262 Rev. A Mobiles should not request EAC/ECC until officially Mobility Products
supported
Flexent CDMA Base Station

AutoplexSII CDMA Base


Station

07-0133 AR1-1549574: 1x-EV-DO Rev. A Low Throughput with multiple Mobility Products
users in sector
Flexent CDMA 1xEV-DO

07-0102 1xEVDO service nodes connected to a Juniper Router without the Mobility Products
large buffer size feature may experience low throughput.
Flexent CDMA 1xEV-DO

06-0851b EVDO RNC Frame - UATI Allocation Failure on an AP Mobility Products

Flexent CDMA 1xEV-DO

ALL RIGHTS RESERVED ALCATEL-LUCENT 2007 122

Das könnte Ihnen auch gefallen