Beruflich Dokumente
Kultur Dokumente
Date:
Version 1.0
ABSTRACT
This document provides a detailed test plan for KPI Optimization of the LTE field
technology deployment as detailed in the Verizon High Level Test Plan. The tests will be
executed in Boston, MA and surrounding areas. The primary objective of the
Optimization is to validate basic KPI functionality, evaluate performance of LTE AirInterface functionalities. The scope of test cases included in this KPI Test Plan spans
several areas including access, latency, coverage and Capacity
Alcatel-Lucent Proprietary
Alcatel-Lucent - Proprietary
This document contains proprietary information of
Alcatel-Lucent and is not to be disclosed or used
except in accordance with applicable agreements.
Copyright 2006 Alcatel-Lucent Technologies
Unpublished and Not for Publication
All rights reserved
Revision History
Version
Description
1.0
Initial version of test plan that will be executed for
KPI Optimization
Date
04/22/10
References
[1] LTE KPI Optimization High Level Test Plan, Version 1, April 22 2010, Verizon
Wireless
Alcatel-Lucent Proprietary
See notice on first page
Glossary of Terms
DLLS
EAT
ePC
GbE
LLDM
LMT
MME
MPLS
MS
NPO
OAM
OLSM
RAMSES
RRH
SAE
TLS
VLAN
Alcatel-Lucent Proprietary
See notice on first page
Table of Contents
ABSTRACT......................................................................................................................................................1
REVISION HISTORY...................................................................................................................................2
REFERENCES...............................................................................................................................................2
1
INTRODUCTION.................................................................................................................................7
OPTIMIZATIOIN SYSTEM................................................................................................................8
3.1
DEPLOYMENT SYSTEM ARCHITECTURE......................................................................................8
3.2
AIR INTERFACE OVERVIEW............................................................................................................11
3.3
ACCESS TERMINALS.......................................................................................................................11
3.4
LTE ENODEB FUNCTIONS..............................................................................................................11
3.5
LTE MME FUNCTIONS..................................................................................................................11
3.6
LTE SAE FUNCTIONS....................................................................................................................12
3.7
DATA LAPTOP CONFIGURATION.....................................................................................................12
3.8
APPLICATION SERVERS..................................................................................................................12
3.9
RF OPTIMIZATION TOOLS.........................................................................................................12
3.9.1
eDAT Tool............................................................................................................................13
3.9.2
Enhanced Analysis Tool (EAT)............................................................................................13
3.9.3
Agilent tool...........................................................................................................................14
3.9.4
WPS......................................................................................................................................14
3.9.5
WINDS.................................................................................................................................14
3.9.6
SYNCRO TEST.....................................................................................................................14
RF OPTIMIZATION EXECUTION.............................................................................................17
4.4.1
Cluster Optimization............................................................................................................13
4.4.2
System Verification...............................................................................................................13
TEST CASES.......................................................................................................................................20
5.1
PHYSICAL LAYER THROUGHPUT TESTS PEAK...............................................................................20
5.1.1 Single User Downlink Physical Layer Throughput Test Peak.................................................20
5.1.2 Single User Uplink Physical Layer Throughput Test Peak......................................................23
5.2
RLC THROUGHPUT TESTS PEAK.................................................................................................24
5.2.1 Downlink RLC Throughput Peak Test......................................................................................24
5.2.2 Uplink RLC Throughput Peak Test .........................................................................................25
5.3 PHYSICAL LAYER THROUGHPUT TESTS MEDIAN..................................................................24
5.3.1 Downlink Physical Layer Throughput Median Test.................................................................24
5.3.2 Uplink Physical Layer Throughput Median Test ....................................................................25
Alcatel-Lucent Proprietary
See notice on first page
5.4
5.5
5.6
5.7
5.8
5.9
5.10
5.11
5.12
5.13
5.14
5.15
5.16
5.17
5.18
5.19
5.20
5.21
Alcatel-Lucent Proprietary
See notice on first page
List of Tables
Table 1: Revision History...............................................................................................................2
Table 2: DEPPLOYMENT Cell Locations....................................Error! Bookmark not defined.
Table 3: SNR of Different Cell Locations......................................Error! Bookmark not defined.
List of Figures
Figure 1: LTE DEPLOYMENT Network Architecture...................................................................9
Figure 2: LTE DEPLOYMENT Network Transport ConfigurationError! Bookmark not defined.
Figure 3: LTE DEPLOYMENT Network Transport ConfigurationError! Bookmark not defined.
Figure 4: EAT configuration in LTE DEPLOYMENT..................................................................14
Alcatel-Lucent Proprietary
See notice on first page
Introduction
Category
Sub-Category
Scope
Target Value
C
L
L
C
L
N
N
N
0.70%
2.50%
2 seconds
2.50%
0.5 second
1.50%
0.5 seconds
1.20%
1.20%
1.20%
N
C
L
0.5 seconds
L
60/20 Mbps
DL/UL RLC Throughput, peak
55/18 Mbps
DL/UL Physical Layer Throughput,
median
DL/UL Physical Layer Throughput,
5th %-ile
Alcatel-Lucent Proprietary
See notice on first page
7/3 Mbps
R
1/0.5 Mbps
1%
30 msec
7
C
1.20%
RF-SINR
RF-RSRP
L
100 msec
L
R
R
R
R
100 msec
2%
95%
10%
R
90%
R
90%
DEPLOYMENT SYSTEM
Multiple eNodeBs
An ePC encompassing two functions: MME and SAE Gateway
Metro Ethernet Backhaul
Applications servers
As shown in the figure, the cluster of eNodeBs will be connected via the 7705 and 7750
routers to the other network elements such as the ePC and application servers. Each
eNodeB will consist of a D2U and three TRDUs. Each D2U will consist of a uCCMs
(controller with interface to backhaul) and three eCEMs (modems; one for each sector).
Each eCEM will be connected via fiber optic cable to the TRDU.
Alcatel-Lucent Proprietary
See notice on first page
Alcatel-Lucent Proprietary
See notice on first page
The information below details the hardware description and the associated OAM
equipment list for the Trial network:
eNodeB
o D2U V2 (1 uCCM + 3 eCEM)
o TRDU (remote-radio-heads comprising of amplifiers and filters), 40W Tx
power
Backhaul
o 7750 SR (service router)
o 7705 SAR (service aggregation router)
Transport services
o Cisco TLS EVC
ePC
o PDN/MME/SAEGW - IPD ATCA
Security
o Cisco PIX
LAN switching
o Module from 7750
Remote support
o RAMSES
In order to manage the eNodeBs, a complete OAM system has to be designed to host the
following functions:
Configuration management
Fault management
Performance management
Traces
Alcatel-Lucent Proprietary
See notice on first page
10
Link aggregation
DSCP mapping
VLAN and LAN switching
IP router to reach different Application servers
11
o S1 connection establishment
o Idle to active mode transition
o Active to idle mode transition
Session management
o QoS control
S1 handling during HO
UDP
DOS FTP - TCP/IP
Ping
12
DOS-FTP, WINDS, Ping To generate TCP/IP and UDP based traffic data for
measuring data capacity and network latency
Ping scripts for access tests
Logging tools
Alcatel-Lucent Proprietary
See notice on first page
13
14
Alcatel-Lucent Proprietary
See notice on first page
15
Once the data has been collected, eDAT post-processes the UE and eNodeB log files to
generate the analyses of the KPIs. eDAT is a standalone tool that does not need to be
connected to the fixed infrastructure. The output includes maps, graphs, plots, reports
and message decoding. All these can be used to evaluate the RF performance of the LTE
Trial network. eDAT makes use of event timestamps and locations of the UEs to
geographically plot out the data.
Alcatel-Lucent Proprietary
See notice on first page
16
4 Process Overview
4.1 Deployment Site Locations
maps
17
Drive routes and Key Performance Indicators will be the same as the ones used later for
System Verification. It is important to keep the drive routes and KPIs identical for
performance validation and comparison purposes. Drive routes and KPIs must be agreed
upon with the customer.
4.3 RF Optimization Planning
The Optimization planning phase ensures system and tool readiness for RF Optimization
before beginning the actual drive testing.
4.3.1 Perform RF Parameter Audit
RF parameters must be inspected for consistency with the LTE parameter catalogue.
The RF parameter settings used in the network can be obtained from the NDP project
database. These settings are then audited using the LTE parameter catalogue WPS
4.3.2 Validate Initial Neighbor Lists
An important step within the RF Optimization preparation phase is associated with the
neighbor list verification. The complete neighbor lists in the LTE network are required to
compare the neighbor relations with network design plots. Neighbor relations need to be
verified for recent updates, validity and appropriateness. The recommended strategy is to
have a minimum number of neighbor relations in the neighbor lists. The neighbor lists
used in the network can also be obtained from the WPS project database.
4.3.3 Tool Readiness
Appropriate drive test tools and post-processing tools, need to be prepared for
optimization.
4.3.4 Define Clusters
Approximately 15-19 eNodeBs should be combined into one cluster. The actual number
used is based on the network expansion as well on topographical environment. The
clusters are selected to provide a center eNodeB with tow rings of surrounding eNodeBs .
4.3.5 Drive Route Planning
Drive routes need to be defined for Sector Verification, Cluster Optimization and System
Verification. Coverage prediction plots, morphology and clusters can define all drive test
routes.
The drive route should maintain a distance equal to of the cell site radius for sector
verification.
The routes for Cluster Optimization shall consist of major roads, highways and hotspots.
Total time to drive all routes in atypical cluster should be approximately 6 to 8 hours.
Additional border route is chosen by the way it crosses the cluster borders witout going
into the cluster areas.
The System Verification drive route are used to collect the metrics for the Exit Criteria.
The routes are a combination of individual clusters.
Alcatel-Lucent Proprietary
See notice on first page
18
Alcatel-Lucent Proprietary
See notice on first page
19
20
Procedure:
Action
Response
4
5
6
7
8
Alcatel-Lucent Proprietary
See notice on first page
21
Key Metrics:
1. Physical Layer Downlink Airlink Throughput Peak
2. Application Layer Throughput
3. Initial Block Error Rates
4. Residual Block Error Rates
5. Scheduled Transport Format distribution
Expected Result:
Peak Tput should be 60Mbps
Alcatel-Lucent Proprietary
See notice on first page
22
Procedure:
Action
Response
4
5
6
7
8
Alcatel-Lucent Proprietary
See notice on first page
23
Key Metrics:
1. Physical Layer Uplink Airlink Throughput Peak
2. Application Layer Throughput
3. SIR target
4. Initial Block Error Rates
5. Residual Block Error Rates
6. Scheduled Transport Format distribution
Output:
Physical Layer Uplink Airlink Throughput Peak should be recorded.
Expected Result:
Peak Tput should be 20Mbps
4.6.3
Test Objectives: Test Validate the performance by conducting single-user stationary and
limited mobility tests on pre-selected locations in an embedded sector and in a limited
drive route within the same sector, respectively. The tests shall be performed using both
UDP applications for performance comparison, and under 50% cell loading conditions.
Test Description:
Tests will be executed in Close-Loop Spatial Multiplexing (CLSM).
For the stationary tests, the test UE will be located at selected locations in an embedded
sector corresponding to the appropriate SNR ranges for Near Cell (NC). See for SNR
ranges. For the mobility tests, the test UE will be driven according to the predefined drive
route.
DLLS will be used to load the DL of the cells neighboring the target cell. Loading will be
generated by occupying portions of the Resource Blocks (RB). For example, to generate a
cell loading of 50%, 50% of the total DL RBs will be occupied.
Agilent Tool, Backhaull Bandwidth: 100 Mbps, SINR: 17 to 21 dB, Loaded Conditions:
50%
.
Test Routes:
Near Cell, SNR17dB 21 dB
Alcatel-Lucent Proprietary
See notice on first page
24
Terminal Speeds:
Stationary, Limited Mobility (25-30) km/hr
Test Set Up:
Procedure:
Action
Response
4
5
Key Metrics:
1. Physical Layer Downlink RLC Throughput Peak
2. Application Layer Throughput
3. Initial Block Error Rates
4. Residual Block Error Rates
5. Scheduled Transport Format distribution
Output:
Peak Downlink RLC layer throughput should be recorded.
Expected Result:
Peak Tput should be 55Mbps
Alcatel-Lucent Proprietary
See notice on first page
25
Procedure:
Action
Response
4
5
Alcatel-Lucent Proprietary
See notice on first page
26
Key Metrics:
1. Physical Layer Uplink RLC Throughput Peak
2. Application Layer Throughput
3. SIR target
4. Initial Block Error Rates
5. Residual Block Error Rates
6. Scheduled Transport Format distribution
Output:
Peak Uplink RLC layer throughput should be recorded
Expected Result:
Peak Tput should be 18Mbps
Alcatel-Lucent Proprietary
See notice on first page
27
Alcatel-Lucent Proprietary
See notice on first page
28
4.6.5
Test Objectives: Evaluate the Average Downlink Physical Layer Throughput in a cluster
of 15 20 eNodeBs
Test Description: Tests will be executed in Close-Loop Spatial Multiplexing (CLSM).
For the cluster test, the test UE will be driven according to the predefined drive route.
The route definition would cover SNR distribution that reflect NC, MC, CE.
DLLS will be used to load the DL of the cells neighboring the target cell. Loading will be
generated by occupying portions of the Resource Blocks (RB). For example, to generate a
cell loading of 50%, 50% of the total DL RBs will be occupied.
Agilent Tool, Backhaul Bandwidth: 50 Mbps Backhual
Test Setup:
A drive test van will be used with rooftop mounted antennas
1. Ensure that 500MB file
Test Routes:
Near Cell, SNR17dB 21 dB
Terminal Speeds:
Stationary, limited Mobility (25-30) km/hr
Test Set Up:
Procedure:
Action
Response
Alcatel-Lucent Proprietary
See notice on first page
29
IP address verified
Key Metrics:
1. Physical Layer Uplink RLC Throughput Peak
7. Application Layer Throughput
8. SIR target
9. Initial Block Error Rates
10. Residual Block Error Rates
11. Scheduled Transport Format distribution
Output:
Peak Uplink RLC layer throughput should be recorded
Expected Result:
Peak Tput should be 18Mbps
Alcatel-Lucent Proprietary
See notice on first page
30
Alcatel-Lucent Proprietary
See notice on first page
31
4.7.2
Test Objectives: Evaluate the sector throughput for multiple UEs at stationary locations
in an embedded sector and for limited mobility drive tests within the same sector. Tests
will be conducted under unloaded and loaded conditions, and for UDP and FTP
applications.
Test Description: For the stationary tests the test UEs will be located at selected
locations corresponding to the appropriate SNR ranges for Near Cell (NC), Mid Cell
(MC), and Cell Edge (CE). See Error: Reference source not found for SNR ranges. For
the mobility tests the test UEs will be driven according to a predefined limited mobility
(single sector) drive route
Loading is generated by placing loading UEs in neighboring cells at pre-selected
locations. Loading of 100% will result in an IoT of TBD dB in the target cell, while a
loading of 50% will result in an IoT of TBD dB in the target cell.
All tests will be conducted with the default scheduler setting.
Test Setup:
1. One or more drive test vans will be used with rooftop mounted antennas
2. Each stationary location consists of a unique set of (NC,MC,CE) locations
Key Metrics:
1. Physical Layer Throughput
2. Application Layer Throughput (UDP/FTP)
3. Initial Block Error Rates
4. Residual Block Error Rates
5. Scheduled Transport Format distribution
Alcatel-Lucent Proprietary
See notice on first page
32
4.7.3
Test Objectives: Evaluate the sector throughput with MU-MIMO at stationary locations
and for limited mobility drive tests. Tests will be conducted under unloaded and loaded
conditions, and for UDP and FTP applications.
Test Description: For the stationary tests the test UEs will be located at selected
locations corresponding to the appropriate SNR ranges for Near Cell (NC), Mid Cell
(MC), and Cell Edge (CE). See Error: Reference source not found for SNR ranges. For
the mobility tests the test UEs will be driven according to a predefined limited mobility
(single sector) drive route.
Loading is generated by placing loading UEs in neighboring cells at pre-selected
locations. Loading of 100% will result in an IoT of TBD dB in the target cell, while a
loading of 50% will result in an IoT of TBD dB in the target cell.
All tests will be conducted using the default scheduler setting.
MU-MIMO implementation allows for up to 4 UEs paired (2 pairs).
Test Setup:
1. One or more drive test vans will be used with rooftop mounted antennas
2. Each stationary location consists of a unique set of (NC,MC,CE) locations
Key Metrics:
1. Physical Layer Throughput
2. Application Layer Throughput (UDP/FTP)
3. Initial Block Error Rates
4. Residual Block Error Rates
5. Scheduled Transport Format distribution
Alcatel-Lucent Proprietary
See notice on first page
33
4.8
Downlink Scheduler
Test Objectives: Evaluate the Scheduler performance for multiple UEs at stationary locations
and for limited mobility drive tests within the same sector. Tests will be conducted under loaded
conditions with UDP and FTP applications. Three Scheduler settings will be tested: proportionalfair (PF), conservative (CO), and aggressive (AG).
Test Description:
Settings common to all tests:
- Open-Loop Spatial multiplexing (OLSM) mode
- 100% loading on DL of neighbor cells
For the stationary tests, 8 test UEs will be placed at selected locations corresponding to the
appropriate SNR ranges for Near Cell (NC), Mid Cell (MC), and Cell Edge (CE) locations. The 8
UEs will be placed in four different vans with 2 UEs in each van. The number of active UEs will
be increased incrementally to illustrate the scheduling gain. For the stationary cases, each
combination of a subset of the 8 UEs will be depicted as a triplet (x, y, z) in the tables below to
represent the number of active UEs at each of the three locations. For the cases with a mix of
stationary and mobility UEs, each combination of a subset of the 8 UEs will be depicted as a
quartet (x, y, z, m) where m will denote the number of mobility UEs. See Error: Reference source
not found for SNR ranges.
For the mobility tests the test UEs will be driven according to a pre defined limited mobility
(single sector) drive route.
DLLS will be used to load the DL of the cells neighboring the target cell. Loading of 100% will
be used for the tests.
Test Setup:
1. One or more drive test vans will be used with rooftop mounted antennas
Key Metrics:
1. Physical Layer Throughput
2. Application Layer Throughput
3. Initial Block Error Rates
4. Residual Block Error Rates
5. Scheduled Transport Format distribution
Alcatel-Lucent Proprietary
See notice on first page
34
4.9
Uplink Scheduler
Test Objectives: Evaluate the uplink scheduler performance for multiple UEs at stationary
locations and for limited mobility drive tests within the same sector. Tests will be conducted
under loaded conditions with UDP and FTP applications. Three Scheduler settings will be tested:
proportional-fair (PF), conservative (CO), and aggressive (AG).
Test Description:
For the stationary tests, 8 test UEs will be placed at selected locations corresponding to the
appropriate SNR ranges for Near Cell (NC), Mid Cell (MC), and Cell Edge (CE) locations. The 8
UEs will be placed in four different vans with 2 UEs in each van. The number of active UEs will
be increased incrementally to illustrate the scheduling gain. For the stationary cases, each
combination of a subset of the 8 UEs will be depicted as a triplet (x, y, z) in the tables below to
represent the number of active UEs at each of the three locations. For the cases with a mix of
stationary and mobility UEs, each combination of a subset of the 8 UEs will be depicted as a
quartet (x, y, z, m) where m will denote the number of mobility UEs. See Error: Reference source
not found for SNR ranges.
For the mobility tests the test UEs will be driven according to a pre-defined limited mobility
(single sector) drive route.
All tests will be executed with 100% UL loading. The uplink loading will be generated by placing
loading UEs in the neighboring cells to generate an IoT corresponding to 100% loading in the
target sector.
Test Setup:
1. One or more drive test vans will be used with rooftop mounted antennas
Key Metrics:
1. Physical Layer Throughput
2. Application Layer Throughput
3. Initial Block Error Rates
4. Residual Block Error Rates
5. Scheduled Transport Format distribution
Alcatel-Lucent Proprietary
See notice on first page
35
Priority
5.5.1.1
5.5.1.2
5.5.1.3
5.5.1.4
H
H
H
H
Call Setup
UE Initiated
UE Initiated
UE Terminated
UE Terminated
Procedure:
BE C-Plane Latency
1. Set the log mask for the DM tool to include the debug messages.
2. Initiate a call from the test UE with the UE in BE mode
3. Initiate a call from the network side to the UE with UE in BE mode
4. Repeat 100 times each
QoS C-Plane Latency
5. Initiate a call from the test UE with UE in GBR QoS mode
6. Initiate a call from the network side to the UE with UE in GBR QoS mode
7. Repeat 100 times each
Key Metrics (per UE):
1. Call Setup Time
Alcatel-Lucent Proprietary
See notice on first page
36
Number
of UEs
1
1
1
1
Alcatel-Lucent Proprietary
See notice on first page
37
Procedure:
1.
2.
3.
4.
5.
38
3.
4.
5.
6.
7.
8.
Latency
Mean Opinion Score
Physical Layer Throughput
Application Layer Throughput {UDP}
Initial Block Error Rates
Residual Block Error Rates
Alcatel-Lucent Proprietary
See notice on first page
39
Procedure:
UL Tests
1. Set SINR target in neighboring cells to control the power of loading UEs
2. Place loading UEs in neighboring cells at pre-selected locations to generate desired IoT in the
target cell
3. Make a UDP Best Effort (BE) call to the target cell on the test UE and measure both UL and DL
physical-layer data rates, SINR, and record the signaling messages
4. Place the test UE in a van and drive on a pre-selected route from NC to CE of the target cell until
the call drops
5. Repeat steps 1 to 4 for each interference condition
DL Tests
1. Set DLLS in neighboring cells to generate desired DL interference levels
2. Make a UDP BE call on the test UE from the target cell and measure both UL and DL physicallayer data rates, SINR, and record the signaling messages
3. Place the test UE in a van and drive on a pre-selected route from NC to CE of the target cell until
the call drops
4. Repeat steps 1 to 3 for each interference condition
Key Metrics:
1. Physical Layer Throughput
2. SINR
3. PDCCH error rate
Alcatel-Lucent Proprietary
See notice on first page
40
4.14 Handover
Test Objectives: Evaluate handover performance in following scenarios:
- Intra-Site (different sectors within one eNodeB)
- Inter-Site (different eNodeBs)
- Loaded and Unloaded Destination eNodeBs
Test Description:
Test UE will be driven along two routes:
Handover Route comprising of intra and inter eNodeB handovers between 3-4
sectors. On this route, additional UEs will be stationed in each sector along the
drive route. These UEs will load both downlink and uplink of their respective
sectors with BE traffic.
Cluster Route comprising of intra and inter eNodeB handovers in the entire 10
eNodeB Cluster. Only DL loading will be generated via DLLS on this route.
Non-guaranteed and guaranteed Quality of Service (QoS) tests will be conducted via Best
Effort (BE) and Guaranteed Bit Rate (GBR) QoS classes, respectively.
Application performance will be measured quantitatively and subjectively. While
quantitative measurements are throughput (physical-layer data rate) and latency,
subjective performance will be based on users perception of the application. For
example, quality of a Voice over IP (VoIP) call can be either clear, choppy but
audible, or not audible.
Both UL and DL SINR (Signal to Interference plus Noise Ratio) will be measured and
Signaling messages will be recorded in the tests.
Key Metrics:
1. Physical Layer Throughput
2. SINR
3. Latency
Alcatel-Lucent Proprietary
See notice on first page
41
Procedure:
1. Set the eNB and UE to V-pol configuration
2. Run DL/UL UDP/FTP tests in each of NC, MC and EC locations
3. Repeat for X-pol configuration setting
Key Metrics:
1. Channel Correlation Statistics at UE for DL tests
2. Physical Layer Throughput
3. Application Layer Throughput
4. Initial Block Error Rates
5. Residual Block Error Rates
6. Scheduled Transport Format distribution
Alcatel-Lucent Proprietary
See notice on first page
42
Appendix A:
Performance Metrics
The following metrics will be collected during the trial execution phase. The list shall
include (but not necessarily limited to):
Air Interface
o UE Tx power
o RSSI
o SINR
o BLER
o Retransmission statistics (HARQ and RLC)
o Transport Format
o Number of resource blocks (DL/UL)
o Channel rank statistics
o MIMO mode (Tx diversity or Spatial Multiplexing)
o Serving sector
o Location (GPS)
o UE Velocity
Throughput
o Individual user throughput and aggregated sector throughput
o UDP individual user throughput and aggregated sector throughput
o TCP individual user throughput and aggregated sector throughput
o User statistics (peak rates, average rates, standard deviations)
Latency
o U-plane latency
o Connection set up times
o Handover interruption time within the same site and across different sites
Alcatel-Lucent Proprietary
See notice on first page
43