GPRS KPI MEASUREMENT TECHNIQUES FOR THE RAILWAY

ENVIRONMENT – LESSONS LEARNED

Dirk Michel1 and Vaidyanathan Ramasarma2
1
Wireless Network Engineering, Nortel Networks, Maidenhead Office Park, Westacott Way, Maidenhead,
Berkshire, UK SL6 3QH Email: dmichel@nortel.com
2
Network Planning and Technology, Bechtel Telecommunications, 5275 Westview Dr., Frederick, MD,
21703, USA Email: vramasar@bechtel.com

Abstract - The railway environment presents a unique challenge for
cellular radio coverage. European cellular operators, among others,
have realized the potential of the mobile user on railways and have
constantly sought to improve the performance of cellular networks
to accommodate current and next generation wireless services.
Improving cellular coverage and services on railways is becoming
increasingly important, considering the serious efforts of various
European nations to boost public rail usage. However, performance
limitations that certain environments such as railways might place
on current technologies such as 2.5G general packet radio service
(GPRS) have yet to be accurately verified. This paper highlights
operational challenges and introduces strategies for measuring and
verifying GPRS key performance indicators (KPIs) on railways,
describes manual and automated field-testing concepts, and
presents selected KPI results from a trial campaign conducted in
the UK.
I. INTRODUCTION
Key performance indicators (KPIs) are becoming
increasingly important in the context of network rollouts, as
well as within optimization cycles of mature networks. KPIs
are typically used to quantify network performance
parameters and can be defined at three levels: individual cells,
cell clusters, and the entire network. Defining KPI targets
and methodologies for verifying them is critical for a cellular
operator, specifically for target coverage areas in difficult
terrain, including railway environments, where the train and
hence the mobile stations (MSs) may travel at very high
speeds. This paper presents a methodology for measuring
and verifying general packet radio service (GPRS) KPIs for
railways, with emphasis on network delay and GPRS
throughput performance. Other KPIs such as GPRS attach,
routing area update, PDP context activation, cell reselection
and packet loss are outside the scope of this paper. GPRS
throughput characteristics and specific KPI verification
issues in the railway environment are examined and possible
solutions are provided. The first section provides a brief
overview of the GPRS network architecture. Then, possible
solutions for verifying GPRS KPIs for railway coverage are
analyzed and evaluated. Finally, sample GPRS KPI
performance metrics and the impact of mobility on
throughput are presented. The measurements presented in
this paper are part of a trial measurement campaign
conducted in the UK.
II. GPRS NETWORK ARCHITECTURE
The GPRS network architecture is an extension to the
global system for mobile communication (GSM) network,
which has been introduced to enable packet switched data
services via the public land mobile network (PLMN).
Considering the overall GPRS architecture, two main
building blocks are defined: The core network and the base
station subsystem (BSS). Both blocks are important to assess
the performance of end-to-end packet data services. Figure 1
shows a high-level overview of the logical GPRS network
architecture. The BSS governs the GSM enhanced data rates
for global evolution (EDGE) radio access network (GERAN)
and provides the wireless interface to the MS. GPRS uses the
physical air interface defined for GSM, but GPRS specific
frame formats and logical channels have been standardized.
Several channel coding schemes (CS1 to CS4) incorporating
different levels of error detection and error correction
schemes are supported, which help mitigate the undesirable
effects of a fluctuating radio environment. Commercial
GPRS networks commonly support CS1 and CS2. CS3 and
CS4 including link adaptation however are supported by
several equipment vendors, but involve hardware upgrades of
the network and the MS. Reliable packet delivery over the air
interface is ensured by a variety of transmission-plane
protocols, including multiple access control (MAC), radio
link control (RLC), logical link control (LLC), and sub-
network dependant convergence protocol (SNDCP).

Fig. 1. Logical GPRS architecture.

16 Texas Wireless Symposium 2005

 MAC enables multiple users to share the same physical
resources, RLC provides a reliable link over the physical
radio frequency (RF) channel with a selective automated
repeat request (ARQ), LLC provides reliable data transfer for
user data between the MS and the serving GPRS support
node (SGSN) along with the base station subsystem GPRS
protocol (BSSGP), which provides the necessary quality of
service (QoS) and routing information for the data transfer,
and SNDCP is a convergence protocol mapping OSI Layer 3
protocols into LLC frames [1,2,3,5]. Additionally, the PLMN
operator may choose to use acknowledged or un-
acknowledged packet transfer for the RLC and LLC.
GPRS mobility management is similar to GSM. Individual
or several cells can be grouped in routing areas and every
routing area is served by one SGSN. In accordance with the
mobility management state model, the MS may be in idle,
ready, and standby state [5]. A MS can move to ready state
by attaching to the GPRS network and by activating a packet
data protocol (PDP) context, which activates GPRS user
profiles, including IP addresses and QoS parameters. Upon
request, temporary radio resources may be assigned to the
MS. A temporary block flow (TBF) may then be established
to facilitate the transfer of LLC frames on one or several
PDCHs. However, the physical resources and the TBF are
only assigned for the duration of packet transfer (“capacity
on demand”). In the case were the packet transfer is
temporarily suspended, e.g. the user reads an Internet web
site, the allocated radio resources are released after a certain
period of time, while the PDP context remains established.
PDCHs and TBF can be re-assigned when new LLC become
available for transmission over the radio link [1]. The core
network (CN) of the packet switched domain is essentially a
cluster of routers and databases interconnected by different
interfaces and protocols. Typically, the network
interconnecting the CN devices is referred to as the packet
data network (PDN). The Gb interface connects the BSS and
the SGSN, allowing the exchange of user data and signaling
messages. The SGSN is the service access point of the GPRS
network for all MS and “relays” IP traffic originating from
the MS to the gateway GPRS support node (GGSN) and vice
versa. The GGSN provides interworking with external IP
networks, such as the Internet. Existing databases and
infrastructure of the GSM core network are however still
required, including home location register (HLR), visitor
location register (VLR), and the short message service center
(SMSC). For additional detail on GPRS core networks and
BSS, see [2, 4].
III. GPRS KPI TRIAL METHODOLOGY FOR RAILWAY
ENVIRONMENTS
Verifying GPRS performance on railways can be
particularly challenging. Usually, KPI verification
methodologies fall into two categories: (1) field trials and (2)
analysis of network counters. Field trials then distinguish
17
between automatic and manual testing methodologies as
shown in Fig. 2.
Automatic Testing
Operators can use automatic testing in urban areas by
installing test equipment in vehicles of the public transport
system, taxis etc. Automatic testing on railways however
should use the rail infrastructure to assess the performance
within the train carriages to account for the carriage specific
penetration loss. The approximate penetration loss for current
train carriages in operation can vary between 5 dB and 25 dB,
including approximately 8dB standard deviation. In some
cases, increased losses may be introduced due to the use of
metalized carriage windows. Typically, the PLMN operator
may choose to employ a custom- built or a modified standard
passenger carriage for automatic testing on the railways,
which can be used to house the measurement equipment. The
carriage may then be attached to a train used for track
geometry measurements, usually employed by train/rail track
operators, or trains specifically customized for radio
measurement trials.
The location of the test equipment within the carriage,
especially the aerials, should be carefully chosen in order to
attain worst-case measurements. Typically, this requires the
aerials to be installed towards the aisle side of the carriage at
an approximate height of 1.5m. As continuous positional
data is generally required during post processing, a GPS
receiver should be used that provides a minimal satellite
acquisition time and continuous update. Long tunnels and
roofed railway stations can interrupt the GPS signal; hence a
minimal satellite acquisition time is highly desirable. A GPS
with dead reckoning system (DRS) capability, including
gyroscope and odometer can help to mitigate the issue, as the
sensors allow the calculation of the train’s position even
when GPS signals are blocked. For carriage mounts, the GPS
aerial is typically installed on the roof of the carriage,
enabling short satellite acquisition times. Such an equipment
setup within a carriage is expected to inherently produce
repeatable and consistent measurement results, as the
measurement equipment is installed in permanent positions.
Figure 3 illustrates an automatic test setup.
GPRS KPI Verification on Railways
- Session application
throughput
Network
- Session RLC
throughput
Field Trials
Counters
- LLC throughput
- RTT delay
- PDCH allocation
(% Success)
- Volume/Cell (MB)
Manual Automatic - PRACH requests
(% Success)
Testing Testing - TBF assignments
Data Post Processing
GPRS KPI Results
Fig. 2. Trial methodology.
 Aisle area of the
passanger carriage

Carriage
Carriage Window

GPS Aerial
GPS Receiver
USB Hub
Laptop Computer
Test Mobile Station

Desktop Holder

Data Cables

Manual download of
measurement
results

Fig. 3. Automatic test setup. Fig. 4. Manual equipment setup.

Manual Testing
The manual testing method however requires measures to
ensure repeatable and consistent results. Firstly, suitable test
equipment should be selected to support real-time
measurements of the GSM/GPRS air interface. Secondly, it
is considered beneficial to use test equipment from one
vendor only, which helps to maintain consistent
measurement results and post processing activities. Finally,
all equipment should be calibrated and tested before it’s
released to field personnel. Pre-configured equipment and a
fixed location of the test MS within the carriage will ensure a
stable measurement environment in most cases. Using a
desktop holder can help to maintain the MS in the same
upright location during the KPI measurement campaign.
Locating the MS at the aisle side of the table (approximately
1.5m height) allows for near-accurate simulation of an actual
traveling MS user. It also helps factoring effects of worst-
case scenarios of radio propagation into the measurements.
Positional data can be obtained by using a mobile GPS
receiver with minimal satellite acquisition time. The GPS
aerial can be attached to the carriage window by using
suction plugs. A possible equipment setup including
computer terminal, test MS and GPS receiver is shown in
Fig. 4.
Both methods have their own specific advantages and
disadvantages, which are listed in Table I. It should be noted
that the presented methods for field trials generally allow
introducing adjustments to the KPI results. Post processing
generally very limited, but can provide a rough indication of
the user experience of train passengers.
The difficulty of this approach is that the cells, even if
constructed for specific rail coverage employing high gain
directional antennas with horizontal beam-widths of less than
15°, may “pick up” other users outside of the train. Generally,
network performance counters provide cell or routing area
based counters, which are not user or session specific. GPRS
throughput on the application layer is typically not supported
either, as the GPRS network does not discriminate between
different applications, although such information is e.g.
provided within the destination port field of the transmission
control protocol (TCP) header. However, network statistics
such as PDCH allocation failures, PDCH drops, successful
TBF assignments, congestion time, as well as successful
“hand-outs” and “hand-ins” during cell reselection can be
useful in identifying low performing cells or cluster of cells.
The analysis of such network performance statistics can
allow addressing corrupted hardware, congestion, RF
planning weaknesses and software configuration issues
before the actual KPI measurements take place. Table I lists
advantages and disadvantages for manual versus automatic
testing.
TABLE I
MANUAL VERSUS AUTOMATIC TESTING: ADVANTAGES AND DISADVANTAGES
Advantages Disadvantages
-Allows a GPS system -Cost intensive
with DRS -Travel schedule constraints
Automatic

of the measurement data can be used e.g. to exclude and

Testing

-Intrinsically consistent -Equipment setup only in

discard measurements from certain areas of rail routes, which -Continuous customized coaches
do not form part of the targeted coverage area. Such measurement of data -Loss of Measurement Data
exclusion areas can be tunnels of a certain length or cuttings -Easier repeatability -Continuous performance reporting
of a certain depth, depending on the required scope. The
-Cost effective -Possible inconsistency
impact and the possible bias of excluded areas on the KPI
Manual
Testing

-Ad-hoc collection of -Possible loss of GPS fix

results can thus be removed. Using network counters to measurement results -Incomplete measurement log files
assess the GPRS performance of railway coverage is -Possible difficulty in repeatability

18
 IV. SAMPLE GPRS DATA KPI RESULTS
There are two basic end-to-end parameters that help
estimating the service performance of a GPRS network: data
throughput and roundtrip time (RTT). The results discussed
in this section are based on a trial area (~100 km) along
important rail routes in the UK. Network-wide results,
although important are beyond the scope of this paper. GPRS
throughput can be determined on various protocol layers,
including RLC, LLC, TCP or the application layer, whereas
network latency is typically measured with “pings”, which
are Internet control management protocol (ICMP) echo
requests implemented on raw IP. Measuring the application
throughput however best resembles the user experience, as
the actual performance figures of the end-to-end connection
is captured. Naturally, application throughput values will be
less than those measured on lower protocol layers, due to
encapsulation overheads and signaling traffic [4]. There is a
tradeoff between mobility (fast-moving MSs) and link Fig. 6. GPRS latency characteristics.
performance [5]. Phase errors (due to rapid variation in phase
Such cell configurations can be beneficial when providing
of the received signal and Doppler effects) force the receiver
coverage for linear transport networks, such as motorways
to incorrectly detect some bits independent of the received
and railways.
signal-to-noise-ratio (SNR). To assess the GPRS throughput
The time period the MS resides within a cell however
on the railway, a 100kB file can be continuously downloaded
would naturally decrease with sectorized cells, as the covered
from a file transport protocol (FTP) server that is directly
area per cell is decreased. Typically, number of possible
connected to the GPRS core network via a virtual private
GPRS cell reselections on a given rail route impacts the
network (VPN). Using a small file size can be useful to
throughput performance and increases the overall network
determine network accessibility, especially within a train
latency. Latency is a critical parameter to characterize the
moving at high speeds (e.g. 150 km/hr). In case the standard
performance of real-time and interactive applications such as
software of the test MS does not support FTP, alternative
streaming video and network gaming. In addition to the
software can be used to enable repeated downloads and log
network quality requirements of different applications, the
file generation. Figure 5 shows the test network used during
RTT has a significant impact on the performance of higher
the trials. Assuming that the railway coverage is provided by
layer protocols above the subnet connection boundary [5].
base stations configured as standard “quasi-omni” cells, it
High latency and packet loss can severely degrade the
can be estimated that a MS would spend approximately 40-
achievable application throughput. GPRS network latency is
60 seconds within a given cell. “Quasi-omni” cells are
typically in the area of 700 ms to 1000 ms with a standard
typically equipped with RF signal splitters and bi-directional
deviation of approximately 150 ms [4].
narrow- beam antennas pointing up and down the target
Figure 6 shows the typical characteristics of RTT in a
coverage area.
commercial GPRS network measured over successive
sessions. Traveling MSs that transcend cell boundaries
experience higher delays including higher variability of
delays, caused by “packet transfer idle mode” times during
cell reselection procedures [2]. The physical radio resources
including TBF are released by the “old” cell and re-assigned
by the “new” reselected cell. GPRS throughput performance
is impacted in a similar fashion. Packet transfer can be
interrupted by approximately 0.25s-4s depending on how
quickly the MS acquires the full set of system information of
the “new” cell. Also, the probability of packet loss may be
increased due to LLC frame re-routing between the
reselecting cells. The protocol stack at the MS shows that
flow control and congestion control mechanisms are
implemented within RLC and TCP [5]. Packet
Fig. 5. Test network for FTP. retransmission combined with high network latency may
severely degrade throughput performance.

19

Fig. 7. GPRS throughput characteristics.
Figure 7 provides session application throughput results
obtained from 950 individual 100kB file downloads on a rail
route with traveling speeds of up to 200 km/hr. The test
mobile station used supports 4 PDCH channels in the
downlink and 1 PDCH channel in the uplink. It is apparent
that the mean application throughput is approximately 25
Kbps, as opposed to the theoretical maximum of nearly 40
Kbps. Various theories can explain the reason for such
performance, most notably, the criticality of an induced-
delay in the core-network, details of which will be presented
in an alternate publication. From Fig. 7, the red-curve
represents the running sum average (RSA) calculated over
the measured throughput values. Despite the high traveling
speeds of the train the RSA stabilizes approximately after
200 sessions. Figure 8 shows the error analysis for the RSA.
The difference of two consecutive RSA values has been
derived and plotted in order to identify the point at which the
measurements stabilize within a range of e.g. 0.1 Kbps. This
would be achieved after approximately 200 sessions. The
error analysis can be computed on an ongoing basis during a
TS « S File »
measurement campaign, which can help to understand and
(1)
estimate the variability of the session application throughput.
Additionally, the error analysis can help determining the
point beyond which further measurements have negligible
impact on the mean session application throughput of a given
coverage area.
V. MOBILITY EFFECT AND TARGET LEVELS FOR THROUGHPUT
Considering the level of mobility is imperative in the context
of defining achievable targets for mean session application
throughput, as the direct comparison of throughput levels
derived from stationary and fast moving MS may lead to
unrealistic performance expectations. This emphasizes the
difficulty of employing network wide KPIs for GPRS
throughput (Tkpi). Such network wide KPIs however may still
be used if the performance impacting elements of a given
special coverage area can be quantified reliably.
20
Fig. 8. Throughput error analysis.
The number of available downlink PDCHs, interference (C/I),
multiplexing and the mobility effect are quantifiable
parameters [5,6]. Assuming that the railway environment and
the overall network exhibit comparable levels of mean
PDCH availability, average interference levels and
multiplexing, then the discriminating factor would be the
mobility effect. Isolating this effect and making the
appropriate adjustments to the throughput measurements
would make the adjusted values comparable to stationary
measurements. (1) provides a possibility of isolating the
mobility effect and other performance impacting conditions,
where M represents the mean level of PDCH multiplexing,
TS is the mean number of available downlink time slots, S is
the file size, ttotal denotes the total file download time, tci and
tri are packet outage time during cell reselection and routing
area / location area update respectively and n represents the
number of cell reselections.
ª º
« »
M
Tkpi
4 « § n ·»
« t total  ¨ ¦ tc i  tri ¸ »
©i 0 ¹¼
¬
It should be noted that packet idle time is equivalent to radio
outage, i.e. the time period in which the MS has no TBF
assigned. The effective packet outage time on the application
layer however is greater than the radio outage, as it begins
with the last TCP packet sent by the “old” cell and ends with
the first TCP packet sent by the new cell. The effective
packet outage time has been measured and analyzed with e.g.
windump [7] and ethereal [8] respectively.
It may be expected that the post-processed performance
(applying (1)) on the train would approach or be very close
to the performance level of stationary measurements.
Following figure presents a comparison of raw
measurements, their post processed values and stationary
measurements of the mean session application throughput.
Fig. 9. Throughput Characteristics
Figure 9 shows that the average value of the raw
measurements on the train are 7.7 kbps lower compared to
the stationary results. It can also be seen that the average post
processed results are 3.2 kbps lower that the stationary
measurements. This outcome suggests that (1) accounts for
approximately 60% of the mobility effect. Further analysis of
the log files has shown that the average TCH availability and
the average level of multiplexing are homogeneous across
the test area. This however did not apply to the average level
of C/I, as regional differences have been identified. The
remaining difference between the post processed results on
the train and the stationary measurements may therefore be
attributable to the varying interference levels within the test
area. C/I levels are generally highly dependent on frequency
planning and traffic density and are likely to change on a
regular basis, especially in a mature GSM/GPRS networks,
considering the ongoing introduction of new sites or re-
location or decommissioning of existing sites.
Defining railway specific KPIs that account for packet
outage time due to cell reselections and location / routing
area updates can be a possibility to determine achievable KPI
levels, although inaccuracies may be encountered due to the
variation of interference levels.
VI. CONCLUSIONS
GPRS performance verification trials on railways can be
performed using automatic and manual testing methods.
Automatic testing involves a customized railcar and is
generally expensive. Manual testing is comparatively cost
effective but requires a field team to generate the
performance measurements and a measurement methodology
that produces consistent and repeatable results. GPRS
throughput performance is affected by cell reselection and
routing area update procedures, especially for fast-moving
MSs. Increasing mobility levels, which lead to a higher
21
number of cell reselections and routing area updates, increase
the amount of packet outage time. It is therefore to be
expected that GPRS throughput performance decreases with
increased mobility. Removing the mobility effect from trial
measurements on the railways improves their comparability
to stationary measurements that are typically used for
establishing KPI levels. The linear nature of coverage along
railway tracks and the fact that trains travel through the same
space inherently provide, in time, quickly stabilizing GPRS
measurements.
REFERENCES
[1] 3GPP, TS 03.64 V8.11.0 (2003-04), Technical Specification
Group GSM/EDGE Radio Access Network; General Packet
Radio Service (GPRS); Overall description of the GPRS radio
interface; Stage 2, (Release 1999)
[2] 3GPP, Technical Specification Group GSM/EDGE Radio
Access Network; General Packet Radio Service (GPRS);
Mobile Station (MS) - Base Station System (BSS) interface;
Radio Link Control/ Medium Access Control (RLC/MAC)
protocol (Release 1999)
[3] 3GPP, TS 04.64 V8.7.0 (2001-12), Technical Specification
Group Core Network; Digital cellular telecommunications
system (Phase 2+); General Packet Radio Service (GPRS);
Mobile Station - Serving GPRS Support Node (MS-SGSN)
Logical Link Control (LLC) layer specification (Release 1999)
[4] R. Chakravorty and I. Pratt, “Performance issues with GPRS,”
Journal of Communications and Networks (JCN), Vol. 4, No. 2,
December 2002, p.266-281.
[5] Timo Halonen, Javier Romero and Juan Melero, GSM, GPRS
and EDGE Performance – Evolution towards 3G/UMTS,
Wiley and Sons, 2nd ed., 2003.
[6] M. Meyer, “TCP performance over GPRS,” Proc. 1999 IEEE
Wireless Communications and Networking Conference
(WCNC), Sept, 21-24, New Orleans, LA, pp. 1248-1252.
[7] Windump, http://winpcap.polito.it
[8] Ethereal, www.ethereal.com