QoS Measurements Methods and Tools

Contact:
Jarmo prokkola
Jarmo.prokkola@vtt.fi
Tel: +358 20 722 2346
VTT Technical Reseach Centre of Finland

Easy Wireless Workshop,

IST Summit, Budapest, 05.07.2007
VTT TECHNICAL RESEARCH CENTRE OF FINLAND

Network performance vs. QoS

•Network traffic measurements are employed to study the pure network

performance
•Active measurements are performed by injecting traffic with known
properties into the network
•Passive measurements consists of monitoring the existing traffic
flow(s) at one or more points
•QoS measurements lie logically above network traffic measurements,
and relate to the performance of networking applications.
•Objective QoS relates to something concrete and quantitative
•Packet loss, Delay, Jitter, Connection break length…
•Subjective QoS corresponds to the service quality from the user
perspective.
•Mean Opinion Score (MOS) tests are often used
•Subjective QoS can be estimated within certain limits from the
basis of objective QoS (e.g., PESQ algorithm)

Easy Wireless Workshop, IST Summit, Budapest, 05.07.2007, J. Prokkola 2

VTT TECHNICAL RESEARCH CENTRE OF FINLAND

The purpose of QoS measurements & monitoring

•Answers to the question: How do the applications see the network?

•Real-time QoS monitoring benefits

•Network administrators
•Network equipment manufacturers (testing)
•Networking application developers
•Operators
•End users
•Networking performance
•QoS aware applications (e.g., adapting video traffic flow)
•Handover triggering
•Congestion control management
•Accurate QoS measurements can be used by
•Network equipment manufacturers (testing)
•Networking application developers
•Researchers

Easy Wireless Workshop, IST Summit, Budapest, 05.07.2007, J. Prokkola 3

 VTT TECHNICAL RESEARCH CENTRE OF FINLAND

Common QoS Measurement Architecture

•QoS measurement architecture includes:

•(a) Measurement points
•Located at network nodes
QoS (e.g., hosts, routers, firewalls)
(real-time monitor
or database) •(b) Traffic measurement tool
d
•Captures packets and collects
QoS analysis information of the desired traffic flow
tool
c •(c) QoS analysis tool
•Analyzes the collected data and
Traffic
measurement tool
Traffic
measurement tool
calculates the actual QoS statistics
b
•(d) QoS monitor or database
Measurement point Measurement point
a (network node) (network node) … •Analysis can be done in real-time,
and/or after the traffic traces have been
Figure 1. Functional decomposition of QoS collected.
measurement architecture. •Analysis can be centralized to a
specific server, or distributed to the
network.

Easy Wireless Workshop, IST Summit, Budapest, 05.07.2007, J. Prokkola 4

 VTT TECHNICAL RESEARCH CENTRE OF FINLAND

Single-point measurements

•Measurement tool is operating in a

single network node
•Round trip (RT) performance
Round trip performance
•RT delay, RT jitter, RT packet loss
•Service response time
•E.g., HTTP, or commonly TCP
performance
•Single-point traffic statistics
•Offered traffic load, throughput,
number of packets, packet sizes…
Figure 2. A single-point measurement
architecture.
•One-way performance can not be
measured

Easy Wireless Workshop, IST Summit, Budapest, 05.07.2007, J. Prokkola 5

 VTT TECHNICAL RESEARCH CENTRE OF FINLAND

Two-point measurements

•If the measurement tool is connected to

both communicating nodes, this is called
end-to-end performance end-to-end measurement.
Uplink (UL) performance •One-way performance can be measured
Downlink (DL) performance separately for UL and DL
•Delay, jitter, packet loss, …
•Directional traffic flow performance,
e.g., real-time traffic flows, UDP
•Traffic statistics from both points

•Real-time monitoring requires control

traffic
•Enables also two-way monitoring
possibility in a single point
Figure 3. An end-to-end measurement •Requires flow identification and timing
architecture.
synchronization (for delay) between
measurement points

Easy Wireless Workshop, IST Summit, Budapest, 05.07.2007, J. Prokkola 6

 VTT TECHNICAL RESEARCH CENTRE OF FINLAND

Multipoint Measurements

•End-to-end performance can be split to network segment performances.

•E.g., what is the access network performance vs. core network performance?
•Enables bottleneck identification (e.g., how do individual network devices work?)

end-to-end performance •Measurement tool units

Network segment 1 performance
Network segment 2 performance
could be passive
components, and e.g.,
network administrator
could activate monitoring
when needed
•Analysis and result
collection is challenging
•Measurement units could
e.g., communicate with a
centralized analysis tool

Figure 4. An example of a 3-point measurement architecture.

Easy Wireless Workshop, IST Summit, Budapest, 05.07.2007, J. Prokkola 7

 VTT TECHNICAL RESEARCH CENTRE OF FINLAND

QoS measurement tools used in EW: MOSET

• MOSET (VTT’ s freeware) is an active

mobile service testing tool
• A testing application for mobile
phone
•Execution of measurements /
tests
•Instant results
• A testing server
•Collect & manage results
•Distribute test applications and
tests
• Service response time measurements
•Connection establishment
•Retrieval of service content from
network
•Measures HTTP/TCP performance
Figure 5. MOSET architecture. from the user point or view

Easy Wireless Workshop, IST Summit, Budapest, 05.07.2007, J. Prokkola 8

 VTT TECHNICAL RESEARCH CENTRE OF FINLAND

QoS measurement tools used in EW: QoSMeT

•QoSMeT (VTT) is a passive tool for measuring

one-way end-to-end network QoS from the
application’s point of view
•Real-time measurements & accurate per
packet statistics
•Is able to measure QoS of real-time
applications (e.g., VoIP, video conferencing),
but other networking applications can be
measured as well.
•The tool works practically over any kind of
network as long as IP is supported
•QoSMeT can be run in the same device with
the measured application or within the network
path at desired points
•Measures e.g.: Delay, Jitter, Packet loss,
Connection break duration (e.g., during a
handover), Throughput and Offered load, and
Figure 6. QoSMeT architecture. the volume of data sent/received
•GPS is used for clock synchronization

Easy Wireless Workshop, IST Summit, Budapest, 05.07.2007, J. Prokkola 9

VTT TECHNICAL RESEARCH CENTRE OF FINLAND

QoS measurement tools used in EW: M5 Multi-Analyzer

•M5 (NetHawk) enables protocol monitoring, call and session tracing, key
performance indicator (KPI) analysis, QoS measurements, and radio
optimization measurements
•Monitors and analyzes UTRAN, GERAN, UMAN, IMS, WiMAX and core
network interfaces at the same time.
•Supports all common transport techniques –STM-1/OC-3, STM-4/OC-
12, E1/J1/T1 in different modes and Ethernet
•Product components: Standard PC, NetHawk Adapters & cables and
NetHawk M5 analysis SW
•Enables multipoint flow measurements with QoSMeT
•QoSMeT handles the end-points, while M5 takes traces within the
network path and decodes QoSMeT’ s control packets

Easy Wireless Workshop, IST Summit, Budapest, 05.07.2007, J. Prokkola 10

 VTT TECHNICAL RESEARCH CENTRE OF FINLAND

QoS measurement tools used in EW: Agent based solution

• The EW agent (Moviquity) is a software component

attached to the terminal and server software:
•Measurement of main E2E QoS parameters (BW,
delay, jitter, packet loss)
•Action control (switch to preferred network, codec
change)
• Three kinds of agents are required:
•Server Agent (EWSA) including Central Agent
•Network/Domain Agent (EWDA)
•User/Mobile Agent (EWMA)
• Via continuous feedback, it is possible to monitor
actively the communications environment.
• Complete knowledge of the network and service status
helps the server:
•To share and administrate resources with better
efficiency
•Provide the maximum possible quality to the user
taking into account the network the application
Figure 7. Agents taking delay measurements: It is service and the user profile
seen that during video reception there is clearly
more delay in the network.

Easy Wireless Workshop, IST Summit, Budapest, 05.07.2007, J. Prokkola 11

 VTT TECHNICAL RESEARCH CENTRE OF FINLAND

EW Measurement Examples:
VHO Performance
0.2

0.18
Delay 10
Connection break length
0.16

0.14

Connection loss length [s]

1
0.12 0.0 50.0 100.0 150.0 200.0 250.0
Delay [s]

0.1

0.08

0.06 0.1

0.04

0.02

0
0.01
0.0 50.0 100.0 150.0 200.0 250.0 Time [s]
Time [s]

•A simple handover scenario (VoIP application using Mobile IP):

LAN WLAN 3G WLAN LAN
•Great differences in delay and jitter between network technologies
(e.g., 3G one-way-delay is only barely acceptable for VoIP)
•Connection break length is on the order of seconds during handovers

Easy Wireless Workshop, IST Summit, Budapest, 05.07.2007, J. Prokkola 12

 VTT TECHNICAL RESEARCH CENTRE OF FINLAND

EW Measurement Examples:
HSDPA vs. WCDMA performance in a live network
Downlink goodput/throughput performance as a function of data packet size

1200.0
•A great improvement with
Goodput HSDPA is got as compared to
1000.0 the basic 3G/WCDMA
•With UDP, goodput near to the
800.0
UDP HSDPA
operator’s limit (1 Mbit/s) is
Goodput [kbit/s]

UDP 3G achieved with reasonably small

600.0 TCP HSDPA packet sizes.
TCP 3G •However, good TCP
400.0
UDP HSDPA Throughput performance still requires quite
large packet size
200.0
• RT delay is high also in
HSDPA, thus limiting the
performance of TCP
0.0
10 100 1000 10000 100000 1000000 10000000 => A typical WWW user does
UDP packet size [B], TCP downloaded file size [B] not see the real potential of
HSDPA!

Easy Wireless Workshop, IST Summit, Budapest, 05.07.2007, J. Prokkola 13

VTT TECHNICAL RESEARCH CENTRE OF FINLAND

EW Measurement Examples:
Multipoint Measurements in VTT’s CNL 3G Network
0.100

0.090
•Uplink delay behavior in
0.080

0.070
different sections of the
0.060
network
Delay [s]

0.050 •Lightly loaded network,

0.040 End-to-end
Radio access
to minimize the effects
0.030
RNC simulator of queuing delay
0.020 Ethernet (after RNC)

0.010

0.000 •Most of the total delay is

0.0 5.0 10.0 15.0 20.0
Measurement time [s] caused by the radio
access part
•The effect of laboratory
LAN is practically
meaningless

Easy Wireless Workshop, IST Summit, Budapest, 05.07.2007, J. Prokkola 14

VTT TECHNICAL RESEARCH CENTRE OF FINLAND

Conclusions

•The complexity of QoS measurements grow when increasing the

number of measurement points
•While single point measurements are simple, the information
provided is very limited
•Analysis of even a large monitoring network can be simple if
only traffic statistics are needed. However, traffic flow behavior
analysis is complicated and needs also clock synchronization if
delay measurements are desired.
•QoS measurements play important role in EW, several tools are
used, and new tools have been developed
•The tools are used in revealing the performance of the existing
networks, giving feedback to applications, and in measuring the
performance of the developed QoS methods.
•Interesting performance behaviors have been found in numerous
measurements in EW, and several publications have been done.

Easy Wireless Workshop, IST Summit, Budapest, 05.07.2007, J. Prokkola 15