Sie sind auf Seite 1von 26

EW

1 (26) 01.06.2005

QoS Measurement Methods and Tools

Projects full name: Projects short name: Project number: Customer reference: Drawn up by: Version: Status:

Easy Wireless, Allow seamless roaming between wireless networks while maintaining Quality of Service EW

E4SU00169 Tekes Dnro 2625/31/03


Jarmo Prokkola, Mikko Hanski

[] [] [x]

Draft Proposal/change proposal approved

Change history: Version 0.1 0.2 0.3 0.4 0.5 0.6 0.7 1.0 Date 03.12.2004 20.12.2004 17.01.2005 25.01.2005 26.01.2005 03.02.2005 05.04.2005 01.06.2005 Made by Jarmo Prokkola Jarmo Prokkola Jarmo Prokkola Mikko Hanski Mikko Hanski Jarmo Prokkola Jarmo Prokkola Comments ToC Chapters Added Text Added Chapters added Text added Text added Proposal version The first approved version

DISTRIBUTION

EW

2 (26)

1 2

Introduction......................................................................................................................................... 4 On The Principles of QoS Measurements........................................................................................... 5 2.1 Subjective QoS vs. Objective QoS ............................................................................................. 6 2.2 Simple end-to-end performance measurements.......................................................................... 7 2.2.1 On the timing accuracy ....................................................................................................... 8 2.3 About the QoS performance metrics........................................................................................... 9 2.4 Multi-point measurements ........................................................................................................ 10 2.4.1 Link-to-link measurement................................................................................................. 11 2.5 Measurement analysis ............................................................................................................... 11 2.5.1 Sampled measurement vs. full measurement .................................................................... 12 3 QoS Measurement Methods and Tools............................................................................................. 14 3.1 Network analyzers..................................................................................................................... 14 3.1.1 Ethereal ............................................................................................................................. 15 3.1.2 Packet Capture Libraries................................................................................................... 17 3.1.2.1 Libpcap.............................................................................................................................. 17 3.1.2.2 WinPcap ............................................................................................................................ 17 3.2 Real-Time QoS Monitoring Tool.............................................................................................. 17 3.3 Mobile Service Testing and Measurement Tool (MOSET)...................................................... 19 3.3.1 Multi-point Measurements with MOSET ......................................................................... 21 3.4 Other QoS Monitoring Tools .................................................................................................... 22 3.4.1 Commercial QoS Solutions............................................................................................... 23 4 Summary and Conclusions................................................................................................................ 24 5 References......................................................................................................................................... 25

EW

3 (26)

ABBREVIATIONS
API ARQ ATM CRC DoS EDGE EW GPRS GPS GUI HTTP IP MOS MOSET NTP OS OTA OWD QoS RTCP RTP RTT TCP UDP WLAN Application Programming Interface Automatic Repeat Request Asynchronous Transmission Mode Cyclic Redundancy Check Denial of Service Enhanced Data Rates for Global Evolution Easy Wireless General Packet Radio System Global Positioning System Graphical User Interface Hypertext Transfer Protocol Internet Protocol Mean Opinion Score Mobile Service Testing and Measurement Tool Network Time Protocol Operating System Over The Air One Way Delay Quality of Service Real-Time Control Protocol Real Time Protocol Round Trip Time Transmission Control Protocol User Datagram Protocol Wireless Local Area Network

EW

4 (26)

INTRODUCTION The ITEAs Easy Wireless (EW) project will focus on service continuity for mobile users of wireless information network services. The aim is to ensure transparent connectivity as users are transitioning between different wireless network technologies. The Easy Wireless scenario is described in FPP document. The EW results will be the provisioning of systems and technology that allow building applications that empower and simplify the daily life of people, both private and professional. To achieve service continuity it is mandatory to study end-to-end QoS and resource management across different network technologies and across multiple paths. The project will study the fundamental QoS mechanisms and the level of QoS that can be achieved for each of the technologies addressed.
Factory WLAN Network Office WLAN Network

IP NETWORK Wide Services & Interactions

AdHoc Mobile Net Community PAN Network

Local Services & Interactions WLAN H/2 WLAN 802.11 GPRS/UMTS

Figure 1 The Easy Wireless scenario

The work package 4 of the EW project deals with end-to-end quality of service (QoS) monitoring in heterogeneous network environment. This report is mainly a literature review on the current QoS measurement methods and tools with a survey of the philosophy behind the QoS measurements. This report also functions as a basic ground for our goal to develop a QoS measurement tool, which is application independent and provides a possibility for real-time monitoring in addition to the accurate measurement database for latter analysis. The document is basically split into two parts. One part (chapter two) deals with general issues that are related to QoS and network measurement methods. The other part (chapter three) deals with the actual tools available for QoS & network monitoring and measurements.

EW

5 (26)

ON THE PRINCIPLES OF QOS MEASUREMENTS Measuring network quality of service (QoS) is basically very close to network traffic measurements. In fact, at idea-level QoS measurements lie usually on top of traffic measurements. The difference is the way how the measurement results are analyzed. In network traffic studies, the interesting thing is the traffic itself and its effects: network load, queuing performance, source traffic processes, large scale traffic flow models. Especially the analysis of traffic processes and models requires accurate information of collected traffic [8]. In QoS analysis, on the other hand, network traffic itself is not the interesting thing, but it is rather just used as a tool to reveal the performance characteristics (delay, maximum throughput, etc.) of links, networks and systems. Generally, the measurement methods of telecommunication networks can be categorized to passive and active methods. These can be further divided to probing, tracing and monitoring. Tracing is usually active (the traffic is specially generated for the measurements) and the purpose is often to trace the performance of a certain route hop by hop. Monitoring, on the other hand, usually refers to passive methods, where information about the existing network traffic passing through some nodes (routers, servers, etc.) is collected. Probing can be either active or passive. Active probing measures the effects of specially generated test flows, while passive probing collects information about the effects of existing network traffic. [3] The basic idea of QoS measurements is illustrated Figure 2. The following separated functional entities can be extracted from the figure: Measurement point: The actual measurements are done at measurement points that are, in practise, some network nodes under interest (e.g., routers, terminals). General rule is that the more measurement points, the more accurately the network behaviour can be determined, while the analysis also gets more complicated. Traffic measurement tool: This is the tool that performs the measurements, i.e., captures the packets and collects information about them. Hence, there must be one measurement tool per measurement point. If passive methods are used, it is very important that the measurement tool has minimal effect to the traffic itself. At this stage some preliminary processing can be done to the measurement information. QoS analysis tool: This tool analyses the data provided by traffic measurement tools and calculates the actual QoS metrics. In practice, this means, for example, delay calculation of single packets travelling through measurement points. There usually exists a single QoS analysis entity in the measurement system, but naturally the actual calculation process can be distributed in nature. Thus, the QoS analysis tool must have some common understanding of the measurement setup and it may control several traffic measurement tools and, thus, measurement points.

EW

6 (26)

QoS: The final purpose, naturally, is to obtain information about the QoS of the system under interest. This information can simply be a database for more accurate post analysis purposes, or the information might be in real-time format, in which case some sort of real-time QoS monitoring tool is needed. This tool might also be a complicated QoS application with a possibility to control the actual traffic measurements through QoS analysis tool.

QoS (real-time monitor or database)

QoS analysis tool

Traffic measurement tool Measurement point (network node)

Traffic measurement tool Measurement point (network node)

Figure 2. The basic principle of QoS measurements.

A good overview on QoS monitoring is provided in [4], which also presents a somewhat similar principle to that presented in Figure 2, but it focuses especially on real-time QoS monitoring. 2.1 Subjective QoS vs. Objective QoS The first thing here is to define, what is meant by quality of service. QoS can be classified as subjective and objective QoS. Objective QoS means usually something concrete and quantitative, i.e., something that can be measured directly (delay, packet loss). Subjective QoS, on the other hand, corresponds to the service quality from the user perspective that is, how does the user feel about the quality. Naturally, subjective QoS is much more difficult to measure than objective QoS, since the user experienced quality does not necessarily follow technical solutions that can have direct impact to the objective QoS. [20] The only reliable way to measure subjective QoS includes test cases with real users, which leads to complex test setups with a lot of people involved. Mean opinion score (MOS) tests are often used in actual measurements [21], [22]. Figure 3 clarifies the idea behind the classification of QoS. In this report, our focus is mainly on objective QoS.

EW

7 (26)

Figure 3. Subjective vs. objective quality. [19]

2.2

Simple end-to-end performance measurements The simplest form of end-to-end performance measurements can be executed as single-point measurements directly from the terminal, which uses some service. Consider e.g., a case in Figure 4, where a terminal device measures the performance of some service over the network. The measurement software can be an application on top of the protocol hierarchy, in which case the measurement gives directly an idea about the application layer performance that is usually the desired case in QoS.

Figure 4. The principle of single-point end-to-end performance measurement.

As it can be assumed, with this kind of measurement setup, it is only possible to obtain information about the total round trip performance of the system. Often it is assumed that the links in both directions have about the same performance and, therefore, for example, one way delay is got simply by dividing the total delay by two. However, this can lead to serious errors since, as it is known, many communication systems are asymmetric in nature. The simplest enhancement to end-to-end performance measurement is to attach measurement software to both ends. In Figure 5 this kind of enhancement is added to the simple end-to-end measurement setup. In this way, the performance of different directions (from terminal to server and from server to terminal) can be analyzed separately in addition to the total round trip performance. In addition, this kind of setup provides a possibility for real time QoS performance analysis (one way delay can be continuously

EW

8 (26)

calculated, for example). Even with this fairly simple enhancement, we come up against the basic problem of multi-point measurements: the measurement points must have some shared knowledge in order to measure same event.

Figure 5. The principle of directional end-to-end measurement setup.

2.2.1

On the timing accuracy The accurate multi-point measurements require accurate synchronization of clocks of the measurement devices. Consider e.g., delay calculation of a single packet in two-point end-to-end measurement in Figure 5. If for example time stamps are used in packets, careful clock synchronization is needed between end devices in order to calculate delay to both directions. This clock synchronization now acts as the shared knowledge of the separate measurement points. The timing accuracy is not a trivial problem, since computer real-time clocks are very inaccurate they might drift several seconds in a day [3]. This is definitely not enough, since (at least) millisecond-order accuracy is usually needed in network delay measurements. Of course, the needed accuracy is dependent on the QoS requirements and the application under interest. Also the accuracy is dependent on network structure (bandwidth, especially), since e.g., in GPRS network (nominal channel bit rate of 53.6 kbit/s) 1 ms corresponds only to 53.6 bits, but in basic 10 Mbit/s network, it corresponds already to 10 kbits, not to mention faster modern networks of several Gbits/s. Clock synchronization is its own form of art [9], [10], [11]. Many of the various methods use some third party clock reference. Network time protocol (NTP), for example, uses external servers for providing timing information, which is transferred over the network with IP. However, the accuracy of NTP at its best is only about 10 ms, which is not enough for accurate delay measurements in high speed networks [5]. Accurate synchronization can be achieved e.g., with Global Positioning System (GPS) [12], but still it can be argued that self-consistency checks should made periodically [6].

EW

9 (26)

2.3

About the QoS performance metrics Probably the most essential and general performance metrics are: Delay Jitter Throughput Packet loss

Another somewhat different, yet important, metric is data transmission efficiency, i.e., how much control information is needed in order to get the actual data through. This metric it is not usually taken into consideration even though in power limited wireless devices it is quite essential for power consumption minimization. Single-point measurements (Figure 4) provide end-to-end performance information, which can be used to determine QoS. This setup enables the possibility to measure round trip time (RTT), i.e., the time from the initiation of the service request to the reception of the service reply. This information is valuable QoS metric and gives direct insight of the total performance of the system. From RTT (tRTT) it is easy to calculate the average throughput of the system S ete = N d t RTT , (1)

if the amount of transferred data (Nd) is known. In this way delay and throughput are related, but the relation is not necessarily strict. Consider e.g., a single-point passive measurement, where passing traffic is measured at some point of a network. In this, throughput can be easily measured, but delay can be not. Jitter is also related to delay, since it is understood as the variation of the delay. Jitter has several definitions [7], e.g., the maximum variation of the delay, but the most common one is probably the standard deviation of the delay. Jitter can be calculated from the exact delay measurements, but not from the average values. Jitter is easier to calculate than absolute delay, since it only needs the delay difference between sequential packets, but not absolute clock synchronization of the measurement points. Jitter can be controlled with buffers, but this is done at the expense of delay [7]. Packet loss is more or less independent metric as compared to the other metrics even though some approximation of it can be drawn from raw throughput, if offered traffic load is known. Lost packets are usually detected via sequence numbers and ARQ-methods (automate repeat request). Packet loss has its effect to the other metrics. Consider e.g., an application layer jitter measurement, where jitter is measured directly from sequentially arriving packets. If lower layers fail to deliver some packets correctly, jitter is increased because of the gaps of missing packets. It might also be the case that the application layer packets are all delivered correctly, but jitter is still

EW

10 (26)

increased because of the erroneous lower layer packets and retransmissions provided by ARQ-methods. In this case the delay is also increased. 2.4 Multi-point measurements Bidirectional end-to-end measurements give good insight in to the overall performance in both directions. However, if one is interested in the reasons behind the behavior of the performance, more complicated measurement setup is needed. Consider for example a case in Figure 6, where a mobile station uses a service in internet via cellular network. The network structure has several diverse segments, which all affect to the total performance. Roughly, two major network segments exist in such a scenario: cellular network and the Internet service. Internet service can be split to the actual backbone network (Internet) segment and to the server portion, which offers the service to the mobile user. The server performance should be higher than the network performance, in most cases, but the server can also be a bottleneck if the number of potential users of the service is underestimated or the service is otherwise badly implemented. Cellular network can also be divided to the cellular core network and radio access portions, where the latter is usually the most challenging part of the network and potentially the greatest bottleneck, because of the typically lower data rates of the radio interface as compared to the fixed network. Figure 6 also presents a possible principle for measuring the performance of the network segments separately. Now, as in the presented directional measurements, shared knowledge between measurement points must exist [27]. This means clock synchronization, which probably is the most difficult part of it, and also flow identification. By flow identification we mean that there must be a way to identify certain traffic flows, and more specifically, individual packets, at the multiple measurement points in order to be able to calculate QoS metrics (delays, losses, etc.). Flow identification is usually based on the information in packet headers (addresses, ports, packet identification numbers, timestamps). In addition to this, the most difficult problem of this kind of measurement, in practice, can be the actual execution of the measurement, because in such a heterogeneous network, there usually exists also several companies in the chain (e.g., mobile phone operators, Internet service providers, customized service providers). A mobile phone operator, for example, might not be very interested to allow traffic measurements inside their network, because of the confidentiality issues of the customers. Some more information about multi-point QoS measurements or especially QoS distribution monitoring, as it is sometimes called, is provided in [4]. In here, it is described, how to manage complex multipoint measurement systems with several distributed monitoring devices around the network. Also, some attention is put to synchronization of reporting of the results in real-time.

EW

11 (26)

Radio Access

Cellular core network

Internet

Server

Measurement method Measurement method

GPRS / 3G (packet switched) BTS/Node B Mobile Station BSC/RNC SGSN GGSN Operators network SGSN (public) Internet

Customized Server

Figure 6. An example of a heterogeneous network with multi-point measurement setup.

2.4.1

Link-to-link measurement Ultimately multi-point measurements lead to link-by-link measurements or hop-by-hop measurements, where the performance of every link or every hop in the communication chain determined, respectively [13], [3]. This is very difficult and time-consuming, but is needed, if for example some broken or single bottleneck network components are needed to be found. In terms of QoS-handling, a more common multi-point measurement, like network segment measurement (Figure 6), is usually enough. A well-known tool for hop-by-hop measurement is traceroute, while an example of actual linkby-link measurement tool is pathchar [14]. The performance of pathchar is quite good with slow and moderate speed links (up to 10 Mbit/s), but weakens radically with fast links. Another problem is that the performance gets worse with increasing hop count, and also, the number of required probes increases quickly.

2.5

Measurement analysis In accurate QoS measurements, the effect of the network traffic itself must be included. High traffic load of the measured network creates extra queuing delays, which might give false information about the real network performance. Therefore, one should be aware about the current load of the network when performing measurements. Otherwise no accurate decisions can be drawn from the basis of the measurements. On the other hand, the end user is only interested in end-to-end quality of service, regardless of the underlying technologies, i.e., the user is only interested that does the service work or not, and does not care what are the reasons behind bad service level. In this respect, simple end-to-end performance measurements in a public network with all the other network traffic included are well-grounded and valuable. For a reader, who is interested in QoS methods in IP-based mobile access networks, a good overview is provided in [2]. QoS provision methods also

EW

12 (26)

relate closely to analysis, since QoS analysis must usually be done in order to do accurate QoS management. 2.5.1 Sampled measurement vs. full measurement Several principles for the actual measurements exist. In accurate traffic trace (full measurement), every packet must be registered at every measurement point. In this way, there are no limitations put to future analysis. The traffic measurement database should contain at least the following attributes of the packets: Timestamp (with sufficient accuracy) Total packet size Source and Destination (e.g., network level addresses) Packet ID (e.g., IP protocol ID number) Protocols in the packet (+ some important protocol parameters, e.g., TCP flags to distinguish between data and acknowledgement packets)

With this information, it is possible to track single packets in the network and calculate accurate delays, jitters, throughputs and losses between measurement points. It is also possible to inspect the exact behavior of the system, even though it might be somewhat laborious. The drawback of this method is that the database will quickly become very large in highly loaded systems. Consider e.g., a 10 Mbit/s link with an average load of 30 %. If the network level packets, which are usually are split to about 1500 Bytes (12 000 bits) because of the limitations of Ethernet, are now tracked, there will appear about 250 packets/s in the link. Consequently, after 1 h measurement, there will be already about 900 000 lines of packet information in the measurement database! This is the reason why other, less demanding, methods are often used. The most straightforward modification to the full measurement is to perform some pre-processing of the measurement data before writing the database. Usually, this involves calculating averaged values of the metrics in interest in given intervals. This is an efficient way to compress the measurement database, but it also disables many possibilities for future analysis, since the information of single packets will no longer be available. Another possibility to simplify measurement is to track only the application layer performance, which might be also relevant for pure end-to-end QoS measurement purposes. The drawback, however, is that in this way one can not see inside lower layers, so it is nearly impossible to explain the behavior of the performance, if it is not application dependent. The used measurement method should be chosen from the basis of QoS requirements of the system. If, for example, only average total delay is under interest, simple averaged end-to-end performance measurements are enough.

EW

13 (26)

If, again, one is interested in explaining the characteristics of the delay behavior, accurate multi-point measurements with single packet traces might be needed.

EW

14 (26)

QOS MEASUREMENT METHODS AND TOOLS In this chapter, first the concept of network analysis is introduced with some review on popular network analysis tools. After this, the actual QoS measurement methods and tools are taken under discussion. Even though the focus is on QoS analysis, network analysis very important and is needed in a way as a tool for QoS analysis, i.e., QoS analysis operates logically above network analysis.

3.1

Network analyzers Network analysis means the event of capturing the packets traveling in the network and analyzing them to conclude what is happening on the network. Network analysis software is used to decode the common protocols and show the network traffic in a form that is human-readable. The capability to set the network interface into promiscuous mode combined with the ability to read the packets from the data link layer gives application a chance to monitor all the packets on the local cable [15]. Network analyzer usually comprises of five common parts [16]: Hardware Even though most of the network analyzers are softwarebased, there are some hardware network analyzers, which are used to detect hardware faults such as Cyclic Redundancy Check (CRC) errors, voltage problems and negotiation problems among others. Capture driver The part which performs the actual capturing and filtering of the network traffic (the core of the analyzer). Buffer The captured data is stored in the buffer. The data can be stored to buffer until it is full or the latest data can replace the oldest. Real- time analysis This feature is used to analyze the data as it comes out of the cable. Decoder Each protocol has its own decoder and the analyzer needs to have a decoder per protocol it supports. Decoder is responsible for showing the content of the traffic in a human-readable form.

Network analysis is a valuable tool for diagnosing network traffic and troubleshooting network problems. It can be a helpful tool for network engineers and system operators among others but is used for harmful purposes as well. There are many reasons why the administrators use network analysis. Here is a list of some of the reasons [16]: Converting the binary data in packets to human-readable format Troubleshooting problems on the network Analyzing the performance of a network to discover bottlenecks

EW

15 (26)

Network intrusion detection Logging network traffic for forensics and evidence Analyzing the operations of applications Discovering a faulty network card Discovering the origin of a denial of service (DoS) attack Detecting spyware Network programming to debug in the development stage Detecting a compromised computer Validating compliance with company policy As an educational resource when learning about protocols For reverse-engineering protocols in order to write clients and supporting programs

The following is a list of some of the common network analyzers: Ethereal Probably one of the best sniffers available. Ethereal is a free open source network monitoring tool which has become one of the most popular products used. It has a graphical user interface, can decode more than 400 protocols and runs on both UNIX- and Windows based systems. Ethereal is described in more detail later. [31] Tcpdump This UNIX-based sniffer is one of the oldest and widely used. [32] WinDump A Windows version of tcpdump. [33] Analyzer A free Windows-based sniffer from the developers of WinPcap and WinDump. [34] Packetyzer This is a free Windows user interface for Ethereal and can decode more than 483 protocols. It is built with Ethereal and winpcap. [35] Carnivore The codename for FBI's network analyzer. Carnivore is used to monitor network communication in connection with criminal investigation. [36]

3.1.1

Ethereal Ethereal is a free open source network analyzer that reads the packets from the network, decodes them and presents them in a human understandable form. Ethereal is possibly the most widely used analyzer and is a respectable

EW

16 (26)

competitor to the commercial ones. There are still bugs in the Ethereal code and it is still considered beta, but the wide range of authors fix the bugs after they are reported. Also new enhancements and features are developed all the time. Figure 7 presents the Ethereal Graphical User Interface (GUI), which contains a summary of the capture, detailed information on the selected packet and the raw captured data. In the summary pane the packet number, time, source and destination addresses, name and information on the highest layer protocol are presented. New columns can be added from the preferences. The middle pane is called protocol detail pane. It includes detailed information on all of the layers inside the captured packet. The lowest pane contains the raw captured data in both hexadecimal and ASCII format.

Figure 7. Ethereal GUI

Ethereal is able to decode more than 480 protocols. Because Ethereal uses a popular libpcap packet capture library, it is compatible with more than 20 other sniffers and capture utilities. It is able to read many other formats in addition to libpcap as well. The capture database can be saved to a file and opened for later analyzing. In Ethereal it is possible to filter the captured traffic so that you will only get the packets you are interested in. There are two types of filters: capture and display filters. Capture filter captures only the packets defined in the filter and display filter shows the defined packets from already captured traffic. The shown protocol fields can be defined by a filter as well. With Ethereal there are also some other supporting programs installed. These include Tethereal, which is a command line based version of Ethereal, Editcap to remove packets from a file and to translate the format of capture

EW

17 (26)

files, Mergecap, which is used to merge many capture files into a single file and Text2pcap, which can read ASCII hexadecimal dumps and write the data into a libpcap output file. 3.1.2 Packet Capture Libraries Packet capture libraries provide interfaces for applications such as network analyzers for straightforward packet capture from desired protocol level. 3.1.2.1 Libpcap Libpcap is a packet capture library, which provides access to the data link layer. It has functions to dump the binary packet contents to a file, to read the dumped file and a possibility to filter the capture. Libpcap works in UNIX environments. "The packet capture library, libpcap, provides implementation-independent access to the underlying packet capture facility provided by the operating system. Currently it supports only the reading of packets (although adding a few lines of code to the library lets one write datalink packets too)." [15] Many network monitoring software use libpcap for capturing the packets, Ethereal for example. 3.1.2.2 WinPcap WinPcap is an open source packet capture and network analysis library for Win32 platforms. It has a kernel-level packet filter, a low-level dynamic link library (packet.dll), and a high-level and system-independent library (wpcap.dll), which is based on libpcap version 0.6.2. [17] Packet.dll is used as an API, which offers the packet driver functions independent from the Windows OS. Wpcap.dll offers libpcap compatible high-level capture primitives, which makes the capture independent of the underlying operating system. [17]

3.2

Real-Time QoS Monitoring Tool A QoS monitoring tool for real-time measurement is under development at VTT. With the tool it is possible to monitor communication QoS of two applications communicating over networks. The tool has an end-to-end aspect to performance measurement.
Figure 8 shows the overall architecture of the tool.

EW

18 (26)

Figure 8. An overview of the monitoring tool

The software is installed to both the communicating computers. The tool uses WinPcap packet capture library to capture the packets. It is possible to capture all the traffic in the cable but a filter is set to monitor only the packets sent by the certain application running in the computer and the ones arriving from the peer computer as well. The packets are captured at the data link layer. The software records the departure times of the packets sent by the application in the same computer and the arrival times of the packets sent by the peer application. WinPcap gets these timestamps from the system clock. Other essential information, such as IP packet identifier field, is recorded as well. One feature that many of the monitoring software available lack is the counting of one way delay (OWD). With one way delay it is possible to tell the end-to-end delay between the source and the destination instead of calculating the round-trip delay, which assumes the one way delay to be symmetric. The end-to-end delay measurements can be used by the application at the endpoint to adapt to the situation. To be able to count these delays the packet departure and arrival timestamps are needed. The monitoring software records the departure timestamps and receives the arrival timestamps from the peer monitor. The corresponding timestamps are identified by using the identification field value of the IP-header as a key to store the timestamps. The user can define how often the monitoring software sends the timestamps and ids to the peer monitor. It could be done in realtime, after the monitoring or in 10 second intervals for example. TCP is used to guarantee the reliable transfer of this packet information. Another demand in counting the one way delay is that the computer system clocks must be synchronized. GPS synchronization is used to meet the demand. The packets are identified on the basis of the identifier and timestamp and so it is possible to count the one way delays of respective packets by comparing the departure and arrival times. If a match is not found, the packet has not arrived and it is considered lost. In addition to the delay and packet loss, jitter and throughput are calculated. The time interval at

EW

19 (26)

which the packet information is sent to the peer monitor defines how frequently the calculated QoS measures are shown to the user. The tool uses information from IP packets and thus is independent from the upper layer protocols. The timestamps are derived from the synchronized system clocks so the software is not restricted to monitoring applications using protocols such as RTP, which carries timestamps in its header. The application to monitor could be a VoIP software using UDP and RTP or video or voice stream using TCP or UDP for example. 3.3 Mobile Service Testing and Measurement Tool (MOSET) VTT has developed a novel mobile service testing and measurement tool (MOSET), which can be used in commercial mobile devices (e.g., mobile phones). The tool is designed for measuring the total end-to-end service response time (delay), between a mobile device and a server over the networks, from the user point of view. System architecture is described in Figure 9 [1]. The system is used to measure and collect performance information of HTTP (Hyper Text Transfer Protocol) requests over cellular network (GPRS, EDGE, 3G, ) as perceived by the user. The system consists of a testing server, which is used for collecting the benchmark result and analysis, and a J2ME application in a mobile device for executing the actual measurements. The testing server is also used for distributing the measurement application over the air (OTA), so basically the idea is that a user can download MOSET mobile software and measure the performance of the used mobile system painlessly with no other setups or requirements.

Figure 9. The basic principle of MOSET.

When the application is launched, it collects information about service response times and, finally, sends the measurement data to server, which saves the results and delivers them to internet. For making the management of the measurement results more straightforward, the user can include

EW

20 (26)

information also about the measurement environment such as mobile device type, measurement location, etc. A lot of information can be extracted from MOSET and this includes: Service connection time (average, maximum and minimum times) Service content loading time (average, maximum and minimum times) Total amount of bytes transferred as payload in each request Total count of failed service requests The total time of the whole test run Total amount of data transfer in bytes of the test run Average data transfer [kB/s] of the test run.

In actual measurements, it is usually required that MOSET will request a certain amount of data, i.e., a specific sized data packet. This can be done by actually having some specified files of predetermined sizes in some test server or by dynamically creating these files within the server based on the request. MOSET is at its best in analyzing averaged end-to-end performance. An example of average throughput analysis is presented in Figure 10. Figure presents the measured performance (measured at VTT with MOSET) of GPRS vs. EDGE as a function of requested application data packet size (i.e., the file size requested from server). As it can be expected, EDGE outperforms GPRS, but an interesting notification is that the performance is highly dependent on the requested packet size and the advantage of EDGE over GPRS is deteriorated with smaller packets. With small data packets, the setup of TCP connection and HTTP request will have high relative effect, so it is obvious that the performance of actual data throughput will be lower with smaller data packets, but it still does not adequately explain why the difference between the performance and EDGE and GPRS narrows so drastically.

EW

21 (26)

100 000

Average end-to-end troughput [bit/s]

90 000 80 000 70 000 60 000 50 000 40 000 30 000 20 000 10 000 0 1 10

EDGE GPRS

100

1000

10000

Requested application data packet size [kB]

Figure 10. An example of MOSET throughput measurements as a function of requested application data packet size in GPRS and EDGE networks.

3.3.1

Multi-point Measurements with MOSET The service testing tool allows separate estimation of the delay caused by the user terminal, wireless network and server. As seen in the Figure 11, testing tool similar but separate Java-based measurement applications are used simultaneously in three locations. The test includes a mobile phone and a laptop computer connected to cellular network and a Linux workstation connected directly to fixed network (Internet).

Figure 11. Mobile service testing tool and three-point measurement setup.

The first measurement is taken by a measurement application running in a mobile phone, which represents a total end-to-end service response time as

EW

22 (26)

seen by the user. The second measurement is taken by a measurement application running in a laptop computer, which is connected to cellular network. This measurement represents the service response time over cellular network without the application and the mobile phone, in which it is assumed that they will provide measurable extra delay. The problem, however, is that the delay caused by mobile platform software is so small that it tends to drown to the deviation of the delay caused by the radio interface. The third measurement is taken by a measurement application in a Linux workstation, which is directly connected to the Internet, to measure the service response time without the delay caused by the cellular network and mobile phone. In all these, the delay response times includes the time needed to open a TCP connection, send a HTTP request to the server and receive the content as a reply. [1] The problem of this measurement setup is that the distinct measurement points do not have shared knowledge and, in fact, they do not measure the same flow or event. Instead, each measurement point does its own measurements. In this way, accurate knowledge of the performance of different network segments can not be made, but by evaluating long time averages at the measurement points, one gets information about the average performance differences between network segments. 3.4 Other QoS Monitoring Tools In [4] there are introduced several methods that have been developed for application-layer and QoS monitoring: RMON2 With RMON2 probe it is possible to monitor and decode also the protocols operating higher than network layer. It sees above the IP layer and can read the encapsulated higher-layer headers. This provides application-level monitoring. [23] RTFM Real-time flow measurement architecture was developed to monitor the network flows. A flow is identified by its source and destination addresses at various layers so this mechanism provides application-level monitoring as well. [24][25] RTCP It is possible to perform end-to-end QoS monitoring with Real-time control protocol (RTCP) messages. They contain the fields such as timestamps, which can be used to calculate QoS parameters. [26] Chen et al. have developed a method to use test cells to monitor QoS in an ATM networks. They measure the performance of a virtual connection by sending test cells that will experience the same QoS as the selected user connection. This way the QoS could be derived from the test cells. [28] Mourelatou et al. have introduced a QoS monitoring method in which agents monitor the end-to-end QoS and provide the management system with this information. [29]

EW

23 (26)

Ehab Al-Shaer has developed an agent based QoS monitoring method as well. This method is designed specially for multimedia networks. It is required that the multimedia application distribution is provided before the monitoring. [30]

3.4.1

Commercial QoS Solutions Acterna Acterna is a company offering communications test and management solutions. Some examples of its products are Quality Management System for end-to-end QoS management and PVA-1000 VoIP Network Analysis Suite, which provides analysis of VoIP calls including jitter and packet loss but lacks one way delay. [37] Telchemy Telchemy is a company providing tools to monitor and manage the performance of VoIP, video over IP and other real time services. With their products it is possible to monitor the service quality and get an estimate of the user perception of Quality of Service and help to realize the reasons for quality degradation. [38]

EW

24 (26)

SUMMARY AND CONCLUSIONS In this document, we introduced a literature review on the current QoS measurement methods and tools, and also paid some attention to the philosophy behind the QoS measurements and QoS measurements versus network traffic measurements. The service performance research has mainly focused on network analysis, explaining accurately some specific behaviour of the network and protocols in a certain situation, and analyzing the traffic models generated by some applications. There are also various tools available for this purpose. However, the actual QoS measurements in heterogeneous network environment have been getting less attention until recently. This means application layer performance from the user point of view. VTT has developed a tool for end-to-end QoS measurements over cellular networks. While this tool uses active measurement technique, we are now developing a new tool for real-time passive QoS monitoring of applications. With this tool it is possible to get immediate measured feedback of the application performance with a possibility for more detailed performance analysis of the recorded measurement database. As we deal with complex heterogeneous networks consisting of several different kinds of network segments, understanding the end-to-end performance behaviour becomes difficult. To find out the individual QoS characteristics of the network segments that add to the total end-to-end performance, multi-point measurements are needed. The actual implementation of these measurements, however, proposes new problems of measurement point timing accuracy, flow synchronization, and also user privacy problems, if for example detailed traffic analysis of public cellular network is needed. The idea of this document was to scratch the surface of the extensive topic of QoS measurement and to survey QoS monitoring related tools available today and maybe tomorrow.

EW

25 (26)

REFERENCES [1] M. Palola, M. Jurvansuu, J. Korva; Breaking Down the Mobile Service Response Time, IEEE International Conference on Networks, icon2004, Singapore, 16-19 November 2004. [2] J. Manner, Provision of Quality of Service in IP-based Mobile Access Networks, Academic Dissertation, University of Helsinki, Department of Computer Science, Faculty of Science. Dec. 2003. [3] M. Peuhuri, Internet traffic measurements, aims, methodology and discoveries, Licentiate Thesis, Helsinki University of Technology, Department of Electrical and Communications Engineering, May 2002. [4] Y. Jiang, C-K. Tham, C-C Ko, Providing Quality of Service Monitoring: Challenges and Approaches, IEEE/IFIP Network Operations and Management Symposium 2000 (NOMS 2000). 10-14 April, 2000. pp.115 128. [5] V. Paxson, Measurements and Analysis of End-to-End Internet Dynamics, Ph.D. dissertation, University of California, Berkeley, April 1997. [6] V. Paxson, On calibrating measurements of packet transit times, Technical Report LBNL-41535, Lawrance Berkeley National Laboratory, Network Research Group, 1998. [7] A. Tannenbaum, Computer Networks, 4th edition, Prentice-Hall, 2003 [8] J. Prokkola, Modeling, Measurement and Simulation of Teletraffic Processes, Master's Thesis (in Finnish only). University of Oulu, Department of Electrical Engineering, 2001 [9] D. Mills, Experiments in Network Clock Synchronization, RFC957, M/A-COM Linkabit, September 1985. [10] C. Ellingson, R. Kulpinski, Dissemination of System Time, Communications, IEEE Transactions on [legacy, pre - 1988] , Volume: 21 , Issue: 5 , May 1973. pp. 605 624 [11] Chung-Sheng Li, Y. Ofek, Distributed source-destination synchronization using inband clock distribution, Selected Areas in Communications, IEEE Journal on, Volume: 14, Issue: 1, Jan. 1996. pp.153 - 161 [12] W. Lewandowski, J. Azoubib, W. Klepczynski, GPS: primary tool for time transfer, Proceedings of the IEEE, Volume: 87, Issue: 1, Jan. 1999. pp. 163 172 [13] R. Carter, M. Crovella, Measuring bottleneck link speed in packet-switched networks, Technical Report, Performance Evaluation 27 & 28, 1996. pp. 297 318. [14] A. Downey, Using pathchar to Estimate Internet Link Characteristics, SIGCOMM99, September 1999. [15] W.R. Stevens, "Unix network Programming", 2nd edition, Prentice-Hall,1998 [16] A. Orebaugh, "Ethereal Packet Sniffing", Syngress Publishing, 2004 [17] URL: http://winpcap.polito.it/ [18] A. Orebaugh, "Ethereal Packet Sniffing", Syngress Publishing, 2004 [19] C. Scaefer, et al., Subjective Quality for Multiplayer Real-Time Games, In: Proc. of the 1st on Network and System Support Games (NetGames2002), Braunschweig, Germany, April 16-17 2002, pp. 74-78.

EW

26 (26)

[20] T. Sutinen, End User Service Quality In Multi-Access Networks, Masters thesis. University of Oulu, Department of Electrical and Information Engineering, 2004. [21] ITU-T Rec. P. 800, Methods for subjective determination of transmission quality, August 1996. [22] N. B. Seitz, et al., User-Oriented Measures of Telecommunication Quality, IEEE Communications Magazine, January 1994 [23] S. Waldbusser, "Remote Network Monitoring Management Information Base Version 2 using SMIv2", IETF RFC 2021, 1997 [24] N. Brownlee, C. Mills, and G. Ruth, "Traffic Flow Measurement: Architecture", IETF RFC 2063, 1997 [25] N. Brownlee, "Traffic Flow Measurement: Experiences with NeTraMet", IETF RFC 2123, 1997 [26] H. Schulzrinne, S. Casner, R. Frederick, and V. Jacobson, "RTP: a Transport Protocol for Real-Time Applications", RFC 1889, 1996 [27] F. Lange, R. Kroeger, M. Gergeleit, JEWEL: design and implementation of a distributed measurement system, Parallel and Distributed Systems, IEEE Transactions on, Volume: 3, Issue: 6, Nov. 1992. pp. 657 671. [28] T. M. Chen, S. S. Liu, M. J. Procanik, D. C. Wang, and D. D. Casey, "INQIRE: A Software Approach to Monitoring QoS in ATM Networks", IEEE Network, pp. 32-37, March/April 1998 [29] K. E. Mourelatou, A. T. Bouloutas, and M. E. Anagnostou, "An Approach to Identifying QoS Problems", Computer Communications 17, pp. 563-570, 1994 [30] Ehab Al-Shaer, "Hierarchical Filtering-based Monitoring Architecture for Large-Scale Distributed Systems", PhD Dissertation, Old Dominion University, July 1998 [31] URL: http://www.ethereal.com [32] URL: http://www.tcpdump.org [33] URL: http://windump.polito.it [34] URL: http://analyzer.polito.it [35] URL: http://www.networkchemistry.com/products/packetyzer/index.html [36] URL: http://www.fbi.gov [37] URL: http://www.acterna.com [38] URL: http://www.telchemy.com