You are on page 1of 30

Seventh Framework STREP No.

317846

D4.A
Public

Socially-aware Management of
New Overlay Application Traffic with
Energy Efficiency in the Internet
European Seventh Framework Project FP7-2012-ICT- 317846-STREP

Deliverable D4.A
(Annex to D4.1 and D4.2)
MUCAPS Prototype and Experimentation

The SmartenIT Consortium
Universität Zürich, UZH, Switzerland
Athens University of Economics and Business - Research Center, AUEB, Greece
Julius-Maximilians Universität Würzburg, UniWue, Germany
Technische Universität Darmstadt, TUD, Germany
Akademia Gorniczo-Hutnicza im. Stanislawa Staszica w Krakowie, AGH, Poland
Intracom SA Telecom Solutions, ICOM, Greece
Alcatel Lucent Bell Labs, ALBLF, France
Instytut Chemii Bioorganicznej PAN, PSNC, Poland
Interoute S.P.A, IRT, Italy
Telekom Deutschland GmbH, TDG, Germany

© Copyright 2015, the Members of the SmartenIT Consortium
For more information on this document or the SmartenIT project, please contact:
Prof. Dr. Burkhard Stiller
Universität Zürich, CSG@IFI
Binzmühlestrasse 14
CH—8050 Zürich
Switzerland
Phone: +41 44 635 4331
Fax: +41 44 635 6809
E-mail: info@smartenit.eu

Version 1.0

Page 1 of 30
© Copyright 2015, the Members of the SmartenIT Consortium

D4.A

Seventh Framework STREP No. 317846
Public

Document Control
Title:

D4.M MUCAPS Prototype and Experiments

Type:

Public

Editor(s):

Sabine Randriamasy, Frédéric Faucheux

E-mail:

sabine.randriamasy@alcatel-lucent.com, frederic.faucheux@alcatel-lucent.com

Author(s):

Sabine Randriamasy, Frédéric Faucheux

Doc ID:

D4.A-v1.0

AMENDMENT HISTORY
Version

Date

Author

Description/Comments

V0.1

2015/10/19

Sabine Randriamasy

First version

V0.2

2015/10/30

Frédéric Faucheux

Revised version after comments by Roman Lapacz and Jeremias
Blendin

V1.0

2015/10/30

Sabine Randriamasy, Frédéric
Faucheux

Completed version after fixing formal issues

Legal Notices
The information in this document is subject to change without notice.
The Members of the SmartenIT Consortium make no warranty of any kind with regard to this document,
including, but not limited to, the implied warranties of merchantability and fitness for a particular purpose. The
Members of the SmatenIT Consortium shall not be held liable for errors contained herein or direct, indirect,
special, incidental or consequential damages in connection with the furnishing, performance, or use of this
material.

Page 2 of 30

Version 1.0
© Copyright 2015, the Members of the SmartenIT Consortium

Seventh Framework STREP No. 317846

D4.A
Public

Table of Contents
1

Executive Summary .................................................................................................. 5

2

Introduction ............................................................................................................... 6

3

MUCAPS and VITALU in the SmartenIT Architecture............................................. 7
3.1
MUCAPS ..............................................................................................................................................7
3.1.1
Integrating MUCAPS with the SmartenIT architecture..................................................................8
3.1.2
MUCAPS implementation and execution: entities and interfaces .................................................9
3.2
Using VITALU to Evaluate MUCAPS .................................................................................................10

4

MUCAPS Prototype Design and Instrumentation ................................................. 11
4.1
MUCAPS Prototype Design................................................................................................................11
4.2
MUCAPS Prototype Set-up and Configuration...................................................................................11
4.2.1
TinyDNS......................................................................................................................................11
4.2.2
Service handling..........................................................................................................................12
4.2.3
MACAO Block .............................................................................................................................13
4.2.3.1 Newconfig_initial.cfg................................................................................................................13
4.2.3.2 Mappingconfig.cfg ...................................................................................................................13
4.2.3.3 user_endpoint_config.cfg ........................................................................................................13
4.2.3.4 Metric_weights.cfg...................................................................................................................13
4.2.3.5 Newconfig.cfg ..........................................................................................................................14
4.2.4
ALTO Server ...............................................................................................................................15
4.2.4.1 Network-map.txt ......................................................................................................................15
4.2.4.2 Pidcosts.txt ..............................................................................................................................15
4.3
Prototype Instrumentation for Evaluation and Assessment ...............................................................16
4.3.1
QoE performance measurement points and KPIs ......................................................................17
4.3.2
The VITALU QoE analyzer..........................................................................................................17

5

Experiments Definition and Set-up for MUCAPS.................................................. 19
5.1
MUCAPS Experiments .......................................................................................................................21
5.1.1
Functional tests on the DIG command........................................................................................21
5.1.1.1 Parameters, Measurements, and Metrics ...............................................................................22
5.1.1.2 Functionality test documentation .............................................................................................23
5.1.2
Performance tests with QoE analysis .........................................................................................24
5.1.2.1 Parameters, Measurements, and Metrics ...............................................................................24
5.2
MUCAPS Showcase ..........................................................................................................................24
5.2.1
MUCAPS Showcase Scenario 1 – Video Streaming on a High Resolution PC ..........................25
5.2.1.1 Scenario Topology...................................................................................................................25
5.2.1.2 Showcase Scenario.................................................................................................................26
5.2.2
MUCAPS Showcase Scenario 2 – Video Streaming on a Mobile Phone....................................26
5.2.2.1 Scenario Topology...................................................................................................................26
5.2.2.2 Showcase Scenario.................................................................................................................26

6

Summary and Conclusions .................................................................................... 28

7

References ............................................................................................................... 29

8

Abbreviations........................................................................................................... 30

Version 1.0

Page 3 of 30
© Copyright 2015, the Members of the SmartenIT Consortium

D4.A

Seventh Framework STREP No. 317846
Public

(This page is left blank intentionally.)

Page 4 of 30

Version 1.0
© Copyright 2015, the Members of the SmartenIT Consortium

Seventh Framework STREP No. 317846

D4.A
Public

1 Executive Summary
The SmartenIT project has selected two core mechanisms which are DTM and RBHORST for full prototyping and experimentation in WP4. These mechanisms provide a full
traffic management and transportation solution for the EFS and OFS scenarios. Multipartner specifications of these mechanisms and their implementations started in Q1 2014
within SmartenIT. In Q1 2015, it was decided to add MUCAPS to the achievements of
SmartenIT in an Annex deliverable. Contrary to DTM and RB-HORST, MUCAPS is not a
fully fleshed multi-partner developed mechanism, but was proposed as an add-on to
optimize application traffic management based on a transport network layer aware
paradigm that is being standardized at the IETF as the ALTO (Application Layer Traffic
Optimization) protocol, specified in RFC 7285. MUCAPS has been defined and prototyped
by ALBLF.
MUCAPS revises the initial Endpoints selection done by the application overlay by adding
awareness on the underlying network topology and on transport costs through information
not available by traditional end to end on-line measurement mechanisms. The selection
criteria used in MUCAPS are an abstraction of end-to-end performances on application
paths. This abstraction relies itself on an ISP-centric abstraction of the Internet topology,
provided to applications via the IETF ALTO protocol and one of its extensions supporting
several network cost metrics and constraints combining several logical operators and
metrics, see [1]. MUCAPS also integrates the access technology used by the device
receiving the application resource in its decision making.
Besides MUCAPS, ALBLF had been developing a QoE measurement and analysis tool
called VITALU that measures a set of QoE specific metric values at devices receiving
video files and devices and derives a Video Quality Score (VQS) built in a similar way to a
Mean Opinion Score. VITALU is in a pre-product stage and represents an effort of about 6
person years. VITALU has been customized within the SmartenIT project to support
measurements done on mobile phones and further analysis functions, candidates to the
QoE analysis entity pictured in the SmartenIT architecture.
The purpose of this annex is to complement, for MUCAPS, the test-bed definition provided
in D4.1 and experimentation definitions provided in D4.2 that were provided for the multipartner implemented solutions DTM and RB-HORST.

Version 1.0

Page 5 of 30
© Copyright 2015, the Members of the SmartenIT Consortium

D4.A

Seventh Framework STREP No. 317846
Public

2 Introduction
MUCAPS has been defined and prototyped by ALBLF as an add-on mechanism to further
optimize application traffic management based on a transport network layer aware
paradigm that is being standardized at the IETF as the ALTO (Application Layer Traffic
Optimization) protocol, specified in RFC 7285.
MUCAPS stands for "MUlti-Criteria Application endPoint Selection" and has been
specified in Section 4.11 of Deliverables D2.2 [4] and D2.4 [5]. MUCAPS addresses the
EFS scenario, in particular the use cases “Locality in UNaDa” and “Service and content
placement” for users. The MUCAPS added value to application traffic management and
SmartenIT is a further optimization of overlay application traffic. To this end, MUCAPS
revises the initial Endpoints selection done by the application overlay by adding
awareness on the underlying network topology and on transport costs through information
not available by traditional end to end on-line measurement mechanisms. The selection
criteria used in MUCAPS are an abstraction of end-to-end performances on application
paths. This abstraction relies itself on an ISP-centric abstraction of the Internet topology,
that is available to applications via the IETF ALTO protocol, documented in [1]. MUCAPS
leverages proposed ALTO protocol extensions, extending the base protocol set of ALTO
metrics and allowing ALTO transactions with multiple metrics and constraints combined
with multiple logical operators allowing richer compromises. So MUCAPS can help out any
SmartenIT mechanism that supports applications having multiple candidate resources
locations and that have no awareness of the underlying network.
The MUCAPS design integrates an extension to the ALTO protocol called Multi-Cost
ALTO that enables ALTO Clients to request path costs on several metrics jointly and also
allows clients to place constraints combining several metrics and logical operators. MultiCost ALTO is now an IETF ALTO WG item, see [1]. MUCAPS also integrates awareness
of the access technology used by a device receiving the application resource.
In the remainder of this document, section 2 complements D4.1 with the prototype set-up
and configuration for MUCAPS and section 3 complements D4.2 with the experiments
definition and Set-up.

Page 6 of 30

Version 1.0
© Copyright 2015, the Members of the SmartenIT Consortium

Seventh Framework STREP No. 317846

D4.A
Public

3 MUCAPS and VITALU in the SmartenIT Architecture
While MUCAPS main outline is presented first, the use of VITALU to evaluate MUCAPS is
described thereafter.

3.1

MUCAPS

The ALTO protocol is integrated in the SmartenIT paradigm: the ALTO Server stores and
provides abstracted transport network information to requesting ALTO Clients associated
to applications. However, the ALTO Client alone cannot automatically decide what
information to request and how to process it once received. MUCAPS builds the decision
making around the ALTO Client. It assists the choice of the location from which to
download content. Locations can be various entities including Video Server, Data Center,
uNaDa and Peers. Content covers video streams as well as computing resources for
virtualized or distributed applications. MUCAPS is suitable for applications having the
choice among several feasible Application Endpoints (AEP) and supports multi layer and
multi party incentive decision making by involving the following aspects in the AEP
selection:

Topology distance: number of hops (inter or intra AS), impacting delay,

Routing cost: reflecting the ISP policy in terms of AS peering agreements,

ISP preferences in terms of resources availability (e.g., path bandwidth).

An ALTO Client has no intelligence to choose the AEP selection metrics and the ranking
method, as ALTO is mainly a transport means for abstracted network information.
The initial selection of overlay Endpoints, referred to in MUCAPS as Application Endpoints
is usually produced by gathering functions such as DNS Servers and Clients, DHTs, Peer
Trackers and referred to in MUCAPS as Application EndPoint Gathering (AEPG)
functions. Figure 1 illustrates how MUCAPS is involved in a content downloading scenario,
as detailed in SmartenIT deliverables D2.2 [4] and D2.4 [5]. In this figure, the term
MACAO stands for Multi Access and Cost ALTO. Basically, the MACAO Client block does
(or performs) the following actions:

decides for which metrics an ALTO Client must request values,

decides which weight is applied to the metrics, and then

uses the obtained values to rank the AEPs.

MUCAPS comprises three parts: a MACAO Service Client (MARC) that requires from the
MACAO Service interface (MARS) an additional ranking of the initial AEP selection. The
MARC is hooked to an AEPG that may call it for enhanced ranking and hands it from it the
initial list of AEPs, identified in the current implementation, by their IP addresses. The
MARS is just an interface to the MACAO block that does the actual selection and ranking.
The MACAO block with its MARS is entirely deployed in the provider network. Finally, the
MARC and the MARS communicate via a simple IP Socket carrying a light message. The
message typically includes: a list of AEP IP addresses, the MUCAPS service ID pointing
here to “AEP Ranking” and the number of AEPs to rank.

Version 1.0

Page 7 of 30
© Copyright 2015, the Members of the SmartenIT Consortium

D4.A

Seventh Framework STREP No. 317846
Public

Figure 1 Involvement of MUCAPS components (MACAO, MARC and MARS) in a content
downloading scenario.
3.1.1 Integrating MUCAPS with the SmartenIT architecture
The impact of MUCAPS on the architecture is shown in Figure 2 with following
modifications:

The ALTO component in the SBox is replaced by the set {MACAO block, MARS
interface and ALTO Server}. The ALTO Client is now in the MACAO block.

There is no more need to have an ALTO Client in end systems involved in overlay,
that is: Data Centers, uNaDas and End User Entities. Instead, the ALTO Client
there is replaced by the MARC that sends to the MARS in the SBox, MUCAPS
requests, that is requests to re-order the candidate AEP list wrt ALTO based
network information.

The MARC is invoked by the Overlay Manager or the application of EUEs and
communicates with the MARS located in the SBox via a simple IP socket.

An alternative mapping of MUCAPS to the SmartenIT architecture is also described in
deliverable D3.3. It consists in replacing the ALTO (Client) component in the Overlay end
systems by the MARC, thus keeping the current interfaces established with
MUCAPS/ALTO replacing ALTO. This setting must however assume that the MARC can
be hooked with an AEPG function located in the Overlay Manager of the Data Center and
the uNaDa or in some appropriate place in the End User Entity.
The following SmartenIT architecture extensions are needed:

A placeholder is needed in the EUE for an AEPG that could call the MARC.

Interfaces should now be established between the MUCAPS/ALTO block and the
MARCs.

Page 8 of 30

Version 1.0
© Copyright 2015, the Members of the SmartenIT Consortium

Seventh Framework STREP No. 317846

D4.A
Public

Figure 2 Entities and components involved in MUCAPS.
3.1.2 MUCAPS implementation and execution: entities and interfaces
In the current MUCAPS deployment, the MARC is located in the ISP network. This is the
case for instance when the ISP wants to ensure that the DNS selection of the application
network is compliant to its policy in resources usage. In that case, the ISP captures the
response of its DNS Resolver, processes it with the MACAO module and sends the
MACAO selection to the DNS Resolver that sends it to the DNS Client. All this is done in a
transparent way for end users and applications.
As illustrated in Figure 3, the MACAO functional block has an external interface called
MARS for “MACAO Request handling Server” allowing it to communicate with a MARS
Client (called MARC) hooked to the AEPG.
The network-side deployment enables a lightweight transaction between MUCAPS and
the ISP DNS resolver by: adding to the MUCAPS block a MARS service interface,
becoming thus a MACAO Request Server (MARS). This deployment includes the following
steps:
1. The application client queries the DNS server for the list of content servers to
download/upload from.
2. The DNS Server,
2.1. selects the MACAO Service “Endpoint Ranking”

Version 1.0

Page 9 of 30
© Copyright 2015, the Members of the SmartenIT Consortium

D4.A

Seventh Framework STREP No. 317846
Public

2.2. hands to its embedded MARC its query result, that is the list of the candidate
Application Endpoint IP addresses, together with the IP address of the User
Endpoint, that is the DNS Client associated to the user device,
3. sends this light information to the MARS IF via an established IP Socket, as a query to
perform the multi-cost ranking of the AEPs,
4. The MUCAPS block performs the ranking and the MARS sends the reordered list of
AEPs to the MARC.
The MARC passes it to the AEPG/DNS Server that sends it to the requesting AEPG/DNS
Client.

Figure 3: MUCAPS entities and interfaces: MARC integrated in a network located AEPG,
here a DNS server.

3.2

Using VITALU to Evaluate MUCAPS

VITALU uses and analyses PCAP (Packet CAPture) messages either produced by tools
such as: wireshark, tshark, tcpdump, tpacketcapture, or that it captures by launching a
tshark instance, or gets on a non rooted Android mobile phone via tpacketcapture (with all
related issues…)
The procedure to follow is to use disjoint networks for media download and PCAP sending
(e.g. WiFi for video and Ethernet for PCAP sending ) to avoid interferences between pcap
and media flow, or simply (and probably better) to wait until the end of the media flow and
then transfer the captured PCAP to the VITALU machine for analysis.

Page 10 of 30

Version 1.0
© Copyright 2015, the Members of the SmartenIT Consortium

Seventh Framework STREP No. 317846

D4.A
Public

4 MUCAPS Prototype Design and Instrumentation
The MUCAPS test-bed has been designed to:

Evaluate the ability of MUCAPS to revise the initial AEP selection whenever it was
appropriate,

Evaluate the impact of the MUCAPS selection on the QoE.

A MUCAPS intervention is qualified as appropriate whenever the initial AEP selection
does not optimize ISP costs and other metrics that impact the application QoE. The
applicable ISP is the one hosting the UEP. The other metric considered is a bandwidth
score, rating the bandwidth performance of the end to end path between an AEP and the
UEP. All metrics and their values are assumed to be set by the ISP.
In the present evaluation, the initial AEP selection is assumed to be done by the ISP DNS
resolver.

4.1

MUCAPS Prototype Design

The design of the prototype for MUCAPS with the involved hardware is shown in Figure 4.

Figure 4 MUCAPS prototype and involved hardware

4.2

MUCAPS Prototype Set-up and Configuration

4.2.1 TinyDNS
TinyDNS, a part of djbdns software package, is a popular open source DNS server.
It has been modified in order to include the support of Macao API.

Version 1.0

Page 11 of 30
© Copyright 2015, the Members of the SmartenIT Consortium

D4.A

Seventh Framework STREP No. 317846
Public

In order to install the customized version of tinydns, the following command must be run:
cd /home/alto/workspace/tinydns
./install-tinydns-alto.sh
tinydns-conf tinydns dnslog /etc/tinydns @IP_DNS_Server

where @IP_DNS_Server: represents the IP address of DNS Server or of host.
The next step is to fill the tinydns database with the IPs of the different video servers. This
can be done using the following commands:
./add-ns alto 10.201.65.165

# add dns server

./add-host video.alto 10.201.70.1
./add-host video.alto 10.201.70.2
./add-host video.alto 10.201.170.133
make # recompilation to create file data.cdb

The first commands modify a file called /etc/tinydns/root/data that contains the
link between the IP addresses and the domain names. It can also be modified manually.
Here is an example of the data file:
.alto:10.201.65.165:a:259200
=video.alto:10.201.70.1:10
=video.alto:10.201.70.2:10
=video.alto:10.201.170.133:10
=server1.alto:10.201.70.1:10
=server2.alto:10.201.70.2:10
=server3.alto:10.201.170.133:10

The last three entries have been added in order to be able to address the three servers
individually. After the data file is modified, the tinydns-data command should be run in
order to validate the changes in the running tinydns server.
Another file to consider is /etc/tinydns/env/IP. It contains the IP address at which the
tinydns server listens. If it is set to a specific IP (like 10.201.65.165 in the example) it will
only listen from this address. As the server is using a different IP address for LAN access,
Wi-Fi and Cellular, SmartenIT needs the tinydns server to be able to listen to all of these
addresses. The simpler way to achieve this behavior is to put the IP address 0.0.0.0 in
the file /etc/tinydns/env/IP. By doing this tinydns will answer dns queries coming
from any network interface. Tinydns needs to be restarted after changing this file.
4.2.2 Service handling
In order to automate the launching of tinydns it can be used as a linux service. In order to
achieve this the following commands can be issued:
ln -s /etc/tinydns /service/
sudo svscan /service &

Page 12 of 30

Version 1.0
© Copyright 2015, the Members of the SmartenIT Consortium

Seventh Framework STREP No. 317846

D4.A
Public

svc -u /service/tinydns
svstat /service/*
netstat -anpe | grep 53 # check that dns service is launched on
port 53.

4.2.3 MACAO Block
When no MACAO server is launched, the tinydns server returns the results directly from
its internal database. When the server is launched, the tinydns transmits the results from
the internal database to the MACAO block so that it sorts the results depending on the
configuration.
In order to install and run the server, the following commands should be run in root mode
cd ~/workspace/MACAO2012/ALTO-CLIENT-BLOCK/ALTO_Application_server/
unset http_proxy
./server

# remove proxy if there is.

# launch the MACAO block

The Macao block reads the following configuration files.
4.2.3.1 Newconfig_initial.cfg
This file indicates is ALTO should be used or not and if the Automatic mode is enabled or
not.
Status = 1; # 0=ALTO OFF, 1=ALTO ON
Metric_Automatic = 1; # 0=AUTO mode OFF, 1=AUTO mode ON

4.2.3.2 Mappingconfig.cfg
This file stores the IP addresses of the destination endpoint and identifies the type of
server (video, audio or other) depending of its IP address :
vidservers = [ "10.201.170.133", "10.201.70.101", "10.201.70.102"
];
audservers = [ "10.201.70.5", "10.201.70.6", "10.201.70.7" ];
otherservers = [ "10.201.80.8", "10.201.180.8", "10.201.222.2" ];

4.2.3.3 user_endpoint_config.cfg
This file is used to recognize on which link the client IP address (i.e. the address issuing
the DNS request) is connected.
An example of its content is the following:
LAN = [ "10.201.70.102", "10.201.70.103", "10.201.70.14" ];
WiFi = [ "10.201.70.52", "192.168.1.145", "192.168.1.104" ];
Cellular = [ "10.201.70.10", "192.168.42.208" ];

4.2.3.4 Metric_weights.cfg
This file contains the weights associated to each metric that can be considered, and for
each of the network interfaces.

Version 1.0

Page 13 of 30
© Copyright 2015, the Members of the SmartenIT Consortium

D4.A

Seventh Framework STREP No. 317846
Public

The syntax is as follows:
LAN :
{
routingcost = 1.0;
hopcount = 1.0;
propadelay = 1.0;
bandwidth = 1.0;
};
WiFi :
{
routingcost =2.3;
hopcount = 4.0;
propadelay = 1.0;
bandwidth = 1.0;
};
cellular :
{
routingcost = 3.0;
hopcount = 2.0;
propadelay = 3.0;
bandwidth = 3.0;
};

4.2.3.5 Newconfig.cfg
This file is used to modify the weights of the different metrics depending on the chosen
configuration (Routing Cost, Routing Cost + BW Cost).
Its syntax is the following
nummetrics = 2; # number of metrics to use
elements = (
{
costtype = 1;
costmode = 1;
weights = 1.0;
},
{
costtype = 2;
costmode = 1;
weights = 1.0;
} );
Page 14 of 30

Version 1.0
© Copyright 2015, the Members of the SmartenIT Consortium

Seventh Framework STREP No. 317846

D4.A
Public

service = 1;
topN = 0;

4.2.4 ALTO Server
The ALTO server answers to the MACAO requests by giving the costs of the different
metrics for the source and destination IPs considered.
The software was originally designed to use Routing Cost and Hop Count as metrics, but
the decision was later changed to Routing Cost and Bandwidth Score. However, the
software was not updated and the name “hopcount” is still used in configuration files, but
is now used to represent Bandwidth Score.
4.2.4.1 Network-map.txt
This file allows identifying the equipments access type by their IP address.
The sample file used during the experiments is the following:
###### UEP-PIDS ######
## LAN
mypidLAN

10.201.65.165/18

## WIFI
mypidWiFi

192.168.1.0/24

## 3G
mypid3G

192.168.42.0/24

###### AEP-PIDS ######
mypid101

10.201.70.1/32

mypid102

10.201.70.2/32

mypid133

10.201.170.133/32

4.2.4.2 Pidcosts.txt
This file defines the costs between two equipments depending on the access type.
An example of use is the following:
##### COSTS for routingcost #####
type=routingcost
setall=999
## LAN
mypidLAN,mypid101,7
mypidLAN,mypid102,11
mypidLAN,mypid133,60
## WIFI
mypidWiFi,mypid101,7

Version 1.0

Page 15 of 30
© Copyright 2015, the Members of the SmartenIT Consortium

D4.A

Seventh Framework STREP No. 317846
Public

mypidWiFi,mypid102,11
mypidWiFi,mypid133,60
## 3G
mypid3G,mypid101,7
mypid3G,mypid102,11
mypid3G,mypid133,60
##### COSTS for hopcount/BWScore #####
type=hopcount
setall=1
## LAN
mypidLAN,mypid101,8
mypidLAN,mypid102,18
mypidLAN,mypid133,20
## WIFI
mypidWiFi,mypid101,8
mypidWiFi,mypid102,18
mypidWiFi,mypid133,20
## 3G
mypid3G,mypid101,8
mypid3G,mypid102,18
mypid3G,mypid133,20
defaultpid,defaultpid,0

4.3

Prototype Instrumentation for Evaluation and Assessment

MUCAPS aims at jointly optimizing ISP transport costs and path bandwidth usage while
maintaining or improving end user QoE. To do so, MUCAPS must

Appropriately choose the AEP selection metrics wrt the type of application,

Use the ISP defined values on the selection metrics, stored in the ISP managed ALTO
Server,

Appropriately set the selection metric weights according to the access type of the UEP,

Rank the candidate AEPs according to the above cited elements

Once the UEP connects to the AEP selected with the MUCAPS support, the resulting
impact must be measured in terms of

User QoE as measured by a fully fleshed QoE measurement and analysis tool,

Theoretical MUCAPS inferred ISP costs based in the ISP ALTO information

Page 16 of 30

Version 1.0
© Copyright 2015, the Members of the SmartenIT Consortium

Seventh Framework STREP No. 317846

D4.A
Public

4.3.1 QoE performance measurement points and KPIs
The following key performance indicators are considered to evaluate the performance of
MUCAPS. The measurement points are summarized in Table 1.
Table 1 Key performance indicators and measurement points for MUCAPS
KPI
Cross-Layer
(CLU)

End user mobile
devices

End-user PC

Utility

UEP
X

Video Quality Score

Directed to PC

X

Start time delay

Directed to PC

X

NFRz, DFRz (number Directed to PC
and
duration
of
freezes

X

Media Bit Rate

X

Directed to PC

The Cross-Layer Utility (CLU) is the scalar value that MUCAPS outputs on which its final
ranking is based. It can be measured at the UEP or any device that receives the MUCAPS
response information, including ALTO responses and MUCAPS vector based ranking. It is
a theoretical value that assesses the MUCAPS influence on the AEP choice.
The rest of the KPIs are measurements, done by the VITALU QoE analyzer on PCAP files
associated to video streams.
4.3.2 The VITALU QoE analyzer
VITALU has been developed in ALBLF and transferred to an Alcatel-Lucent Business
Unit. The primary goal of the tool is to evaluate/monitor the QoE for root cause of
multimedia delivery issues. VITALU handles multiple streaming protocols such as HTTP
Live, RTSP, RTP Multicast, or Flash HTTP. VITALU is part of service offer portfolio for
operator customers.
VITALU has the following main functions for each monitored video flow:

Collect/Analyze IP network info (Packet Lost, Jitter, Delay, Timestamp, Sequence
number),

Compute video quality metrics ( Video Score, Freezes, Frame Errors)

Provide frame by frame analysis, with jitter buffer simulation for RTP protocol

The metrics measured by VITALU include: DateTime Capture (s), Date, Clip Name,
Resolution, Operator, Session ID, RTT (Avg, Std, Max), Video Access Time (s), Video
Duration (s), Video Bitrate (bps), VQS Avg (Real), VQS Avg (Theor.), VQS Std (Real),
VQS Std (Theor.), Frame Error (%), Freeze (%), Nb of Freezes, Freeze Duration (Avg) (s),
Freeze Duration (Max) (s), Nb of Error Frames, Nb. of Packets, Packets Retransmitted
(%), Nw Video Jitter (Avg) (s), Nw Video Jitter (Max) (s), Nw Bitrate (Avg) (bps), Nw Bitrate
(Std) (bps), Late & Err. Frame Ratio (%), Frame Rate (Avg) (fps), Frame Rate (Std) (fps),
Nb of Frame Resolutions, Frame Resolution (px).
Version 1.0

Page 17 of 30
© Copyright 2015, the Members of the SmartenIT Consortium

D4.A

Seventh Framework STREP No. 317846
Public

SmartenIT uses 6 metrics derived from the above measurements list which are: Media Bit
Rate, Start Time, RTT, VQS, Freeze Nb, Freeze Duration (avg), Freeze Duration (max).
VITALU is also able to evaluate these metrics on partially captured streams, if the source
video file is provided.
For this experiment the VITALU tool was used to automatically assess the file captured on
the UEP (after transmission of the captured file to the VITALU server). The results are
then stored to a .csv file that can be analyzed offline.

Page 18 of 30

Version 1.0
© Copyright 2015, the Members of the SmartenIT Consortium

Seventh Framework STREP No. 317846

D4.A
Public

5 Experiments Definition and Set-up for MUCAPS
The end-user focused (EFS) scenario aims at providing increased QoE and ISP costs and
network resources efficiency. In particular, this goal is achieved by applying an in-network
optimization of application endpoint selection that takes interests of both ISPs and
application resources consumers into account. In this context, MUCAPS revises the initial
endpoint selection done by applications by directly involving of ISP-defined network costs
and performance rating together with the access-type of the end-user device in its final
decision. The MUCAPS intervention is transparent to the user, as the functionality is
embedded in the ISP network. In the SmartenIT use case, MUCAPS is hooked to the ISP
DNS resolver.
The EFS experiments type for MUCAPS will be: prototype validation. Their goal is to
validate its set of functionalities, evaluate its performances and benefits to end-users and
ISPs on a set of sample configurations.

Functional tests will validate the ability of the VLC video reader to integrate MUCAPS
when searching the video, the ability of MUCAPS to catch context information like End
User Endpoint (UEP) access, Application Endpoint (AEP) localization by tinyDNS,
establish a connection with the ALTO Server and the MARS. All this for three UEP
access types: LAN/FTTH, WiFi/ADSL, 3G

The performance evaluation will be done by the QoE analyzer VITALU developed by
ALBLF. During a video stream, TcpDump (or other network analyzer software such as
Wireshark) will be used to capture the PCAP files.

Figure 5 shows the topology of the MUCAPS experiments. Figure 6 shows this topology
mapped to the prototyping environment.

Figure 5 Topology of the MUCAPS prototype experiments

Version 1.0

Page 19 of 30
© Copyright 2015, the Members of the SmartenIT Consortium

D4.A

Seventh Framework STREP No. 317846
Public

Figure 6 MUCAPS topology mapped to the evaluation environment
Two types of experiments are being conducted

Functional tests by:
o Launching a “dig” command on the UEP, to validate the prototype,
o Leading a video with VLC (for human visual assessment) or WGET (for automated
tests) on an end-user system that is either a PC or a mobile phone, to see
whether the MUCAPS processing was launched via VLC or WGET.

Impact tests by:
o Capturing the PCAP files generated by TCPDUMP of Wireshark,
o Computing and analyzing the QoE metric values and resulting VQS.

For LAN tests, a web proxy had to be used in order to be able to access external video
servers on the internet. In this case the name resolution was done on the web proxy
server and SmartenIT had to change its configuration so it uses our tinydns server.
For Wi-Fi tests, the PCs had a direct internet connection.
For 3G tests, the mobile phone was used as a usb-modem for the PC hosting the video
player: this was done because SmartenIT has to use a specific DNS server and that is not
possible with an unrooted smartphone. The link to the internet was through the 3G/4G
access of the smartphone but the video player and DNS resolution was performed on a
linux PC.
Some of the problems are linked to the use of a fake name server for our experiments. In
a near future ALBLF plans to use a real name server in order to be able to use the default
DNS configuration and be able to use it directly on unrooted smartphone for example.
In addition, the following Table 2 provides the hosts list and the services used to run the
MUCAPS experiments.
Page 20 of 30

Version 1.0
© Copyright 2015, the Members of the SmartenIT Consortium

Seventh Framework STREP No. 317846

D4.A
Public

Table 2 Hosts and the services running on them for MUCAPS experiments
Host

Services

Comment

isp1-PC1

UEP receiving the video on a PC

isp1-Phone

UEP receiving a video on a mobile phone

isp1-PC2

tinyDNS+MACAO

isp1-PC3

ALTO Server

isp1-PC4

VITALU QoE Analyzer

isp1-AEP1

Video Server 1 in ISP1

high BW Score, moderate
Routing Cost

isp1-AEP2

Video Server 2 in ISP1

Low BW Score, low Routing
Cost

isp2-AEP3

Private Video Server 3 in ISP2

High BW Score, high Routing
Cost

Table 3 provides the IP addresses of the Hosts listed in Table 2 that are involved in
MUCAPS-assisted video streaming. The QoE analyzer is not listed as it performs off line
analysis. AEP3 is a private server that only has a public address, which will for
confidentiality not be given in the SmartenIT documentation. Addresses of hosts other
than LAN addressed are partially masked for the same reasons.
Table 3 Hosts involved in the MUCAPS experiments and their addresses
Host
PC1-UEP

LAN

VPN

PUBLIC

LAN 172.alu.vpn.uep
WiFi 192.168.pc1.uep

Phone-UEP

3G 1982.198.mph.uep

PC2-DNS+MACAO

172.alu.vpn.165

PC3-AOS

172.alu.vpn.153

AEP1

10.201.70.1

172.alu.vpn.142

sp1.pub.srv.113

AEP2

10.201.70.2

172.alu.vpn.51

sp1.pub.srv.111
spB.prv.srv.37

AEP3

Table 4 provides the ISP defined metric costs, for each AEP class.

5.1

MUCAPS Experiments

The following tests are defined to validate functionality and estimate performance.
5.1.1 Functional tests on the DIG command
The dig command performs a DNS name resolution taking as input the domain name from
the URI of the requested video, composed of the domain name and video file name.

Version 1.0

Page 21 of 30
© Copyright 2015, the Members of the SmartenIT Consortium

D4.A

Seventh Framework STREP No. 317846
Public

The dig command shows how the name resolution is performed by the VLC reader when
the URI of the requested video is entered, using a system library function such as
gethostbyname.
On a PC, it can be launched as a standalone command with optionally the address of the
desired DNS resolver.
The functional tests consist in launching a dig command on a PC and on a VLC reader
and verify the following elements:

Response with MUCAPS “OFF”,

Response with MUCAPS “ON” and both selection metrics RC and BW activated, for all
3 access types.

The functionality tests ensure that the MUCAPS functionality works as expected when the
dig command is triggered by VLC.
The assumption to see the MUCAPS changing the initial selection is that the initial
endpoint selection does not jointly optimize the ISP routing cost and path bandwidth
performance.
5.1.1.1 Parameters, Measurements, and Metrics
ALTO Values for the 3 AEPS
The AEPs have been mapped to 3 classes depending on their end2end path
performances on RC and BWS that is specified in the ALTO Server. MUCAPS performs a
multi-objective vector based ranking of the AEPs, which is more reliable. To better
illustrate the abilities of MUCAPS to do so, the 3 classes of AEP are built so as to be
mutually non-dominated in terms of performances. The RC and BWS values on the AEPs
used for the experiments are the values of the AEP classes utilized for the theoretical
evaluation described in D2.5. The three AEP classes are characterized as follows:

AEP1 – Class C1: moderate cost, high path bandwidth: RC = 11, BWS = 18

AEP2 – Class C2: low cost, low path bandwidth: RC = 7, BWS = 8

AEP3 – Class C3: high cost, high path bandwidth: RC = 60, BWS = 20

Setting ALTO BWS and RC values
Although ALTO values are supposed to be specified by the ISP, the BWS values are set
so as to be mapped to the downlink rates measured in France with different access types.
Sources [2] and [3] provide access-dependent value ranges (min-max) for e2e downlink
path bandwidth, based on measurements performed from 2013 to today in France.
SmartenIT, thus, considers the following ranges and access types:

FTTH [45-160] Mb/s

ADSL [6-9] Mb/s

Cell 3G: [4-7] Mb/s,

In the AOS used by MUCAPS, BWS is a score with values in [1, 20]. In order to mitigate to
effect of FTTH (Fiber-to-the-Home) performance, the mapping is done so as to mitigate
the effect of FTTH performance. This approach took FTTX as a mixture of FTTH, FTTB
(Fiber-to-the-Basement), and TTS (Fiber-to-the-Street).
Page 22 of 30

Version 1.0
© Copyright 2015, the Members of the SmartenIT Consortium

Seventh Framework STREP No. 317846

D4.A
Public

Likewise, RC is completely dependent of the ISP policy. SmartenIT nevertheless assumes
that RC may be influenced by factors such as OPEX and peering agreements.
The resulting mapping was, therefore, adopted as a basis for example AOS values.
Table 4: Typical values for metrics depending on access type
ACCESS

BW Range

BWS median

RC basis

Cell 3G

[4-7] Mbs

6.7

15

WIFI/ADSL

[4-9] Mbs

7.2

9

FTTX

[45-140+] Mb

17.5

11

The ALTO values of the AEPs have, therefore, been chosen as defined in Table 5.
Table 5: ALTO values for each AEP
RC

BWS

AEP1

11

18

AEP2

7

8

AEP3

60

20

Finally, Table 6 recalls weights associated to those metrics depending on the UEP access
type.
Table 6: Weights of metrics depending on access type
FTTX

WiFi/ADSL

3G

RC

1

2.3

3

BWS

1

4

2

5.1.1.2 Functionality test documentation
The results of functional tests on the dig command will be documented in 2 different ways:

A screen copy of the result of a dig command will be provided, as an illustration

Results of dig commands will be provided in terms of RC, BWS and CLU value
associated to the AEP selected in the different modalities of MUCAPS activation.

MUCAPS test modalities: for each UEP access type
o G1: MUCAPS OFF
o G2: MUCAPS ON + RC
o G3: MUCAPS ON + BWS
o G4: MUCAPS ON + (RC, BWS)

Version 1.0

Page 23 of 30
© Copyright 2015, the Members of the SmartenIT Consortium

D4.A

Seventh Framework STREP No. 317846
Public

5.1.2 Performance tests with QoE analysis
The performance tests quantify the impact of MUCAPS on the QoE
5.1.2.1 Parameters, Measurements, and Metrics
Deployment Infrastructure
The deployment infrastructure is the one depicted in Figure 6. The location of the different
elements is as follows:

The ALTO Server (AOS), AEP1, AEP2 are located in the ALBLF premises,

The UEP and MACAO blocs are located in remote premises of the Paris area and
connect to AOS, AEP1 and AEP2 via a VPN

Server AEP3 is a private server belonging to another ISP

Assuming the initial selection is AEP3 the tests groups have been defined as follows.

Video sessions (2 types wrt length, resolution, internals such as movements).

With UEP in different access types: LAN/FTTX, wifi/ADSL, cellular 3G

With different devices (PC, Phone)

AEP types: video « public » server, « private » server

QoE Measurements with VITALU:

On each video session: the QoE metrics will be measured

MUCAPS test modalities: for each video class and each UEP access type, the tests
will only consider:

G1: MUCAPS OFF

G4: MUCAPS ON + (routingcost, BWscore)

For each test group Gi, the QoE performance is measured.
The QoE results are compared and impact is derived.

5.2

MUCAPS Showcase

MUCAPS can be showcased following the 2 scenarios proposed below: Scenario 1 as
“Video Streaming on a High Resolution PC” and scenario 2 as “Video Streaming on a
Mobile Phone”.
When the ALBLF VPN is accessible to the UEP, the experimental topology as described
in Figure 7 is suitable.
If the ALBLF VPN is not accessible to the UEP and to minimize the involved hardware
MUCAPS can be showcased using the topology depicted in Figure 8.
The showcase topology could even be reduced to having PC1 and PC2 merged so as to
host the whole (UEP, tinyDNS, MACAO, AOS) set as all its elements can be assumed to
belong to the same ISP.

Page 24 of 30

Version 1.0
© Copyright 2015, the Members of the SmartenIT Consortium

Seventh Framework STREP No. 317846

D4.A
Public

Figure 7 Initial MUCAPS showcase topology

Video
Server
AEP3/ISP2

VM

VM

tinyDNS
resolver

MACAO
VM

PC2/ISP1

ALTO Server

eth

eth

WLAN

Video
Server
AEP1/ISP1

eth

Video
Server
AEP2/ISP1

UEP
VLC
PC 1/ISP1

VLC
PHONE

RAN

Figure 8 MUCAPS showcase topology when the UEP is not accessing its VPN
5.2.1 MUCAPS Showcase Scenario 1 – Video Streaming on a High Resolution PC
5.2.1.1 Scenario Topology
The MUCAPS showcase topology on LAN is illustrated in Figure 9.

Version 1.0

Page 25 of 30
© Copyright 2015, the Members of the SmartenIT Consortium

D4.A

Seventh Framework STREP No. 317846
Public

Figure 9 MUCAPS showcase topology on LAN
5.2.1.2 Showcase Scenario
This scenario goes through the following steps
1. UEP on a PC requests a video V1 without MUCAPS
a. DNS selects AEP3
b. Transfer at high routing cost (RC), high path BW score (BWS)
2. UEP requests V1 with DNS assisted by MUCAPS+RC
a. DNS-MACAO points to AEP2
b. Transfer at low cost but low quality
3. UEP now requests V1 with DNS assisted by MUCAPS + (RC, BW)
a. DNS-MACAO points to AEP1
b. Transfer at moderate RC AND high BWS
5.2.2 MUCAPS Showcase Scenario 2 – Video Streaming on a Mobile Phone
5.2.2.1 Scenario Topology
The MUCAPS showcase topology on 3G is illustrated in Figure 10.
5.2.2.2 Showcase Scenario
This scenario goes through the following steps and follows showcase of scenario 1.
1. UEP on a mobile phone gets V1 on AEP1
2. It gets a bad QoE
a. because the mobile phone cannot sustain the HR bitrate
3. UEP requests video V1 with DNS assisted by MUCAPS + (RC, BW)
a. DNS-MACAO takes 3G-access type into account and points to AEP3
b. UEP gets a good QoE because AEP3 delivers at a sustainable bitrate
c. Transfer at low RC

Page 26 of 30

Version 1.0
© Copyright 2015, the Members of the SmartenIT Consortium

Seventh Framework STREP No. 317846

D4.A
Public

Figure 10 MUCAPS showcase topology on 3G

Version 1.0

Page 27 of 30
© Copyright 2015, the Members of the SmartenIT Consortium

D4.A

Seventh Framework STREP No. 317846
Public

6 Summary and Conclusions
This deliverable complements the SmartenIT Deliverables D4.1 and D4.2 by adding the
respective details of the MUCAPS mechanism. In particular, this annex here

Details how MUCAPS can be integrated in the SmartenIT Architecture

Presents the MUCAPS prototype set-up and how it is configured during the showcase

Analyses the results of experiments

Defines the main showcase scenarios.

The main outcomes from these experiments is that it was possible to build a prototype that
is able to automatically direct an user to the best suiting server depending on the type of
service requested (e.g. video, audio, web), and the type of access of the user (LAN, Wi-Fi,
3G). The use of an offline analyzer also shows an improvement in the quality experienced
by the user thanks to MUCAPS mechanism rather than using static routing tables.
The 2 different showcase scenarios help to understand different configurations possible
with MUCAPS, taking into account different metrics and showing which choice will give the
best result. Each of the metrics is attributed a different weight depending on the chosen
access type, hence providing different results. The showcase, thus, allows to achieve the
impact of each of the metrics and of the access type on the chosen server for a specific
type. The selected application for the demo is video streaming, as the different quality
levels can be easily distinguished and show that the choice of the server will impact the
quality experienced by the user.
In conclusion, thanks to the MUCAPS prototype used in the experiments and showcases,
the benefits linked to this mechanism can be evaluated both objectively and subjectively.
The deployment of this mechanism can bring advantages inside the SmartenIT solution or
independently and proves to be a very good solution in terms of network performance and
user experience.

Page 28 of 30

Version 1.0
© Copyright 2015, the Members of the SmartenIT Consortium

Seventh Framework STREP No. 317846

D4.A
Public

7 References
[1]

draft-ieft-alto-multi-cost-01: “Multi-Cost ALTO”, (work in progress) S. Randriamasy
and W. Roome and N. Schwan, October 19th 2015, https://tools.ietf.org/html/draft-ietfalto-multi-cost-01

[2]

"Baromètre des connexions Internet mobiles en France métropolitaine Deuxième
trimestre
2015
",
nPerf
report,
3
Juillet
2015,
http://media.nperf.com/files/publications/2015-07-03_Barometre-connexions-mobilesnPerf-2015-T2.pdf

[3]

"Observatoire
des
débits
Internet",
(1/05/2013
http://www.ariase.com/fr/vitesse/observatoire-debits.html

[4]

The SmartenIT project: Deliverable D2.2: Report on Definitions of Traffic
Management Mechanisms and Initial Evaluation Results; October 2013.

[5]

The SmartenIT project: Deliverable D2.4: Report on Final specifications of Traffic
Management Mechanisms and Evaluation Results; October 2014.

Version 1.0

to

09/2015),

Page 29 of 30
© Copyright 2015, the Members of the SmartenIT Consortium

D4.A

Seventh Framework STREP No. 317846
Public

8 Abbreviations
ADSL

Asymmetric Digital Subscriber Line

AEP

Application Endpoint

AEPG

Application Endpoint Gathering

ALTO

Application Layer Traffic Optimization

AOS

ALTO Server

AS

Autonomous System

BWS

BandWidth Score

CLU

Cross-Layer Utility

DTM

Dynamic Traffic Management

EFS

End-user Focused Scenario

EUE

End User Entity

FTTB

Fiber-to-the-Basement

FTTH

Fiber-to-the-Home

FTTS

Fiber-to-the-Street

FTTX

Fiber-to-the-X

IETF

Internet Engineering Task Force

ISP

Internet Service Provider

MACAO

Multi Access and Cost ALTO

MARC

MACAO Service Client

MARS

MACAO Service interface

MUCAPS

MUlti-Criteria Application endPoint Selection

OFS

Operator Focused Scenario

OPEX

Operational Expenditure

PCAP

Packet CAPture

QoE

Quality-of-Experience

QoS

Quality-of-Service

RC

Routing Cost

RFC

Request for Comments

SmartenIT

Socially-aware Management of New Overlay Application Traffic with Energy
Efficiency in the Internet

UEP

End User Endpoint

uNaDa

User Nano Data Center

VQS

Video Quality Score

Page 30 of 30

Version 1.0
© Copyright 2015, the Members of the SmartenIT Consortium