Deliverable D1.1 Report On Stakeholders Characterization and Traffic Characteristics

Seventh Framework STREP No. 317846 D1.
1 - Report on Stakeholder Relationships

and Traffic Characteristics
Public
Version 1.4 Page 1 of 161
Copyright 2013, the Members of the SmartenIT Consortium

Socially-aware Management of
New Overlay Application Traffic with
Energy Efficiency in the Internet
European Seventh Framework Project FP7-2012-ICT- 317846-STREP

Deliverable D1.1
Report on Stakeholders Characterization

The SmartenIT Consortium

Universitt Zrich, UZH, Switzerland
Athens University of Economics and Business - Research Centre, AUEB, Greece
Julius-Maximilians Universitt Wrzburg, UniWue, Germany
Technische Universitt Darmstadt, TUD, Germany
Akademia Gorniczo-Hutnicza im. Stanislawa Staszica W Krakowie, AGH, Poland
Intracom SA Telecom Solutions, ICOM, Greece
Alcatel Lucent Bell Labs, ALBLF, France
Instytut Chemii Bioorganiicznej Pan, PSNC, Poland
Interoute S.P.A, IRT, Italy
Telekom Deutschland GmbH, TDG, Germany


For more information on this document or the SmartenIT project, please contact:

Prof. Dr. Burkhard Stiller
Universitt Zrich, CSG@IFI
Binzmhlestrasse 14
CH8050 Zrich
Switzerland

Phone: +41 44 635 4331
Fax: +41 44 635 6809
E-mail: info@smartenit.eu

D1.1 - Report on Stakeholder Relationships Seventh Framework STREP No. 317846
Public
Page 2 of 161 Version 1.4

Document Control

Title: Report on Stakeholders Characterization and Traffic Characteristics
Type: Public
Editor(s): Ioanna Papafili, George D. Stamoulis
E-mail: iopapafi@aueb.gr, gstamoul@aueb.gr
Author(s): Matteo Biancani, Manos Dramitinos, Gerhard Hasslinger, David Hausheer,
Tobias Hossfeld, Roman Lapacz, Frank Lehrieder, Guiherme Machado,
Ioanna Papafili, George Petropoulos, Alessandro Predieri, Julius Rueckert,
Grzegorz Rzym, Sergios Soursos, Spiros Spirou, George Stamoulis,
Burkhard Stiller, Christos Tsiaras, Krzysztof Wajda, Matthias Wichtlhuber,
Mateusz Wielgosz, Arkadiusz Zwierz
Doc ID: D1.1-v1.4.doc

AMENDMENT HISTORY

Version Date Author Description/Comments
V0.1 November 1, 2012 Burkhard Stiller First version
V0.2 November 13,
2012
Ioanna, ALL ToC proposal
V0.3 November 14,
2012
Ioanna, ALL ToC update, work allocation
V0.4 November 20,
2012
Ioanna, David ToC update
V0.5 December 04,
2012
Roman, George P., Sergios, Spiros,
Christos, Krzysztof, Frank, Tobias,
Ioanna, George D., Matteo, Julius,
David
Input in bullet form
V0.6 December 05,
2012
Ioanna, Christos, Guiherme New structure and updated input by UZH
V0.7 December 19,
2012
Gerhard, Frank, Tobias, Julius,
Matthias, David, Christos, Alessandro,
Roman, Matteo, George P. Sergios,
Spiros, Ioanna, Manos, George S.
Input in text form; new structure
V0.7.2 January 03, 2013 Ioanna Interim version; comments for expected input in each section; structure
update.
V0.7.8 January 31, 2013 Julius, Matthias, David, Christos,
Guiherme, Alessandro, Roman, Matteo,
George P., Sergios, Ioanna, Manos,
George S.
Input in text form from all partners integrated - MS1
V0.8 February 22, 2013 George S., Manos, Ioanna Comments to all involved partners
V0.8.5 March 08, 2013 Julius, Matthias, David, Christos,
George S.
Address comments; integrate new input
V0.9 March 15, 2013 Julius, Matthias, David, Christos,
George S., Gerhard
Address comments; integrate new input
V0.9.5 March 26, 2013 Matteo, Chris, Roman, Gerhard, Ioanna,
Guiherme, Thomas, Manos, George P.
Address comments, provide new input; integrate new input
V0.9.6 March 28, 2013 Krzysztof, Grzegorz, Mateusz,
Arkadiusz, Guiherme, Thomas, Julius,
Updated input.
Seventh Framework STREP No. 317846 D1.1 - Report on Stakeholder Relationships
Public

Ioanna
V0.9.7 April 3, 2013 Matteo, Sabine (internal reviewers) Preliminary comments
V1.0.2 April 8, 2013 Ioanna, George S. Address preliminary comments by reviewers; send final comments to
authors
V1.0.3 April 10, 2013 Ioanna, George S. Latest version sent to reviewers
V1.0.4 April 17, 2013 Gino, Sabine Final comments to editors
V1.0.5 April 19, 2013 Tobias (WP1 leader), George S. Comments upon reviewers' comments
V1.1 April 22, 2013 Ioanna, George S. Address part of reviewers' comments; send integrated version to
authors to finalize contributions based on reviewers' comments
V1.1.2 April 22, 2013 Burkhard (coordinator) Overall comments and guidelines
V1.1.5 April 23, 2013 George P. (ICOM), Guiherme (UZH),
Gerhard (TDG), Alessandro, Matteo
(IRT), Manos, Ioanna (AUEB),
Address comments by reviewers
V1.2 April 24, 2013 Ioanna, George S. Integrate partners input; address comments by Burkhard; final
comments sent to authors
V1.2.1 April 27, 2013 Guiherme, Julius, Chris, Gehrard,
Valentin, Alessandro
Address final comments
V1.3 April 29, 2013 Ioanna, George S. Pre-final version in BSCW
V1.4 April 30, 2013 Ioanna, George S. Final version in BSCW

Legal Notices
The information in this document is subject to change without notice.
The Members of the SmartenIT Consortium make no warranty of any kind with regard to this document,
including, but not limited to, the implied warranties of merchantability and fitness for a particular purpose. The
Members of the SmartenIT Consortium shall not be held liable for errors contained herein or direct, indirect,
special, incidental or consequential damages in connection with the furnishing, performance, or use of this
material.

Public

Table of Contents
Table of Contents 4
Table of Figures 8
Tables 10
1 Executive Summary 12
2 Introduction 16
2.1 Purpose of the Deliverable D1.1 16
2.2 Document Outline 17
3 Traffic & Energy Characteristics of Cloud Computing 18
3.1 Cloud Basics 18
3.1.1 Cloud and Data Centre 20
3.1.2 Cloud and Grid 20
3.1.3 Cloud and CDN 21
3.2 Cloud Models 21
3.2.1 Deployment Models 22
3.2.2 Service Models 23
3.3 SLAs of Cloud Services and Indicative Parameters 24
3.4 Financial / Economic Implications of the Cloud 25
3.5 Cloud Services and Real Cloud Deployments 26
3.5.1 Amazon 26
3.5.2 Google 27
3.5.3 IBM 28
3.5.4 Microsoft 28
3.5.5 Interoute 30
3.6 Traffic Characteristics of Clouds and Data Centres 31
3.6.1 Global IP Traffic Statistics and Trends 31
3.6.2 Data Centre Traffic Statistics and Tends 32
3.6.3 Cloud Traffic Statistics and Trends 34
3.6.4 Characteristics of Communication between Cloud Data Centres 36
3.7 Characteristics of Energy Consumption by Cloud Services 37
3.7.1 Data Centres 37
3.7.2 ISPs' Networks 41
3.7.3 Mobile Devices 41
3.8 Summary of Cloud Characteristics 44
Public

4 Characteristics of Cloud Applications 46
4.1 Video Streaming 47
4.1.1 YouTube 47
4.1.2 Netflix 48
4.1.3 Hulu 49
4.1.4 Relevance for SmartenIT 49
4.2 Mobile P2P Video Streaming 50
4.2.1 PPLive 51
4.2.2 PPStream 52
4.3 Online Storage 53
4.3.1 Dropbox 54
4.3.2 Google Drive 55
4.3.3 PiCsMu 57
4.4 Search Engines 59
4.4.1 Google Search 59
4.4.2 Yahoo! Search 60
4.4.3 Bing 60
4.5 Online Social Networks 62
4.5.1 Facebook 62
4.5.2 Twitter 63
4.5.3 Google+ 64
4.5.4 LinkedIn 65
4.6 Online Gaming 65
4.6.1 Onlive 66
4.6.2 Gaikai 66
4.6.3 Zynga 67
4.7 Cloud Content Delivery Networks 68
4.7.1 MetaCDN 68
4.8 Mobile P2P File-Sharing 70
4.8.1 BitTorrent 71
Public

4.9 Mobile Instant Messaging and Voice-over-IP 72
4.9.1 Viber 73
4.9.2 HeyTell 73
4.9.3 AbaCUS 74
4.10 Summary of Overlay Applications Characteristics 75
5 Stakeholders Characterization 77
5.1 Basic Definitions 77
5.2 Tools and Methodologies for Stakeholders Characterization 78
5.2.1 Value Network Configuration 78
5.2.2 Business Model Reference Framework 79
5.2.3 Tussle Analysis Framework 82
5.3 Inter Data Centre / Inter-Cloud communication 83
5.3.1 Scenario 83
5.3.2 Stakeholders, Roles, Interests 85
5.3.4 Business Modeling 88
5.3.5 Potential for Collaboration and Relevance for SmartenIT 90
5.4 Collaboration for Energy Efficiency 91
5.4.1 Scenarios 91
5.5 Global Service Mobility 100
5.5.3 Business Model Analysis 105
5.6 Social-Awareness for Content Delivery 107
5.6.1 Scenarios 108
Public

6 Summary and Conclusions 117
6.1 Key Outcomes & Lessons Learnt 117
6.1.1 Traffic Characteristics 117
6.1.2 Cloud-based Applications Characteristics 119
6.1.3 Stakeholders Characterization 120
6.2 Next steps 121
7 SMART objectives addressed 123
8 References 126
9 Abbreviations 137
10 Acknowledgements 139
A. YouTube Detailed Traffic Characteristics 141
A.1 Steps in YouTube video download 141
A.2 YouTube redirection process 143
A.3 Application-layer traffic management 143
A.4 QoE of YouTube video streaming key influence factors 145
A.5 Consequences for SmartenIT 146
B. Dropbox Detailed Traffic Characteristics 148
B.1 Monitoring setup 148
B.2 Methodology and data sets 149
B.3 Popularity of different cloud storage providers 150
B.4 Performance 151
B.5 Service workload 154
B.6 Consequences for SmartenIT 155
C. Facebook and YouTube Traffic Statistics 157
C.1 Packet and Flow Measurement for YouTube and Facebook 157
C.2 2
nd
order statistics results for YouTube and Facebook traffic variability 158
C.2.1 Definition of 2
nd
order statistics and autocorrelation function 158
C.2.2 2
nd
order statistics of self-similar processes 159
C.2.3 2
nd
order statistics for Gilbert-Elliott and 2-state (semi-)Markov models159
C.2.4 Measurement evaluations of the 2
nd
order statistics 160
C.3 Consequences for SmartenIT 161

Public

Table of Figures
Figure 3-1: Application of 80/20 rule to Company operating his own data centre [193]. .............. 26
Figure 3-2: Interoutes VDC. ........................................................................................................ 30
Figure 3-3: Trends in IP traffic growth reported from different sources. ....................................... 32
Figure 3-4: Global data centre traffic growing rate [20]. ............................................................... 33
Figure 3-5: Global data centre traffic by destination [20]. ............................................................ 33
Figure 3-6: Cloud vs. traditional data centre traffic growth 2011-2016 [20]. ................................ 35
Figure 3-7: Predicted US electricity use for data centres from the EPA report to Congress [151]
and the range estimated by Koomey [152]. .................................................................. 39
Figure 3-8: Breakdown of data centre energy overheads [139]. .................................................. 40
Figure 3-9: Interaction of measurement platform and developer [102]. ....................................... 42
Figure 3-10: Power breakdown of 20 users by hardware component, taken from [98]. ............... 43
Figure 3-11: DFA representing the WiFi energy consumption as modelled in [100]. ................... 43
Figure 3-12: Energy consumption of browser search; the green indicates the percentage of
energy wasted in tail states [100]. ................................................................................. 44
Figure 4-1: Bing ISN structure [198]............................................................................................. 61
Figure 4-2: Google+ User Statistics (Dec. 2012) [33]. ................................................................. 64
Figure 4-3: MetaCDN architecture [47]. ....................................................................................... 69
Figure 5-1: VNC notation. ............................................................................................................ 79
Figure 5-2: Tussle analysis methodology - The SESERV project [107]. ...................................... 82
Figure 5-3: Stakeholders map - The SESERV project [115]. ....................................................... 83
Figure 5-4: A source data centre uses available bandwidth to perform bulk backup to sink data
centre [116]. .................................................................................................................. 84
Figure 5-5: The scenario use cases and network placement of the data centres. ....................... 85
Figure 5-6: Basic VNC model. ..................................................................................................... 87
Figure 5-7: Modified generic VNC model. .................................................................................... 87
Figure 5-8: Generic VNC model. .................................................................................................. 88
Figure 5-9: An example of two data centre federations. .............................................................. 92
Figure 5-10: The NaDa architecture [192]. .................................................................................. 93
Figure 5-11: Generic VNC for separation of IaaS - PaaS Providers. ........................................... 95
Figure 5-12: Updated VNC for integration of IaaS - PaaS providers. .......................................... 95
Figure 5-13: Extended VNC to depict the insertion of the Energy Broker. ................................... 96
Figure 5-14: Extended VNC to depict the distributed approach (NaDa). ..................................... 96
Figure 5-15: Global Service Mobility high-level approaches for moving data resources. ........... 101
Figure 5-16: Application Provider without using Cloud to enable Global Service Mobility. ........ 103
Public

Figure 5-17: Application Provider using Cloud Providers to enable Global Service Mobility. ..... 104
Figure 5-18: Application Provider using Cloud with NaDas to enable Global Service Mobility. . 105
Figure 5-19: Content dissemination of UGC over an OSN. ....................................................... 108
Figure 5-20: Structure of a socially aware P2P scheme for content delivery [149]. ................... 109
Figure 5-21: Basic VNC for Exploitation of Social Awareness for Content Delivery Centralized;
Collaborative. .............................................................................................................. 111
Figure 5-22: Updated VNC for Exploitation of Social Awareness for Content Delivery
Centralized; Non-Collaborative; CDN-driven. .............................................................. 112
Centralized; Non-Collaborative; ISP-driven. ................................................................ 112
Figure 5-24: Extended VNC for Exploitation of Social Awareness for Content Delivery
Distributed; Non-Collaborative. ................................................................................... 113
Figure A-1: Organization of the YouTube Caches; dashed lines indicate possible redirections -
Figure taken from [55]. ................................................................................................ 142
Figure A-2: Identification of key influence factors on YouTube QoE. ........................................ 146
Figure A-3: Initial delays have almost no influence on MOS for videos of duration 60 s and 30 s
compared to influence of stalling length [66]. ........................................................... 146
Figure B-1: An example of the Dropbox communication observed [29]. .................................... 148
Figure B-2: Number of unique clients accessing each cloud-based storage service in Home 1
[29]. ............................................................................................................................. 150
Figure B-3: Data volume per cloud-based storage service (in bytes) in Home 1 [29]. ............... 150
Figure B-4: Share of total traffic volume in Campus 2 [29]. ....................................................... 151
Figure B-5: Traffic breakdown between Dropbox servers [29]. .................................................. 152
Figure B-6: Number of contacted storage servers [29]. ............................................................. 152
Figure B-7: Distribution of TCP flow sizes of file storage for the Dropbox client [29]. ................ 153
Figure B-8: Throughput of storage flows in Campus 2 [29]. ....................................................... 153
Figure B-9: Distribution of the number of devices per household [29]. ...................................... 155
Figure B-10: Dropbox session duration [29]. .............................................................................. 155
Figure C-1: Statistics on packet and flow sizes [142]. ................................................................ 158
Figure C-2: Statistics on the mean traffic rates per flow [142]. .................................................. 158
Figure C-3: Burstiness over time scales for YouTube, Facebook traffic parts [142]. ................. 160

Public

Tables
Table 3-1: Data centre traffic trends for 2011-2016 [20]. ............................................................. 33
Table 3-2: Cloud IP traffic trends 2011-2016 [20]. ....................................................................... 35
Table 3-3: Summary of initiatives for energy efficiency in data centres. ...................................... 38
Table 4-1: Top 10 applications in the Asia-Pacific region at peak time [68]................................. 51
Table 4-2: Typical energy reduction in company after migrating to Google Apps [202][128]. ...... 56
Table 4-3: Top 5 search engines [128]. ....................................................................................... 59
Table 4-4: MetaCDN performance and Content Providers benefits [47]. .................................... 69
Table 4-5: Summary of cloud-based applications' characteristics. .............................................. 76
Table 5-1: Business Model Reference Framework. ..................................................................... 80
Table 5-2: Stakeholder identification for inter data centre / inter-cloud communication. .............. 85
Table 5-3: Integrated stakeholder identification for collaboration for energy efficiency. .............. 93
Table 5-4: Stakeholder identification for global service mobility. ............................................... 102
Table 5-5: Stakeholder identification for social-awareness for content delivery. ....................... 110
Table 7-1: Overall SMART objectives addressed. ..................................................................... 123
Table 7-2: Specific SMART objectives addressed. .................................................................... 123
Table B-1: Datasets for Dropbox evaluation [29]. ...................................................................... 149
Table C-1: Packet and flow measurement for YouTube and Facebook [142]. .......................... 157

Public

(This page is left blank intentionally.)
Public

1 Executive Summary
This document is the deliverable D1.1 Report on Stakeholders Characterization and
Traffic Characteristics of Work Package 1 Traffic Measurements and Scenario
Definitions within the ICT SmartenIT Project 317846. The targets of Deliverable D1.1 are
as follows:
- Target 1 is to identify major characteristics of cloud-based overlay applications and
services such as traffic volume generated, traffic patterns, energy consumption,
QoE aspects, and end-user's devices implications. Additionally, the potential for
intervention and optimization w.r.t. traffic, energy and QoE need to be addressed
as the ultimate target of this investigation is to provide input to Deliverable D1.2 and
the selection process of the overlay applications to be addressed by the SmartenIT
project.
- Target 2 is to describe basic scenarios of cloud-service provisioning in which
SmartenIT can add value with innovative effective traffic management mechanisms
to be developed, identify the involved stakeholders, i.e., network provider, cloud
provider, end-users, etc., in these scenarios, as well as their interests in terms of
cost, traffic, energy and QoE optimization. Finally, Target 2 was to analyze the
identified stakeholders' relationships, so as to gain insight on the technical and
business dependencies among them, and identify potential for collaboration among
them to achieve a mutually beneficial situation.
Initially, addressing Target 1, we define crucial terms related to SmartenIT and provide our
understanding of: what cloud computing stands for practically, the differences between
cloud computing and grid computing, cloud and CDN, or cloud and data centre to narrow
down the field of applications and deployments of our interest leaving legacy CDN
deployments out of the scope of this deliverable and the project in general. Then, we
describe deployment and service models, economic and QoS aspects of cloud computing
to develop a common background on the impact of cloud computing on the various
Internet stakeholders.
Moreover, we overview a variety of cloud services, inter alia Amazon's EC2 and S3,
Windows Azure, IBM's SmartCloud, Google's App Engine, which are offered by well-
known major cloud operators, the so-called Internet giants, such as Amazon, Google, IBM.
Based on the aforementioned services, most of the cloud-based applications of
SmartenIT's interest are built on, e.g., Dropbox employs both Amazon's EC2 and S3 to
provide its personal online storage service. This description allows us to understand the
architecture and operation of the cloud-based overlay applications, to define realistic
scenarios of cloud service provision, and last to identify possible stakeholders
corresponding to real operators.
Furthermore, we overview various sources performing studies of traffic generated by data
centres and cloud, both as a share of global IP traffic, and in terms of absolute traffic
volumes, popularity and future trends. Major outcomes of this investigation are as follows:
- Global data centre IP traffic will nearly quadruple, while it will grow by 31% CAGR
over the next 5 years.
- On the other hand, global cloud IP traffic will increase 6 times and grow by 44%
CAGR over the next 5 years, while global cloud IP traffic will account for nearly two-
thirds of total data centre traffic by 2016.
Public

- The rapidly growing number of mobile devices (e.g., smartphones and tablets) used
to access new overlay applications constitutes a key driver of the cloud traffic
increase.
Concerning the increasing number of mobile devices, it has an important influence on how
application ecosystems are implemented nowadays, namely, applications 'run' in the cloud
(i.e., computation, data storage), while only a 'shell' providing an interface to access the
application remains at the mobile device.
Then, addressing the energy consumption of data centres, ISPs' networks and mobile
devices as an important characteristic of the energy efficiency in the cloud, we resort to
the following conclusions:
- Although the IP traffic was expected to grow by 46% per year in the last 5 years,
the energy consumption for data centres did not increase as predicted. In
particular, it has been estimated to have increased by up to 36% in the US, due to
the use of virtualization that leads to more efficient hardware utilization.
- Currently, only, 30% of total energy is delivered to IT equipment for critical
operations such as hard disk drive operations, networking, memory, CPU,
motherboards, fans, etc., implying PUE equal to 3.3.
- On the other hand, the remaining 70% of total data centre energy consumption
splits to 45% for cooling infrastructure and 25% for power delivery units.
Although virtualization is seen to be a key driver for energy efficiency in the cloud, it does
not provide for energy efficiency in the network domain. SmartenIT will design
mechanisms, which will manage cloud traffic by live VMs migration from data centre to
data centre, migration that will be strictly tied to data centre resources orchestration and
energy efficiency achievements.
Concerning ISPs' networks, the energy consumption has increased with traffic volume and
popularity of Internet applications and has become a critical issue for further traffic and
bandwidth growth. Moreover, energy saving by switching off parts of the network
equipment in phases of lower load and reactivating them on demand is often not feasible
currently due to long set up periods of networking equipment. On the other hand,
considering users' mobile devices, nearly 50% of all smartphone interactions have been
recorded as being related to communication, i.e., email, SMS, instant messaging and
voice calls, while proposed approaches that aim to optimize the consumed energy utilize
the trade-off of transmission range and energy consumption.
Based on our investigation of the energy implications of cloud computing, we observe that
although cloud improved energy consumption w.r.t. traffic growth; yet significant
inefficiencies still exist either in data centres and ISPs' networks, or in end users' mobile
devices. Such inefficiencies provide an opportunity for SmartenIT for intervention and
optimization.
Next, we perform an extensive overview of a wide selection of cloud based overlay
applications. We have provided a description of the characteristics of such applications
addressing their traffic, energy and QoE implications, potential for intervention and
potential for optimization. The outcomes of this investigation are as follows:
- Video streaming, P2P video streaming and file sharing, online storage, cloud CDNs
and online gaming generate high or very high traffic volumes, while online social
Public

networks, search engines and mobile instant messaging and VoIP generate
medium to low volumes,
- All categories of applications, except the mobile ones, cause a significant increase
of energy consumption in data centres, whereas mobile applications increase the
energy consumption in mobile devices, which is higher for video streaming,
- Considering QoE aspects, all of these applications require low latency, which may
not be always achieved, as their performance is highly dependent on network
conditions such as congestion.
Regardless of the type of application, the potential for intervention by a SmartenIT traffic
management mechanism is expected to be higher for non-proprietary solutions. Moreover,
potential for optimization is considered to be directly dependent both on generated traffic
volumes, energy implications and QoE requirements. Thus, potential for optimization is
considered to be higher for video-related applications, e.g., video streaming and online
gaming, where there is higher potential for a more visible impact by SmartenIT.
Addressing Target 2, we describe four basic scenarios of cloud-service provisioning in
which SmartenIT can add value. Then, employing appropriate tools and methodologies we
identify the involved stakeholders in the four scenarios of interest and characterize their
relationships, revenues and money flows, etc. Major outcomes of this analysis is for each
of the four scenarios as follows:
- The inter data centre communication scenario depicts the need to efficiently and
timely carry traffic among data centres that are located in different parts of the
network so as to comply with certain market-driven requirements and deadlines.
ISPs, Data Centre Providers, or even Application Providers, either existing or
emerging, may try to perform bundling of the inter data centre communication
service as provided to either other (possibly smaller) Data Centre, or Application
Providers and End-Users with existing services, e.g., storage, computation, so as to
dominate the market and strengthen their position in the markets where they are
active. Clearly the additional offering of such a sophisticated service would be of
high value to the data centre customers, whereas this in turn could create mixed
incentives for collaboration and competition with the other data centres and also
have adverse implications on the agreements with the ISPs.
- Collaboration between cloud operators to achieve energy efficiency can be
performed either in a distributed or a centralized manner. In the first case, a
SmartenIT mechanism could be seen as a set of architecture/functional elements
embodied within the aforementioned stakeholders, while in the second one, the
SmartenIT mechanism could be developed or placed as an extra entity among
them, so as to coordinate and enforce the collaboration for the efficient energy
consumption behaviour among all players. Such an entity would be responsible for
orchestrating the collaboration between cloud providers, and can be controlled by
either a member of the federation, or by a trusted third-party. However, in order for
the Energy Broker to be controlled by one member of the federation, then it would
be necessary to achieve a win situation not only for the entire federation, but also
for each member of it (due to the attained energy efficiency), at least in the long
term; otherwise, negatively affected members of the federation will either leave
from it, or even decide not to join at all. Nevertheless, depending on the considered
setup, various inter-connection agreements need to be established between the
various stakeholders, whereas different money flows will be generated then.
Public

- Global Service Mobility involves the capability to move resources between data
centres or clouds, close to where the end-users are located, without disrupting the
service delivered. In order to achieve Global Service Mobility, the user's location,
i.e., AS, needs to be detected; then, service-related data need to be moved to the
new user location. For the latter, two major approaches were identified, i.e., one
time data move, and gradual data move. The two approaches serve different
purposes and applications, while they can also be combined. The stakeholders
characterization revealed that when the Application Provider is using a Cloud
Provider in order to move users data/resources closer to the user, then the former
stakeholder may focus on the application layer and on the data-moving policies,
which are subsequently implemented by the Cloud Provider based on the latter's
optimization criteria and constraints, e.g., one-time data move, or gradual data
move. On the other hand, Global Service Mobility could be achieved by a
distributed approach involving also ISPs and end-users; e.g., the NaDas solution
could be extended to address service mobility apart from content mobility.
- Finally, the exploitation of social awareness to perform efficient content
delivery can be employed either in a centralized manner or in a more decentralized
one, i.e., forming a P2P overlay where end-users participate by offering their
storage and computation resources. A major question is who will apply the
mechanism, e.g., the CDN or the ISP; in any case, though, the social awareness
scenario within SmartenIT is considered to be a provider-driven solution. Moreover,
a SmartenIT mechanism could be seen either as a set of functional elements
embodied within the involved stakeholders, e.g., CDN provider, ISP, end-users'
clients, or alternatively as a module that could be employed by a new entity, e.g.,
an OTT Provider, placed among the existing stakeholders, so as to collect social
information and provide it to the CDN, or the ISP. Alternatively, in the peer-assisted
setup, the SmartenIT point-of-intervention could be: either a functional module
within the ISP, or a new OTT entity introduced, or the enhancement of the overlay
tracker orchestrating the P2P overlay.
A general comment based on the stakeholders characterization outcome is that
collaborative approaches addressing the various stakeholders incentives to collaborate
are highly relevant to SmartenITs scope and will drive the design and specification of the
traffic management mechanisms within WP2. The analysis in this deliverable has revealed
that there is indeed considerable potential for collaboration in a variety of cases.
Concluding the contribution of Deliverable D1.1 both to the next phases of SmartenIT but
also in general, we can state that D1.1 can serve as a complete handbook summarizing
the variety and characteristics of a multitude of cloud services and applications provided
over the cloud. D1.1 provides also a categorization of cloud-based applications based on
the type of the application, i.e., the service offered to the end-user, in order to organize
applications and address main characteristics of these categories, e.g., traffic volumes
generated, or QoS requirements. Nevertheless, a complete classification of the cloud-
based applications is to be conducted in D1.2, where also Task T1.1 "Cloud Services
Classification" is to be concluded. Moreover, D1.1 describes the four major scenarios of
high interest to SmartenIT and characterizes stakeholders so as to gain insight in their
interests, relationships under different setups and potential for collaboration among them.
Again, the complete and final definition of the SmartenIT scenarios will be provided in the
upcoming Deliverable D1.2, where also the work of Task T1.4 Scenarios Definitions will
be concluded.
Public

2 Introduction
The Internet has seen a strong move to support overlay applications, which demand a
coherent and integrated control in the underlying heterogeneous networks in a scalable,
resilient, and energy-efficient manner. Cloud-based applications, i.e. applications which
run completely or partly on the cloud or use cloud services, have set a strong need for
network efficiency at all layers; therefore, cooperative cross-layer traffic optimization is a
major goal to follow and can be achieved with tighter integration of network management
and overlay service functionality.
Therefore, SmartenIT targets an incentive-compatible cross-layer network management
scheme for providers of cloud-based applications, network providers, cloud providers, and
end-users to ensure a Quality of Experience (QoE)-awareness, by addressing accordingly
load and traffic patterns or special application requirements, and exploiting at the same
time social awareness (in terms of user relations and interests). Additionally, the energy
efficiency with respect to both end-user devices and underlying networking and application
provisioning infrastructure is tackled to ensure an operationally efficient management.
Moreover, incentive-compatibility of network management mechanisms for improving
metrics in all layers and among all players will serve as the major mechanism to deal with
real-life scenarios.
2.1 Purpose of the Deliverable D1.1
This document is the deliverable D1.1 Report on Stakeholders Characterization and
Traffic Characteristics of Work Package 1 Traffic Measurements and Scenario
Definitions within the ICT SmartenIT Project 317846.
The targets of this deliverable are as follows:
- Target 1 was to identify major characteristics of cloud-based overlay applications
and services such as traffic volume generated, traffic patterns, energy
consumption, QoE aspects, and end-user's devices implications. Additionally, the
potential for intervention and optimization w.r.t. traffic, energy and QoE need to be
addressed as the ultimate target of this investigation is to provide input to
Deliverable D1.2 and the selection process of the overlay applications to be
addressed by the SmartenIT project.
- Target 2 was to describe basic scenarios of cloud-service provisioning in which
The two targets are inter-dependent in that the characterization of traffic and energy
efficiency is performed towards the identification of potential for SmartenIT optimization for
the various stakeholders, while, on the other hand, the characterization of stakeholders
relationship is based on the description of the functioning of some representative cloud-
based applications, with their traffic and energy requirements.
Public

2.2 Document Outline
This document is organized as follows: Chapter 3 focuses on cloud computing and
services provided over it. Initially, in Section 3.1, the definition of basic terms related to
cloud computing is provided, while service and deployment models of cloud computing are
subsequently described. Moreover, we provide definitions of terms which are crucial for
our studies, i.e., what cloud computing stands for practically, and clarify the differences, to
our understanding, between cloud computing and grid computing, cloud and Content
Distribution Network (CDN), or cloud and data centre. Next, in Sections 3.2, 3.3, and 3.4,
we briefly address deployment models, economic and QoS aspects of cloud computing to
develop a common background on cloud computing impact on the various internet
stakeholders.
Then, in Section 3.5, we provide descriptions of a variety of cloud services offered by well-
known cloud operators. This description is highly useful to understand the architecture and
operation of the cloud-based overlay applications in Chapter 4, as well as to define
realistic scenarios of cloud service provision and identify possible stakeholders
corresponding to real operators in Chapter 5. Additionally, in Section 3.6, we provide an
overview of various sources on the implications of traffic generated by data centres and
cloud, both as a share of global IP traffic, and in terms of absolute traffic volumes,
popularity and future trends, whereas in Section 3.7, we overview several studies that deal
with characteristics of energy consumption by data centres and cloud, ISPs' networks, and
users' mobile devices. Sections 3.6 and 3.7 contribute to the qualitative assessment of the
considered cloud-based applications performed at the end of the next chapter.
Chapter 4 focuses on the applications side. In particular, the first part of the chapter
describes categories of applications provided over the cloud based on the type of service
offered. Two specific well-known, representative example applications for two significant
categories, i.e., YouTube for video streaming, and Dropbox for online personal storage,
and their traffic implications are further addressed in the Appendix. Moreover, apart from
well-established applications, we also address some less known emerging ones. The
reason is that they can constitute nice examples for evaluating traffic management
mechanisms where intervention potential is higher than other well established proprietary
applications where intervention may not be applicable. Finally, in Section 4.10, we perform
a qualitative analysis of the considered overlay applications given the description of their
operation, traffic and energy characteristics, potential for intervention and potential for
optimization provided in Chapter 4, and taking into account the traffic and energy
considerations of Sections 3.6 and 3.7.
Chapter 5 focuses on stakeholders and their relationships in cloud computing
environments. We overview selected tools and methodologies in literature for
stakeholders identification and for analysis of their interests, relationships and potential
conflicts; then, we apply some of these tools to identify the major stakeholders in four
scenarios of cloud service provision of high interest to SmartenIT. The outcome of this
analysis will serve as input to WP2 to identify incentives of stakeholders that need to be
addressed by the mechanisms to be designed.
Chapter 6 summarizes the entire deliverable and draws our major conclusions firstly on
the characterization of traffic generated and energy consumed by cloud-based services
and applications, and secondly on the analysis of stakeholders relationships in the
aforementioned setups. Finally, Chapter 7 briefly addresses the SMART objectives, as
described in SmartenITs Description of Work (DoW) [204].
Public

3 Traffic & Energy Characteristics of Cloud Computing
This chapter provides definitions of significant terms that will be employed in the rest of
the document and in the project in general. Additionally, main service and deployment
models of cloud are described, and cloud services provided currently by well-known cloud
operators are briefly overviewed. Furthermore, traffic characteristics of inter data centre
communications and generally traffic generated by cloud services and applications is
reported, while also energy implications related to data centres, ISPs' networks and end-
users' mobile devices are also addressed, based on information acquired from real cloud
service providers and the literature.
3.1 Cloud Basics
The SmartenIT project addresses traffic generated by new overlay applications provided
over the cloud, as well as inter-cloud / inter data centre communications facilitating such
overlay applications. SmartenIT's target is to design and develop mechanisms addressing
the incentives of all involved stakeholders so as to enable their collaboration in order to
efficiently manage the cloud traffic towards a mutually beneficial situation.
However, in order to develop a common understanding of the meaning of significant terms
such as overlay, cloud, data centre, etc., basic terms which will be used in this document
and throughout the entire project are defined in this section. Additionally, we clarify what
the major differences between cloud, data centre and CDN are, and identify the extra
capability provided by the cloud that distinguishes it from the grid.
Generally, an application is a set of software components operated to perform a detailed
task within a specific environment. Overlay applications are all kinds of applications that
are provided to the end-users as the outcome of a service provided by an overlay network,
or simply overlay; examples cover peer-to-peer applications, e.g., peer-to-peer
applications, cloud applications, or social networks.
The term overlay is ambiguous; it refers both to the overlay network and the overlay
application, while it implies that that network's or application's operation is performed on
top of, i.e., "lays over", another network. Specifically, any overlay network is a logical
communications network that operates on top of another network called the underlay, e.g.,
the physical network, or the Internet, and it is typically a user or a customer of that
underlay. In our analysis below, we tend to use the term underlay and physical network
interchangeably. While one overlay node corresponds to one physical node, logical links
may correspond to one or more physical links, i.e., a physical path; while a logical path
definitely corresponds to multiple physical links. An example of an overlay network, s a
social network; a social network represents an Internet platform where typically a large
number of users form a community inter-connecting themselves in friendship relations.
Typical functions of social networks are the following. Every participant has a profile where
personal information is located including hobbies and personal interests. Furthermore,
users create groups according to their interests and share information relevant to the
groups. This can for example be links to web pages or video on the respective topic. In the
SmartenIT context, social awareness links social information such as social network
structure, users preferences and behaviours, etc. to network management mechanisms.
This means that such mechanisms exploit the information contained in such networks in
order to perform efficient network management and traffic optimization.
Public

The SmartenIT project focuses on a major category of overlay applications, the cloud
applications; these include applications that are provided 'over the cloud'; an application
is considered to be provided over the cloud, when a part of it or the entire application is
built upon some cloud service, e.g., storage, computation, which are provided with remote
resources physically distributed on several variable places. Cloud resources can be
accessed with different levels of insight, which have been formalized by the National
Institute of Standards and Technology (NIST) usage models. Users may simply run an e-
mail application running in a Web server, build up an application using the cloud
programming and networking environment and storage resources, manage such an
environment for its applications. Moreover, the cloud concept has roots in fixed distributed
infrastructure such as Grid Computing, accessed remotely by a restricted population such
as scientists, bank and other business support employees. This legacy has been
formalized and broadened to serve all types of population and cover all types of usage
model with various degrees of application resources flexibility and security.
According to the NIST definition [1], the
'Cloud Computing is a model for enabling ubiquitous, convenient, on-demand
network access to a shared pool of configurable computing resources (e.g.,
networks, servers, storage, applications, and devices) that can be rapidly provisioned
and released with minimal management effort or service provider interaction'.
Cloud Computing is a specialized distributed computing paradigm, as it differs from
traditional distributed computing approaches in that 1) it is massively scalable, 2) can be
encapsulated as an abstract entity that delivers different levels of services to customers
outside the cloud, 3) it is driven by economies of scale, 4) it offers flexibility; i.e., the
services can be dynamically configured (via virtualization or other approaches) and
delivered on demand, and 5) it provides the capability to transfer workload to other
engines/data centres, i.e., outsourcing. For instance, a well-known example of cloud
computing is e.g., Amazon's Elastic Cloud Compute (EC2) platform which provides
resizable compute capacity in the cloud (see Section 3.5.1).
Constituent elements of the cloud are data centres. A data centre is a facility used to
house large computer, telecommunications and storage systems and their components
(e.g., servers) that besides backup power and communications also provides
environmental facilities, e.g., air conditioning. Usually, data centres consume enormous
energy amounts for operation, and cooling.
Furthermore, some other significant categories of overlay applications include:
A. Content Delivery Networks: A CDN distributes and disseminates content to the
edge of the access networks across Internet Regions. A CDN employs a large,
distributed system of strategically deployed data centres, i.e., servers, over the
Internet, aiming to store replicas of the content close to end-users and redirect
users requests to one of the closest copies, according to a combination of
selection criteria. Hence, end-users can access the requested content, with less
delay and timeouts, while traffic is reduced as content is not delivered from the
origin content server. Well-known examples of CDNs are Akamai [2] and
Limelight [3]. The cache cluster for support of Google YouTube can be seen as
an internal or content owner provided CDN as described in detail in Section
4.1.1. Moreover, large ISP often set up CDNs within their domain.
B. Peer-to-Peer networks (P2P): P2P networks are computer networks where tasks
and workload is partitioned among the network nodes called peers without the
Public

need for central coordination by servers or stable hosts [4]. P2Ps have become
increasingly popular especially to support file sharing applications, e.g.,
BitTorrent [5], KaZaA [6], and Voice-over-IP (VoIP) applications e.g., Skype [7].
C. Virtual Private Networks (VPNs): A VPN extends a private network, e.g., the
network of an enterprise, and the resources contained in the network across the
Internet [8]. It enables a host computer to send and receive data across the
Internet as if it were within a private network with all the functionality, security
and management policies of the private network. This is accomplished by
establishing a virtual point-to-point connection through the use of dedicated
connections, encryption, or a combination of the two. VPNs include mainly
proprietary solutions; additionally, a popular type of VPN is the IP MPLS VPN
employing the Multi-Protocol Label Switching (MPLS) to prioritize, transport and
route the VPN traffic over links in the public network.
As mentioned in the DoW [204], SmartenIT addresses cloud traffic, i.e., traffic generated
by applications and services provided over the cloud. Moreover, based on the cloud
services classification performed within Task T1.1, and the investigation of the various
cloud application categories for the purposes of Tasks T1.2, T1.3 and T1.4, legacy CDN
architectures, i.e., non-cloud ones, fall out of the scope of the project. Therefore, due to
the fact that services provided by CDNs and special characteristics of the latter practically
constitute a subset of the services and characteristics of the cloud computing itself, we try
to explicitly clarify below the differences between the cloud and the CDN model, taking
into account various criteria related to the physical infrastructure, its ownership and
operation, the content and servers location, the distribution level, and others. Moreover,
we address differences of the cloud to a data centre and cloud's predecessor, the Grid.
3.1.1 Cloud and Data Centre
Data centres provide their customers, i.e., Service Providers, CDNs, Application Providers,
with specific servers, network, and storage. On the other hand, clouds provide their
customers with abstracted versions of these components that are not tied to a specific
data centre: virtual servers, virtual storage, and virtual networking. Being an abstraction of
data centres, clouds are a representation of the very notion of location independence and
virtualization; this is a technique that allows the sharing of one physical server among
multiple Virtual Machines (VMs), where each VM can serve different applications, while
the CPU and memory resources can be dynamically provisioned for a VM according to the
current performance requirements. Clouds provide resources reasonably reliable and
redundant, while where or how these are hosted is variable and dynamic. Reliability and
redundancy comes from cloud providers using multiple data centres, so clouds almost
certainly span one or more data centres, but themselves are more than data centres.
3.1.2 Cloud and Grid
In [9], an extensive comparison between cloud computing and grid computing is
performed taking into account multiple aspects including business models, architecture,
resource management, programming model, application model, and security model. As
stated in [9], cloud has evolved from grid, and relies on it as its backbone and
infrastructure support. The evolution has been a result of a shift in focus from an
infrastructure that delivers storage and computation resources (as in grids) to one that is
economy-based aiming to deliver more abstract resources and services (as in clouds) in a
Public

dynamic manner. Major conclusions of this comparison are that cloud and grid share
common vision and similar architecture and technologies, though they significantly differ in
terms of security, programming model, business model, compute model, data model,
applications, and abstractions.
3.1.3 Cloud and CDN
Comparing Cloud and CDN is not an easy task, due to the fact that many alternative
deployments of both might exist depending on special environments and perspectives
within the general notion of Cloud computing and CDNs, respectively. For instance, there
are global and local CDNs, or CDNs owned by a dedicated entity and CDNs deployed by
ISPs, etc.
As aforementioned, traditional CDNs aim at improving the performance of websites and
content delivery services by deploying an extensive network of geographically distributed
data centres worldwide. They were initially built to cache and serve static content, but
have adapted to the increasing needs of the content delivery market and manage to serve
dynamic content using dynamic content acceleration techniques such as content pre-
fetching, as well as on-the-fly compression techniques. A CDN is a network with static
topology, and whose internal structure is transparent to its users. CDN servers locations
are disseminated to DNS name servers and users requests are redirected via DNS to the
most appropriate stored replica based on a combination of IP-based proximity and
performance criteria, e.g., server condition, achieving low response times, high
throughput, and service availability. CDNs display high content availability, reliability and
QoS, whereas their main disadvantages include high operational and maintenance costs,
difficult and expensive allocation of new resources to the network.
On the other hand, cloud computing is deployed in a dynamic manner and offers a wider
range of services, such as storage, and computation, software and content delivery,
whereas users of a cloud have access to cloud services via a specific interface made
available by the cloud designer/operator. Cloud providers manage their infrastructure and
services in one or more data centres, deployed worldwide based on economic, energy
efficiency and often geographical criteria. In principle, cloud computing relies heavily on
the sharing of resources and services, aiming to achieve lower costs and scalable
performance. Users requests are redirected towards the most appropriate server based
on internal procedures and policies, mostly dependent on servers and private networks
conditions. Advantages of clouds comprise of scalability, cost-efficient performance,
service availability, hardware independence, dynamic allocation of new resources and
virtualization. Their disadvantages include security and privacy issues in the sense that
users' personal information, e.g., data and settings, are stored in their premises.
3.2 Cloud Models
In this section, we describe briefly main deployment and service models for cloud
computing, which will constitute the background to describe in next sections (see Section
3.5) real cloud deployments, as well as to describe interesting application scenarios in
Chapter 5.
Public

3.2.1 Deployment Models
Depending on who owns and manages the cloud infrastructure, the following four
deployment models are identified by NIST [1]:
i. Private cloud: The cloud infrastructure is provisioned for exclusive use by a single
legal organization comprising multiple consumers (e.g., business units). It may be
owned, managed, and operated by the organization, a third party, or some
combination of them, and it may exist on or off premises. Private cloud deployment,
w.r.t a public deployment, can provide cloud services with enhanced security
features as dictated by user requirements, higher availability as resources are
dedicated only to a certain group of users, and higher flexibility due to the fact that
services can be tailored as per specific user requirements.
ii. Community cloud: The cloud infrastructure is provisioned for exclusive use by a
specific community of consumers from organizations that have shared concerns
(e.g., mission, security requirements, policy, and compliance considerations). It
may be owned, managed, and operated by one or more of the organizations in the
community, a third party, or some combination of them, and it may exist on or off
premises.
iii. Public cloud: The cloud infrastructure is provisioned for open use by the general
public. It may be owned, managed, and operated by a business, academic, or
government organization, or some combination of them. It exists on the premises of
the cloud provider.
iv. Hybrid cloud: The cloud infrastructure is a composition of two or more distinct
cloud infrastructures (private, community, or public) that remain unique entities, but
are bound together by standardized or proprietary technology that enables data and
application portability (e.g., cloud bursting for load balancing between clouds).
Moreover, the relationships among different cloud operators, considered as different
administrative entities, are ruled by the following definitions:
- Single Cloud Model is the most typical model, followed by big cloud operators
(i.e., Google, Amazon). who deploy different data centre infrastructures in
geographical diversity, with connectivity on Tier-1 and Tier-2 domains. Connectivity
among different data centres is provided based economic agreements with ISPs,
but no agreements with other cloud operators are established.
- Federated Cloud model is the model in which smaller cloud operators (with
connectivity on Tier-2, and Tier-3 domains) join together to form a federation (a sort
of super-cloud entity), in order to achieve economics improvements in terms of
increasing capacity to serve more end-users and enlarging the platform of offered
services. One major advantage of the federated cloud model is the possibility for
smaller cloud operators to use resources located in data centres that belong to
other cloud operators who have joined the federation. From user perspective, the
differences between the cloud operators joining the federation are transparent. One
of the major concerns of this model related to the protocols used to establish the
federation as well as issues about user identity, security and privacy.
- Interconnected cloud model is the last model of interaction among cloud
operators. Its similar to the previous model except for the fact that no federation is
formed. The Interconnected Cloud Model foresees that each cloud operator
maintain its administrative role, while also establishes economic agreements with
Public

other partners to achieve service mobility, i.e., offload its computing and hosting
capacity, with the final aim to ensure proper QoE to its own customers.
3.2.2 Service Models
According to [1], there are three major service models for the cloud which are defined
w.r.t. the capabilities offered to the end-users:
1. Software as a Service (SaaS)
SaaS is defined in [1], as "the capability provided to the consumer is to use the cloud
providers applications running on a cloud infrastructure. The applications are
accessible from various client devices through either a thin client interface, such as a
web browser (e.g., web-based email), or a program interface. The consumer does not
manage or control the underlying cloud infrastructure including network, servers,
operating systems, storage, or even individual application capabilities, with the possible
exception of limited user-specific application configuration settings".
The deployment model can be either public, e.g., e-mail service such as Gmail, or
private, e.g., a hosted product as in the case of Customer Relationship Management
(CRM) services delivered to large enterprises.
2. Platform as a Service (PaaS)
PaaS is defined as "the capability provided to the consumer is to deploy onto the cloud
infrastructure consumer-created or acquired applications created using programming
languages, libraries, services, and tools supported by the cloud provider. The consumer
does not manage or control the underlying cloud infrastructure including network,
servers, operating systems, or storage, but has control over the deployed applications
and possibly configuration settings for the application-hosting environment" [1].
PaaS is typically priced based on a utility billing model, so users pay only for what they
use. Because PaaS provides a complete hardware and software infrastructure, it
eliminates the need to invest in hardware or software and offers significant
infrastructural savings. Some of the features that can be included with a PaaS offering
may include: operating system, server-side scripting environment, database
management system, server software, support, storage, network access, and tools for
design and development.
The deployment model applied for PaaS can be either public, e.g., Google App Engine,
Windows Azure, or private, e.g., a hosted product provided by Interoute to an enterprise
customer.
3. Infrastructure as a Service (IaaS)
IaaS refers to "the capability provided to the consumer is the provision of processing
power, storage, networks, and other fundamental computing resources, where the
consumer is able to deploy and run arbitrary software, which can include operating
systems and applications. The consumer does not manage or control the underlying
cloud infrastructure but has control over operating systems, storage, and deployed
applications; and possibly limited control of selecting networking components (e.g., host
firewalls or machines with specific characteristics)" [1].
IaaS products are mainly offered over a private cloud deployment model to business
customers who want to offload their IT structure.
Public

Apart from their deployment and service model, the offered cloud services can be also
classified based on their type to 7 major categories; these are as follows:
- Dynamic servers which combine the flexibility and virtualization of cloud computing
with dedicated hardware and computational power of traditional dedicated servers.
Dynamic servers allow users to create VMs and to configure them by scaling up or
down, while using private resources on dedicated hardware.
- Content Delivery which involves content dissemination (through file sharing or video
streaming) to multiple end-users with high availability and high performance, e.g.,
YouTube for video streaming.
- Cloud Storage which is a model of networked online storage where data is stored in
virtualised pools of storage usually hosted by third parties, e.g., Amazon's Simple
Storage Service (S3). In fact, physical storage resource may span multiple servers.
- Hosted email which is a model of email service provided by a mail-server that runs
on the cloud, e.g., in the email-provider's or a third-party's data centre. Thus, the
hosted email account is accessible by a 'light' client or the web browser, e.g.,
Gmail.
- Hosted desktop which is a service within the broader cloud-computing sphere,
delivered as a combination of hardware virtualization and processing capability.
Processing takes place within the cloud providers data centre environment, while
different works or tasks are allocated by the cloud provider, transparently to the
end-user, to different servers.
- Hosted telephony which refers to VoIP telephony where the server handling or
terminating calls is on the cloud. Voice services over the cloud allow users to
expand their options beyond local or regional carriers.
- Finally, application engines which allow end-users to develop and run/host
applications in cloud operator-managed data centres. Applications are then
'sandboxed' and run across multiple servers.
3.3 SLAs of Cloud Services and Indicative Parameters
A Service Level Agreement (SLA) is a formal definition of the relationship that exists
between a service provider and its customer. An SLA is used to specify what the customer
could expect from the provider, the obligations of the customer as well as the provider,
performance, availability and security objectives of the service, as well as the procedures
to be followed to ensure compliance with the SLA.
As for other managed network services, the typical SLAs between a cloud service provider
and its customers should encompass at least the minimum and maximum service levels
across all tiers of service provisioning. This should regulate:
- Planned and unplanned downtime,
- The process in which the cloud provider will notify the company of planned
downtime, and mechanisms to accept or defer,
- Contractual penalties for any unplanned downtime suffered outside of the agreed
SLA,
Public

- Agreements for events, which the cloud provider has no control over; for example,
natural disasters at the cloud providers data centre,
- Operational Level Agreements (OLAs), which record engagement details between
the cloud provider support teams including contact details during business and non-
business hours, and
- Definition of individual support tiers contained within OLAs, including individual
responsibilities for service, process and delivery timeframes.
In addition, service providers should be able to offer SLAs based on application and user
requirements. Moreover, in a context in which cloud operator provides services to a
business customer the SLA may include application requirements like response time,
application availability, minimum time to repair and issue resolution. Finally, in the context
in which ISP offers connectivity services to a customer such as a cloud or data centre
owner, an SLA may include customer requirements like upper bound for delay and
latency, packet loss rate and minimum guaranteed throughput.
3.4 Financial / Economic Implications of the Cloud
Cloud computing provides some strong benefits to mid-sized and big companies who
decide to outsource IT capabilities to the cloud. Main advantages of the IT migration to the
cloud are summarized below.
Organizations that transfer their IT workload to the cloud enjoy reduced IT costs by
transitioning from capital expenses (CAPEX) such as enterprise software licenses, and
servers and networking equipment to more fluid, less expensive operational expenses
(OPEX). Additionally, by leveraging on cloud services, the organization essentially pays for
what it needs, when it needs it, following a pay-per-use model, whereas scalability
provided by the cloud promotes a company's faster growth.
Moreover, the cloud allows businesses to rapidly alter products, processes, and services
to align with shifting customer preferences, as well as the ability to respond quickly to
changing both customer needs and market demands. Because complexity is veiled from
the end-user, a company can expand its product and service sophistication without also
increasing the level of user knowledge necessary to utilize or maintain the product or
service. Furthermore, clouds ability to store more information, due to its expanded
computational power and capacity, provides greater opportunities for customization of
products and services. Because of cloud, businesses can create a more customer-centric
experience by adapting to slight changes in a user-defined context.
Nonetheless, companies can achieve economies of scale increasing volume output or
productivity with fewer people, monitor projects more effectively, and reduce costs for
energy consumption, since the physical hardware is operated in the cloud.
In terms of efficiency achieved with cloud computing solutions, a study from [193] shows
that when looking at organizations running their own data centre infrastructure we
hypothesize that only 20% of the time and effort that goes into running applications, where
all business value is concentrated. The diagram in Figure 3-1 illustrates the extent to
which routine and non-core tasks, like patching operating systems and performing
backups, impact upon the time of IT departments.
Public

Figure 3-1: Application of 80/20 rule to Company operating his own data centre [193].
Cloud computing is a force that helps flip this ratio and gives IT departments the ability to
spend 80% of their time and money on core business processes, like business application
design.
3.5 Cloud Services and Real Cloud Deployments
In this section, we overview and briefly describe cloud services provided by well-known
cloud operators, some of them also known as Internet Giants, e.g., Amazon, Google,
Microsoft, etc., as well as services provided by Interoute (IRT), based on which most (if
not all) cloud-based applications are built, e.g., Dropbox employs both Amazon's EC2 and
S3 to provide its personal online storage service. This description is highly useful to
understand later the architecture and operation of the cloud-based overlay applications in
Chapter 4, well as to define realistic scenarios of cloud service provision and identify
possible stakeholders corresponding to real operators in Chapter 5.
3.5.1 Amazon
Amazon Web Services (AWS) provides a complete set of cloud computing services that
enable you to build sophisticated, scalable applications. Below, the most significant web
services are summarized.
Amazon Elastic Compute Cloud (Amazon EC2) [170] is a web service, in particular a
PaaS solution, that provides resizable computational capacity in the cloud. Furthermore,
EC2 reduces the capacity scaling time, and allows paying only for capacity that is actually
being used. Last but not least, Amazon EC2 provides tools to build failure resilient
applications.
Next, Amazon Elastic Block Store (Amazon EBS) [171] provides block level storage
volumes for use with Amazon EC2 instances. The storage volumes can be attached to a
running Amazon EC2 instance and exposed as a device within the instance. Amazon EBS
is particularly suited for applications that require a database, file system, or access to raw
block level storage.
Another important web service offered by Amazon is Amazon's Simple Storage Service
(Amazon S3) [172], which provides a web services interface that can be used to store and
retrieve unlimited data on the web. Amazon S3 is an IaaS solution and aims to make web-
Public

scale computing easier since it claims that the developer has access to the same
infrastructure that Amazon uses to run its own global network of web sites.
Finally, CloudFront [173] is Amazons CDN, seamlessly integrating with Amazons cloud
services (Amazon S3 cloud storage service, EC2 cloud computing platform, etc.). The
service was initially deployed in November of 2008, after the release of the S3 cloud
storage service in 2006, which led to massive amounts of data delivered to end-users
worldwide. A CDN approach, optimized to work with AWS, was adopted to improve
scalability and performance. Content creators or providers are able to access and manage
the service either through AWSs website or CloudFronts exposed API, initially declaring
their content servers which might be either within (e.g., an S3 storage server) or outside
(e.g., providers private cloud or external cloud storage solution) Amazons AWS. In
addition, the service provides a namespace, and subscribed providers can share their
content to their websites or dedicated applications. The load balancing and redirection
mechanisms handle the end-users requests and redirect them to the closest content
replica. Besides traditional HTTP and HTTPS distribution, since 2012 CloudFront also
supports streaming distributions using the Real Time Messaging Protocol (RTMP). The
Amazon CloudFront Global Edge Network consists of around 40 data centres in 14
countries worldwide and offers reliable and fast content delivery. Tests conducted using
the Compuware Corporation performance network, indicated that CloudFront was 10%
faster than the average latency of 3 major traditional CDNs (Akamai, Limelight and Level3)
for 1 MB objects and 20% faster for 12 KB objects [10]. In addition, results from Last
Mile tests indicate CloudFronts comparable performance to Akamais CDN.
3.5.2 Google
In addition to the well-known search and mail services, the hyper giant Google offers a
considerable number of other cloud services including office software, storage,
computation, web application deployment and even data mining and translation tools. In
the following we give a very brief overview of them with information mainly taken from the
developer web page of Google [148].
The Google App Engine offers to deploy and run web applications on the infrastructure of
Google. This permits the applications to scale with increasing user demands without
setting up new servers. Such tasks are automatically handled by the app engine, which
balances the load within the Google-owned infrastructure. However, application
developers need to base their applications on the application framework provided by
Google. The framework supports application development in Java, Python and the Go
programming language.
The Google Compute Engine can be classified as a PaaS offer, where customers can rent
virtual Linux machines. The machines are priced on a per usage basis and can have
different numbers of virtual CPU cores and varying amounts of memory and disk space.
Within the virtual machine the user can install and run basically every piece of software
that is available for the given operating system. At the moment, CentOS and Google
Compute Engine Linux are supported operating systems.
Google Cloud Storage also targets at application developers and enables them to store,
move, retrieve and download data and share them with other users within the Google
cloud. The data is identified with unique keys, which are in turn organized in buckets. The
service is based on a RESTful architecture, which is stateless and defines clients that
send requests to store data at servers, i.e., in the Google infrastructure, or retrieve them
Public

from there. Note that Representational State Transfer (REST) is a programming paradigm
for web applications, which incorporates the idea that the URL of a website already
defines the content of the website. Then, access control in the Google Cloud Storage is
possible using Google account email addresses for example. Finally, it should be noted
that this service is different from Google Drive, which targets end users and not application
developers.
Google BigQuery and Google Cloud SQL facilitate to manage relational databases and to
execute SQL-like queries on these databases or even on public or shared data sets. The
focus of Google BigQuery is on the execution of database queries on very large data sets.
The structure of such data sets is limited to a single table, even if this table might be huge.
If entire relational databases are required, Google proposes to use Google Cloud SQL.
This service is based on MySQL and provides all features that relational databases
typically have. The main difference is that Google Cloud SQL runs in the Google cloud.
Furthermore, Google offers machine learning and data mining tools to application
developers. As possible use cases, Google mentions the prediction of movies that a user
might like based on the movies he watched in the past. Another use case is the
identification of spam emails. This service is called Google Prediction API.
The Google Translation API service is able to translate text from one language into
another one covering 64 different languages. Like all aforementioned services, it is
charged on a per usage basis and build based on a RESTful architecture. In addition, the
service also offers language recognition, i.e., it can detect the language of a given text.
Finally, the services Google Docs and Google Drive target at home and business users
instead of application developers. These services provide office tools such as word
processing, table calculation or presentation tools to the users and permit the collaborative
editing and sharing of such documents. Therefore, they can be categorized as software-
as-a-service (SaaS).
3.5.3 IBM
IBMs SmartCloud [147] is a line of enterprise-class cloud computing technologies and
services which was launched in April 2011. It can be used for building private, public and
hybrid clouds. IBM Smart Cloud offers all three types of cloud services, IaaS, PasS and
SaaS, as the following three product scopes:
- SmartCloud Foundation consists of software and hardware infrastructure. It gives
needed functionalities to set up clouds on virtualized hardware, manage virtual
machines and monitor the performance of virtual and physical environments;
- IBM SmartCloud Application Services is a PaaS environment for development and
deployment of applications to the cloud. It provides cloud-based development tools,
workload patterns, middleware and databases;
- IBM SmartCloud Ecosystem is a set of new services for software vendors to
support and improve their products. It helps their clients to adopt cloud models and
manage cloud-based actions.
3.5.4 Microsoft
Microsoft declares that its cloud services create a correlated family of solutions and are
dedicated to small businesses in order to make available resources which are scarce for
Public

companies using Microsoft products and being familiar with applications [118]. Cloud
services are split into two categories: as commercial (12 services, e.g., Windows Azure,
Windows Azure Platform Appliance, SharePoint Online, Office Live Meeting ) and
consumer (10 services, e.g., Windows Live, Windows Live Hotmail, Windows Live
Messenger, Bing, Zune, MSN, Xbox Live, etc.).
Windows Azure [119], delivered 1 February 2010 is a cloud services platform and
infrastructure, for building, deploying and managing applications, storing data across a
global network of Microsoft-managed data centres. It provides both PaaS and IaaS, and
supports many different programming languages, tools and frameworks. Windows Azure
allows to easily scale the applications to any size. It is available in multiple data centres
around the world, enabling to deploy the applications close to the customers. As an open
platform, Windows Azure offers choices to developers. It allows them to use multiples
languages (.NET, PHP, Ruby, Python or Java) and development tools (Visual Studio or
Eclipse) to build applications. The Windows Azure supports multiple popular Internet
standards and protocols including HTTP, XML, SOAP and REST.
The Windows Azure CDN [120] is Microsofts CDN solution, to support its cloud computing
platform, i.e., Windows Azure. It was available since the beginning of the cloud platforms
deployment in 2008, targeting the caching of Windows Azure content in strategically-
placed physical servers around the world in order to improve content delivery,
performance and scalability.
Content creators and providers may register and access the service via the Windows
Azure website, where they can either create their personal cloud storage space within
Windows Azure services or define their external sources (other cloud storage services or
other CDNs). By enabling the CDN service, any publicly available content will be cached
to the CDN nodes and redirected to end-users, upon request. In addition to static content,
the Windows Azure CDN caches dynamic content from Azure-hosted applications,
enabling hosted applications to scale and perform better. It also provides all the necessary
tools for multiple programming languages to create applications that use their services.
24 nodes in 17 countries worldwide constitute the Windows Azure CDN network. Each
data centre contains more than 300,000 servers [15], organized in 400-2500 server
containers, including racks of 96 servers with estimated power of 16 kilowatts per rack
[16]. The Dublin data centre is estimated to generate up to 5.4 megawatts of critical
power, with the potential to expand to a total of 22.2 megawatts of critical power. Windows
Azure ensures that the service has an uptime 99.95%.
The Windows Azure SQL Database is a highly available, scalable, flexible database
service hosted by Microsoft in the cloud [158], [159], [160]. It is offered by Windows Azure
platform, formerly known as SQL Azure Database. It is relational cloud databases built on
the Microsoft SQL Server and provides a high-level of interoperability, enabling customers
to build applications using many of the major development frameworks. The latest release
of SQL Databases was completed in November 2012 and offers several enhancements.
The Windows Live [161], launched 1 November 2005, is a collection of online services
and applications from Microsoft for individuals. It combines advantages of Windows
system with cloud computing, provides online services (e.g., Windows Live Hotmail,
Windows Live SkyDrive), software applications (e.g. Windows Live Mail, Windows Live
Messenger, Windows Live Movie Maker) and search services (e.g., Bing). As official
statistics show Microsoft over 330 million registered users.
Public

The Microsoft Zune [162] is a free digital media platform, launched by Microsoft. It
provides a multiple services (e.g., Zune Application, Zune Media Player, Zune Xbox Music
Pass, Zune Marketplace, Zune Software).
3.5.5 Interoute
IRT owns and operates 8 data centres across Europe, directly connected through the
Interoute network. This pan-European platform is designed for the delivery of enterprise
IaaS and virtualized services. Each data centre is deployed with a network fabric deeply
embedded in the IRTs infrastructure to support effective access, geographic fallback and
full product range.
IRTs Virtual Data Centre (VDC) is a multi-tenant, IaaS platform that offers to the
customers on-demand computing and cloud hosting with integrated applications. It creates
the ability to offer either private or public cloud computing, offering public simplicity with
private cloud security. Each of the components selected by a customer for its VDC can be
provisioned and bound to its existing IRT VPN network, a private VDC network or an
Internet routable network. Moreover the customer is allowed to subscribe to two or more
IRT sites for resiliency purposes. Figure 3-2 illustrates IRTs VDC both for public and
private deployment models.

Figure 3-2: Interoutes VDC.
From a customer perspective, IRTs VDC is seen as a pool of compute and storage
resources available on an Interoute provided network and hosted within geographically
separated zones; additionally, the customer is presented with a VDC Control Centre
graphical http-based interface, i.e., the Hub, which allows operations such as
configuration of services, backups reporting, and device performance reporting. After
logging on the portal, VDC resources are instantiated for the customer who is allowed to
perform the following operations: create storage for VMs, select image, create storage
volume for VMs, and further deployment operations.
Public

The VDC product and delivery relies on a modular and scalable design that is vendor
agnostic. Each of the components selected are physically connected and configured in a
standard organized group, referred to as a Pod. A pod must contain all three
components: compute, storage and network. Each pod is integrated into the existing
platform by the Orchestration software. Thus, a VDC can be defined as 4 main
components:
1. Cloud Orchestration: Software that orchestrates the delivery and management of
VMs for a customer/service provider, and the interfaces into the system (VDC
Console & API),
2. Compute: Provides CPU and Memory resources through the use of Type II
Hypervisors on commodity hardware,
3. Storage: Provides shared disk resources for hosting virtual appliances as well as
block storage to the appliances,
4. Networking: Provides pre-provisioned multi-tenanted network and connectivity
between all components, customer external and internal LANs and the Internet.
IRT leverages on a suite of IT infrastructure that enables the customers to configure
proper services in terms of hosting, computational capacity, security which is totally
automated providing fast service provisioning, and rapid delivery model.
3.6 Traffic Characteristics of Clouds and Data Centres
In this section, a brief survey on overall IP and cloud traffic analysis and trends is
presented based on data mainly taken from Cisco Virtual Networking Index [23], Global
Cloud Index [20] as well as other publicly available statistics. Additionally, we address and
compare traffic statistics and the evolution of traffic generated by inter data centre and
inter-cloud communications, while finally, we characterize inter data centre communication
based on major services categories and critical factors affecting it. Moreover, the cloud
traffic survey provided in this section will provide input to the overall assessment of cloud-
based applications to be performed in the next chapter, Chapter 4.
3.6.1 Global IP Traffic Statistics and Trends
The development of IP traffic over the last decade is visible in official statistics issued by
administrative bodies of several countries, e.g., for Germany [17], Australia [18], Hong
Kong [19], and as well as periodical announcements on traffic trends by vendors of routing
and measurement equipment, e.g. the well-known reports by Cisco [23] and Sandvine
[21]. Additionally, measurement results from links on TDGs IP platform reveal major
differences in the traffic mix over a daily load profile and in the up-/downstream direction
[22]. Consequently, meaningful traffic statistics should indicate if results are valid for busy
hours or as mean value over weekdays or also including weekends, as well as where the
statistics has been taken from (region(s), organization(s), core/aggregation network/LAN,
up-/downstream).
Figure 3-3 summarizes the outcomes of the aforementioned studies and illustrates the
common main trend, indicating traffic growth by a factor of the order 100 for fixed IP
network traffic over the past decade, although there are different phases of steep and
slower annual increase. Moreover, traffic in mobile networks is included (dashed lines),
which currently accounts for only a few percentage of the total IP traffic but is catching up
Public

at higher increase rates of 50%-150% per year as compared to 25% - 50% annual
increase as reported from fixed networks.
1
10
100
1000
10000
100000
1000000
2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011
Year
I
P

T
r
a
f
f
i
c

G
r
o
w
t
h

S
t
a
t
i
s
t
i
c
s

[
P
e
t
a
B
y
t
e
/
Y
e
a
r
] Global Internet, Cisco VNI
N-America, Cisco VNI
W-Europe, Cisco VNI
Germany, BNetzA
Hong Kong, OFTA Statistics
Australia, Bureau of Statistics
Global, Cisco VNI, Mobile IP
Germany, BNetzA, Mobile IP
Hong Kong, OFTA, Mobile IP

Figure 3-3: Trends in IP traffic growth reported from different sources.
3.6.2 Data Centre Traffic Statistics and Tends
The cloud traffic statistics presented in this section are derived based on aggregated
statistics related to services delivered to end-users via both CDNs and clouds [20].
Although the Internet is forecast to reach the Zettabyte era in 2016, the data centre has
already entered it. While the amount of traffic crossing the Internet and IP WAN networks
is projected to reach 1.3 ZBs (i.e., 10
9
TBs) per year in 2016, the amount of data centre
traffic is already 1.8 ZBs per year, and by 2016 will nearly quadruple to reach 6.6 ZBs per
year. This represents a 31% Compound Annual Growth Rate (CAGR). The higher volume
of data centre traffic is due to the inclusion of traffic inside the data centre (typically,
definitions of Internet and WAN stop at the boundary of the data centre). Figure 3-4 shows
the growing rate of global data centre traffic including traffic generated within the data
centre by 2016. Data centres traffic can be broadly categorized into three main areas w.r.t.
the destination of the traffic flows, i.e.:
- Traffic that is generated and remains within the data centre,
- Traffic that flows from data centre to data centre, and
- Traffic that flows from the data centre to end users through the Internet or IP WAN.
The portion of traffic residing within the data centre will remain the majority throughout the
forecast period, accounting for 76% of data centre traffic in both 2011 and 2016. Factors
contributing to traffic remaining in the data centre include functional separation of
application servers, storage, and databases, which generates replication, backup, and
read/write traffic traversing the data centre. Furthermore, parallel processing divides tasks
and sends them to multiple servers, contributing to internal data centre traffic.
Public

Figure 3-4: Global data centre traffic growing rate [20].

Figure 3-5: Global data centre traffic by destination [20].
Table 3-1 summarizes the expected evolution of traffic by data centres from 2011 to 2016,
where data centre traffic is categorized by destination (i.e., within data centre, inter data
centre, etc.), or segment (business vs. consumer), or type (traditional vs. cloud).
We observe that traffic within data centres is about 4.5-fold higher than traffic originating
from data centres towards the end-users, and 11-12-fold higher than traffic between data
centres, i.e., inter data centre traffic. The volume of 299 Exabytes (EBs) (i.e., 106
Terabytes (TBs)) of data centre to user traffic makes already most of 360 EBs of the
global IP traffic [20], [23]. Thus, it can be inferred that the Internet traffic is dominated by
more than 70% by traffic from data centre to user; remaining 25-30% of the global IP
traffic is generated by P2P applications, whereas other important applications share like
VoIP is negligible. Global data centre IP traffic is forecast to nearly quadruple and grow by
31% CAGR over the next 5 years.
Table 3-1: Data centre traffic trends for 2011-2016 [20].
Public

Moreover, the three categories of traffic within, between and from the data centres to the
users are estimated to grow at about the same pace, whereas virtualization is foreseen as
a main driver of the traffic between data centres provided by the same or different entities,
e.g., cloud operators. Therefore, a main challenge for energy efficient traffic management
lies in an optimized distribution of data over traditional and cloud data centres with regard
to short transport paths from data centre to the end-user and between data centres
themselves. Note that traditional data centres are defined in [20] as those whose
operation is associated with non-cloud consumer and business applications.
Furthermore, w.r.t. segment and type categorization, it can be observed that consumer
traffic is driven by CAGR of around 30%, while a lower growth rate of 23% is assumed for
business traffic, such that consumer traffic for the mass market of private Internet access
subscribers will exceed business traffic 6-fold in 2016. Moreover, a comparison of cloud
and traditional data centre traffic attributes an essentially higher growth of 44% for clouds,
whereas only 17% for traditional data centres. As a consequence, clouds are expected to
overtake traditional data centre traffic almost 2-fold until 2016.
3.6.3 Cloud Traffic Statistics and Trends
The Cisco Global Cloud Index forecasts the continued transition of workloads from
traditional data centres to cloud data centres. Due to the workload transmission, the traffic
generated by clouds is expected to scale at a higher rate of 44% CAGR compared to 31%
for global data centre traffic, while traditional data centre traffic growth is significantly
smaller (see Figure 3-6). Global cloud IP traffic is forecast to increase 6 times and grow by
44% CAGR over the next 5 years. Moreover, cloud traffic crossed the Zettabyte threshold
in 2012, and by 2016, nearly two-thirds of all data centre traffic are expected to be based
in the cloud. Significant drivers of cloud traffic growth are the rapid adoption of and
migration to cloud architectures, along with the ability of cloud data centres to handle
significantly higher traffic loads.
Cloud data centres can be distinguished in two categories based on services delivered to
the end-users, i.e., business and consumer ones. Business data centres are typically
dedicated to organizational needs and handle traffic only for their company's own needs
that may adhere to stronger security guidelines. On the other hand, consumer data
centres typically cater to a wider audience and handle traffic for the mass consumer base.
Public

Figure 3-6: Cloud vs. traditional data centre traffic growth 2011-2016 [20].
Considering forecasts of cloud traffic generated by data centres, consumer data centres'
traffic is expected to grow with a CAGR of 46%, and to reach 3,659 EBs by 2016, while
business cloud traffic is expected to grow at a CAGR of 37%, increasing to 596 EBs by
2016. Table 3-2 summarizes details for global consumer and business cloud traffic CAGR.
Table 3-2: Cloud IP traffic trends 2011-2016 [20].

Concerning the business segment, the necessity to provide fast and flexible access to
large data archives is an important objective for IT organizations considering cloud-based
solutions. In addition, enabling advanced analytics to tap into the wealth of information
contained in largely unstructured data archives can create a valuable competitive business
advantage. Moreover, enhanced collaboration services delivered through the cloud can
increase employee productivity and customer satisfaction.
On the other hand, regarding the consumer segment, applications such as video and
audio streaming are strong factors in cloud traffic growth, while emerging services such as
personal content lockers are also gaining in popularity. In personal content lockers, i.e.,
online storage applications such as Dropbox, users can store and share music, photos,
and videos through an easy-to-use interface at relatively low or no cost. Especially related
to personal cloud storage, Cisco GCI [20] forecasts that personal cloud traffic will increase
from 0.6 EBs annually in 2011 to 25 EBs in 2016, at a CAGR of 111%. Furthermore, the
proliferation of tablets, smart phones, and other mobile devices which allow access to
such applications in a manner convenient to the end-user, further contribute to the high
growth rate of the global cloud traffic.
Public

Finally, Ipoque statistics [141] have also been considered in order to have a wider
overview of internet traffic characterization; though since they are pertaining only up to
2009, they will not be considered as relevant for our current investigation.
3.6.4 Characteristics of Communication between Cloud Data Centres
Traditional network infrastructure between data centres is organized in a hierarchical
structure which is composed of at least three levels: access level, distribution level and
core level. In this structure, the typical traffic flow is north/south-based, i.e., from server
racks (access layer) to L3 high capacity core switches (acting as root bridges for spanning
tree protocol). However, within the last years, data centres' inter-connection is undergoing
a deep shift towards new design due to the rise of a number of issues, i.e.,: rapidly rising
traffic volumes associated with data distribution and synchronization operations, data
transfer operations in service contexts such as content delivery or storage, high variability
of bandwidth requirements between peak and mean traffic rates, and increased number of
east/west traffic flows (i.e., server-to-server). The changing in the actual data centre inter-
connection is led by the rising of new services that the new generation of data centres
must be able to provide, e.g.,:
- Storage and data replication for IT disaster recovery strategy based on a
geographical diversity approach. Such services initiate relatively small number of
flows or connections, though they generate very high traffic throughput per flow.
- VM migration, with VM disk space either asynchronously replicated to the new data
centre or accessed from the original data centre over the ISP's WAN.
- Synchronous replication between data centres (active-active storage replication),
which allows data to reside at multiple locations and to be actively accessed by
VMs at all sites.
Within a single ISP and data-centre provider, inter data centre communications generally
rely on traditional L2VPN or IPMPLS/VPN services. The new services offered imply a
massive amount of data moved across multiple and geographical distributed data centres.
The peak traffic volumes between data centres are dominated by background, non-
interactive, bulk data transfers due to backup and replication applications running to
transfer bulk data between distributed data centres.
Additionally, the inter data centre communication is significantly affected by some critical
factors; one of which is the bandwidth allocation of the underlying transport structure. Most
cloud operators use a fixed bandwidth allocation (mainly tailored on an upper limit of their
transmission needs), which may result in non-optimal resource allocation since the
variability of bandwidth requirements between peak and mean traffic rates is rapidly
increasing over the years. The actual mitigation actions mainly deal with load balancing
over multi-path, multi-hop Store-and-Forward (SnF) scheduling techniques.
Apart from bandwidth, latency is also a crucial metric for the east/west traffic. Many
Internet services and application provided over the cloud are deployed in highly distributed
environments where data are stored in different locations. Usually they have a limited time
window to fetch requested data, process them and send them back to the requesting
entity. Delay and other problems related to bandwidth limitations may disqualify such
services and applications.
Furthermore, another issue to be considered is related to cloud traffic patterns due to
virtual switching operations. These operations are performed by hypervisors to forward the
Public

traffic between VMs and external network switches. It is highly efficient (i.e., in terms of
low latency), when VMs communicate with each other or external switches without
involving external switching. However, there are also some drawbacks; for instance
internal switching consumes CPU that may limit capability to host more VMs.
Nevertheless, the most significant drawback which is also related to SmartenITs scope is
the inconsistency of internal switching with network management and monitoring of
external network infrastructure by network providers; such an inconsistency may lead to
conflicts in policies and security issues.
Finally, the rapidly growing number of mobile devices (e.g., smartphones and tablets)
used to access new overlay applications has an important influence on how application
ecosystems are implemented nowadays. For all devices that contain only thin graphical
user interfaces, all heavy operations and data storages are placed in the cloud, while the
allocation of resources is transparent for users. Because of shifting the workload toward
the cloud infrastructure, computation needs as well as the power consumption are
optimized. However, high bandwidth Internet access is required for the end-users mobile
devices, while the shifted workload further burdens the cloud, and highly affects data
centre to user, as well as inter data centre communication.
3.7 Characteristics of Energy Consumption by Cloud Services
Energy efficiency is fundamental as a key objective of SmartenIT; i.e., we aim to design
traffic management mechanisms that will achieve low energy consumption, or else energy
efficiency in data centres, ISPs' networks or in mobile devices. In order to address the
energy efficiency of cloud, we first need to capture the current state. Thus, the following
sections will analyse the energy consumption issues related to three main areas, i.e., data
centres, ISPs' networks, and mobile devices, describing also measurements methods at
different stages and energy wasting, thus highlighting possible points on and opportunities
for intervention for SmartenIT purposes. Moreover, the energy consumption survey
provided in this section will provide input to the overall assessment of cloud-based
applications to be performed in Chapter 4.
3.7.1 Data Centres
Cloud computing has proven to be a highly scalable and cost-effective solution for the ICT
industry. Hence, cloud service providers are making significant investments in data centre
infrastructure so as to provide a wide range of cloud services, business and commercial
applications for their customers. However, clouds are composed essentially of data
centres, which are being built at ever-larger scales and with increased server density,
resulting in greater energy consumption. High energy consumption implies though high
energy cost, which constitutes a significant part of a data centre's operating cost as
reported in the next section, leading thus to an erosion of cloud providers revenues. As
expected, cloud operators are highly interested in energy efficiency techniques to
minimize incurred costs, and maximize their benefit.
Nevertheless, the energy efficiency in data centres and clouds is being investigated by
various EU-funded research projects as energy awareness is one of the priorities of the
European Union. Some of those that may also provide valuable input for SmartenIT are
presented in Table 3-3; some of the solutions proposed by the projects presented below
are also briefly overviewed in Deliverable D2.1 [203], while Energy Efficiency is also to be
addressed in Deliverable D1.2.
Public

Table 3-3: Summary of initiatives for energy efficiency in data centres.
Project name Goal of the project Description
GAMES - Green Active
Management of Energy
in IT Service centres
Energy-efficient adaptive
data centres
The project works on energy efficiency along with
QoS and IT resource utilization and performance.
FIT4Green - Energy-
aware ICT optimization
policies
Allocation of ICT
resources in data centres
to minimize energy
consumption.
The project has created an energy-aware layer of
plug-in on top of the current data centres
management tools.
CoolEmAll Tools and knowledge to
enable data centre
designers and operators
to plan and run facilities
more energy efficiently.
The project works on advanced simulation,
visualization and decision support tools along with
blueprints of computing building blocks for modular
data centre environments
All4Green Energy saving policies
and energy demand
patterns in data centres.
The project works on a definition of suitable SLA
including energy-efficiency aspects. A key element
of the investigated architecture, procedures and
functionalities is the collaboration (interface) with
energy producers/providers for energy
consumption optimization.
Thus, energy consumption is a critical concern for IT organizations worldwide as the cost
of operating data centres increases due to the growing use of computing devices and
rising energy costs. In the sections below, we provide measurements, future projections
and break-down of the energy consumption by data centres. Additionally, and we address
potential for improvement, i.e., reduction of the energy consumed due to typical data
centre operations.
3.7.1.1 Measurement Studies
In 2006, data centres consumed in the US 1.5% of the total electricity [151]. Moreover, it
was estimated that the energy consumption would double in the next 5 years. The reason
for this prediction is an increasing use of Internet services such as online banking, or
media services. However, a new study [152] from 2011 shows that the energy
consumption for data centres did not increase as predicted. In [153], although the IP traffic
was expected to grow by 46% per year, the energy consumption is estimated to increase
by up to 36% in the US in 5 years, from 2005 to 2010. The main reasons according to
[152] are the use of virtualization that leads to better hardware utilization. Furthermore, the
economic turbulences in 2008 led to a fewer investments in new servers, and the overall
electricity usage did not grow as fast as predicted in the US. Figure 3-7 depicts the
predictions by [151] and [152] (indicated with red arrows).
Another observation by [152] is that Google, although having many servers, only
contributes 1% to the electricity used by data centres worldwide. However, since Google
does not publish the number of their servers, these numbers are estimations. One reason
for this low value is that Google assembles its own servers and can optimize its servers
and data centres to reach higher infrastructure efficiency. Infrastructure efficiency is
typically measured by Power Usage Effectiveness (PUE), which is the ratio of total amount
of power used by a data centre to the power used by the IT equipment. Google's data
centres have a PUE of 1.13 in 2012, while a value of 1 is ideal [153].
Public

Figure 3-7: Predicted US electricity use for data centres from the EPA report to Congress
[151] and the range estimated by Koomey [152].
According to [139], power is wasted at different stages of data centre operation. A typical
breakdown of data centre energy overheads is (also depicted in Figure 3-8):
- Network-Critical Physical Infrastructure (NCPI) equipment is responsible for
approximately the 70% of total data centre energy consumption; 45% of which is
due to cooling infrastructure and 25% for power delivery units.
- The remaining 30% of total energy is delivered to IT equipment for critical
operations such as hard disk drive operations, networking, memory, CPU,
motherboards, fans, etc.
3.7.1.2 Potential for Improvement of Energy Efficiency in Data Centres
In [139], it is stated that PUE of data centres in 2006 was on average around 2.0, namely
2.0 times more energy is consumed in total by a data centre than the amount of energy
delivered to IT equipment. Moreover, in [139], it was suggested that the energy
consumption can be saved by continuing the trend of consolidation and virtualization,
enabling power management on servers and equipment, and improve data centre cooling.
Typically, facility equipment power load can be monitored, installing energy meters into the
data centre. They return the total electrical energy utilization by the units to which they are
intended for. The meters are usually located at the distribution panel supplying power to
the Precision Air Conditioning PAC units.
Facility networking in a data centre, is widely acknowledged as a promising technology for
energy saving. For this reason, another value which must be controlled is the network
energy consumption. It is associated to the network infrastructure cabled inside the IT
Service Centre, composed of switches, routers, etc. In this case, the kind of data which
must be monitored could be the traffic rate of the network, its maximum and minimum
speed, along with the measurement of the energy consumption by means of appropriate
physical sensors.
Public

Figure 3-8: Breakdown of data centre energy overheads [139].
The recently emerged cloud computing paradigm leverages virtualization technology and
provides the ability to provision resources on-demand on the pay-as-you-go basis.
Organizations can outsource their computation needs to the Cloud, thereby eliminating the
necessity to maintain their own computing infrastructure.
Virtualization perfectly addresses the requirements of energy efficiency in data centres.
The resources of a VM can be provisioned dynamically as the workload demands change
for an application. Virtualization is the most adopted power management and resource
allocation technique used by the data centre operators. It is implemented in both the
server and switch domain but with different objectives. Server domain virtualization usually
achieves energy efficiency by sharing limited resources among different applications. On
the other hand, virtualization not provide for energy efficiency in the network domain. In
effect, network resources are burdened by the virtualization techniques, since live
migration of VMs between physical hosts generates a significant amount of traffic.
The overall key benefits for a data centre owner can be summarized as follows: energy
costs reduction by up to 80%, dynamic power off servers without affecting applications or
end-users, and greening of IT infrastructure, while also improving reliability, availability and
service levels.
SmartenIT's potential of intervention can be mainly related to the management of traffic
generated by live VMs migration from data centre to data centre, migration that is strictly
tied to data centre resources orchestration and energy efficiency achievements.
Public

3.7.2 ISPs' Networks
In ISP networks, the energy consumption has increased with traffic volume and popularity
of Internet applications and meanwhile is a critical issue for further traffic and bandwidth
growth [196]. Fiber optic links provide high bandwidth in fixed networks without electrical
power supply but the switching and routing still requires more and more energy. In core
routing facilities, the energy demand is becoming a limiting factor for bandwidth and traffic
increase, if upgrades of IP transport capacity imply a similar increase factor in capacity as
well as in energy consumption [197]. Moreover, mobile communication is also subject to a
tremendous increase in traffic and energy consumption with regard to network elements
and devices. Section 3.7.3 is discussing the latter area in more detail.
Therefore traffic management and options to reduce the traffic load are crucial as well as
energy saving technologies. Higher resource utilization means service for more users
based on the same equipment and OPEX, part of which is energy costs [194]. Caching
infrastructure within ISPs is useful to shorten traffic paths in content delivery [195]. Their
benefits in transport capacity savings and lower delays have to be balanced against
required data centre and server resources, which again consume energy. Flexible control
and (re-)direction of traffic flows can also reduce over-provisioning. QoS differentiation can
help to run a network at higher load and higher risk for congestion while sufficient service
quality is still met for the most important and critical applications.
Moreover, there have been proposed options for switching off parts of the network
equipment in phases of lower load and reactivating them on demand. This constitutes an
another way of energy saving, which is often not feasible currently due to long set up
periods of networking equipment. On the other hand peak-to-mean ratios of daily traffic
profiles are often in the order of 2 or larger and resilience in core networks is often
achieved by doubled network topologies which could be kept in a less hot standby mode
at lower energy consumption. When usage is highly varying over time e.g. between office,
shops and residential areas or for mobile ad hoc network, hotspots for event sites etc.
then the energy saving potential by on demand provisioning is high as well.
3.7.3 Mobile Devices
Measuring energy consumption of mobile devices and smartphones in particular is a
challenging task. On the one hand, the built-in hardware, which offers a low precision and
sampling rate, might be sufficient in some cases. However, for low-level experiments, e.g.,
frame-based measurements of the wireless interface, more precise measurement
equipment is needed. On the other hand, external measurement hardware is expensive
and counteracts the idea of mobility, preventing devices from moving as intended and
probably needed by experimenters. Additionally, both methods do not allow to measure
single components like the WiFi chip or the consumption of single applications (apps).
In the next sections, we briefly overview methods for measuring energy consumption and
provide results of a number of recent measurement studies on the aspect of energy
efficiency in mobile devices.
3.7.3.1 Measurement Methods
Recently developed energy models as those described in [98], [99], [100], and [101] try to
fill the gap. The common idea of these models is to estimate the current energy
consumption of the device based on a number of data sources provided by the Operating
System (OS). These sources can comprise the actual CPU frequency, the data rate of the
Public

radio chip, and LCD brightness [98], [99] or are based on system calls, e.g., calls related
to network sockets, which are triggered by an application [100], [101]. Thus, the estimate
can be split up to components and apps, allowing a precise assessment of code quality
with respect to energy consumption.
The model parameters are obtained by correlating the data sources and the actual energy
consumption as measured by external measurement hardware and are thus device-
specific. However, in test beds with homogeneous resources, energy models provide the
unique possibility to measure a larger number of devices without the need of additional
hardware. In [102], a system is proposed to measure, collect, and visualize the energy
consumption of P2P streaming processes of mobile devices in a precise manner at
runtime, using the model presented in [99][99]. Figure 3-9 shows the interaction of the
developer and measurement platform, where the lower half is mapped to a central server
entity collecting the data from a number of Android devices and allowing statistical
evaluation using a web frontend.

Figure 3-9: Interaction of measurement platform and developer [102].
3.7.3.2 Measurement Studies
Energy is a scarce good on mobile devices. Thus, a number of recent measurement
studies focus on the aspect of energy efficiency. The following paragraphs discuss the
relevance of the consumption of network related hardware components. This is done in
order to substantiate the necessity of taking energy efficiency into account, when
designing network related algorithms as planned within the scope of SmartenIT.
Besides CPU, RAM and GPS, the network interface is particularly relevant, as mobile
users heavily rely on network access. In [103], a study of the user behaviour of 255 users
measures nearly 50% of all user/smartphone interactions being related to communication,
i.e., to email, SMS, instant messaging and voice calls, not even including browsing, i.e.,
12%, and media consumption, i.e., 9%. The relationship of energy-efficiency and usage
pattern is not evaluated in this study. However, in [98], the power consumption of week-
long traces of 20 users is broken down by components using a model-based
measurement approach. The average aggregated consumption of networking components
(WiFi, edge, voice) accounts for roughly 25% of the overall consumption (see Figure
3-10), while the maximum is at ~40%.
A non-user centric insight into the energy consumption of smartphones can be given by
analyzing models to estimate the energy-efficiency as described beforehand. These have
shown communication related hardware to belong to the main consumers of energy ([99],
[98], [100], [101]).
Public

Figure 3-10: Power breakdown of 20 users by hardware component, taken from [98].

Figure 3-11: DFA representing the WiFi energy consumption as modelled in [100].
The models presented in [98] and [99] are rather simplistic linear regression models. The
authors find the WiFi/GSM chipset to be a main contributor. In [100], [101], and [102], the
energy consumption of a complete smartphone is modeled and validated, using a system
call based approach. An interesting aspect of this work is the modeling of wireless
networking activity as Deterministic Finite Automatons (DFA); an example is depicted in
Figure 3-11. These automatons reflect the interfaces power states, which dominate the
energy consumption, while the actual data rate has minor influence. Another interesting
finding is the existence of so-called tail states: to avoid the frequent oscillation of power
states, the hardware stays in a high power state to for a defined amount of time even
though no data is sent [101].
3.7.3.3 Methods for Improvement of Energy Efficiency of a Network Application
Currently, the literature on methods for energy-efficient network access on mobile phones
can be divided roughly into two categories. Offloading/prefetching approaches try to
minimize energy consumption by doing a handover between transmission technologies.
These approaches utilize the tradeoff of transmission range and energy consumption:
cellular technologies like HSDPA or LTE offer wide range communication at a high price in
terms of energy, whereas low range technologies like WiFi or Bluetooth have a
comparably low energy-footprint and can often provide higher data rates. An example of
such an approach is given in [104].
Public

Figure 3-12: Energy consumption of browser search; the green indicates the percentage
of energy wasted in tail states [100].
As opposed to offloading approaches, hardware-aware approaches embed knowledge of
hardware properties into the application layer, e.g., power models of the WiFi or 3G chip.
In [100], several applications are studied for energy consumption in tail states. Depending
on the application, this amount is remarkably high (e.g., for Google search on a browser,
about 30% of energy is wasted in tail states, see Figure 3-12). Sophisticated aggregation
schemes as presented in [103] can help to avoid the frequent occurrence of these states.
3.8 Summary of Cloud Characteristics
In this chapter, we provided cloud basics, i.e., definitions of major terms related to cloud
services and applications. Additionally, we described fundamental deployment models and
economic and QoS aspects of cloud computing, to develop a common background on the
impact of cloud computing on the various Internet stakeholders. As a next step, in Section
3.5, we overviewed a variety of cloud services offered by well-known major cloud
operators, as the so-called Internet giants, such as Amazon and Google, as most cloud-
based applications are built upon those cloud services, e.g., Dropbox employs both
Amazon's EC2 and S3 to provide its personal online storage service. This description will
allow us to understand the architecture and operation of the cloud-based overlay
applications described in Chapter 4, to define realistic scenarios of cloud service provision,
and last to identify possible stakeholders corresponding to real operators described in
Chapter 5.
Next, in Section 3.6, we overviewed various sources addressing traffic generated by data
centres and cloud both as a share of global IP traffic, and in terms of absolute traffic
volumes, popularity and future trends. The main conclusions and future projections are as
follows: Global data centre IP traffic will nearly quadruple, while it will grow by 31% CAGR
over the next 5 years. On the other hand, global cloud IP traffic will increase 6 times and
grow by 44% CAGR over the next 5 years, while global cloud IP traffic will account for
nearly two-thirds of total data centre traffic by 2016. Moreover, the rapidly growing number
of mobile devices (e.g., smartphones and tablets) used to access new overlay applications
constitutes a key driver of the cloud traffic increase. Specifically, it has an important
influence on how application ecosystems are implemented nowadays, namely,
applications 'run' in the cloud (i.e., computation, data storage), while only a 'shell' providing
an interface to access the application remains at the mobile device.
Then, in Section 3.7, we overviewed studied addressing the characteristics of energy
consumption by data centres, ISPs' networks and end-users' mobile devices. Our major
Public

conclusions for data centres are as follows: Although the IP traffic was expected to grow
by 46% per year in the last 5 years, the energy consumption for data centres did not
increase as predicted. In particular, it has been estimated to have increased by up to 36%
in the US, due to the use of virtualization that leads to more efficient hardware utilization.
Moreover, currently, only, 30% of total energy is delivered to IT equipment for critical
operations such as hard disk drive operations, networking, memory, CPU, motherboards,
fans, etc., implying PUE equal to 3.3. On the other hand, the remaining 70% of total data
centre energy consumption splits to 45% for cooling infrastructure and 25% for power
delivery units.
bandwidth growth. Fibre optic links and equipment consume more energy than older
equipment for switching and routing, while caching infrastructure within ISPs, which is
increasingly adopted to shorten traffic paths in content delivery, leads to installation and
operation of small data centres and server resources in ISP's premises, which again
consume energy. Moreover, energy saving by switching off parts of the network equipment
in phases of lower load and reactivating them on demand is often not feasible currently
due to long set up periods of networking equipment.
On the other hand, considering energy implications on users' mobile devices of all
user/smartphone interactions have been recorded as being related to communication, a)
email, SMS, instant messaging and voice calls, nearly 50%, b) browsing, i.e., 12%, and c)
media consumption, i.e., 9%. Proposed approaches that aim to optimize the consumed
energy utilize the trade-off of transmission range and energy consumption. For instance,
terms of energy, whereas low range technologies like WiFi or Bluetooth have a relatively
low energy-footprint and can provide higher data rates.
The investigation in Section 3.6 and Section 3.7, along with the studies performed in
Chapter 4 addressing specific characteristics of various application categories, e.g., online
storage, and specific examples of them, e.g., Dropbox, will allow us to assess the various
cloud applications (in the end of Chapter 4) w.r.t. to their suitability to be selected as
SmartenIT's key applications, whose traffic is to be handled by mechanisms to be
developed in WP2.
Public

4 Characteristics of Cloud Applications
In this chapter, we provide an overview of a multitude of overlay applications, both highly
popular and well-established and new emerging ones, which all share the characteristic,
i.e., they are provided over the cloud, namely their entire functionality or part of it is based
on some cloud service such as cloud storage or cloud computation. Specifically, we
describe these applications from multiple viewpoints, in terms of their operation, traffic
characteristics and energy consumption. Additionally, taking also into account the
terminal, i.e., the device of the end-user, allowing the access to such cloud-based
applications, we explicitly address categories of applications that are specific for mobile
devices, such as mobile video streaming and mobile P2P file sharing.
Our ultimate goal is to highlight relevant to SmartenIT characteristics of new overlay
applications, i.e., related to the main concepts and scenarios of applications, traffic
generated by them, and potentially the social information that can be extracted from them.
These characteristics will play fundamental role in the selection of those applications that
are to be addressed by SmartenIT (in WP1), the description of interesting scenarios and
use-cases, the design of new socially-aware, energy-efficient traffic management
mechanisms (in WP2), and the development of SmartenIT's architecture (in WP3).
Identification of traffic characteristics, mainly referring to traffic volume generated, as well
as traffic patterns, e.g., variability and burstiness, and QoE requirements, where available,
is an important task done permanently by network operators and administrators. Besides
theoretical aspects of traffic analysis, carried in order to study network-related phenomena
and to follow evolution of services and user behavior, such analysis helps to solve urgent
problems (such as failures or lack of resources) and support traffic management and long-
term network planning. There are long-lasting efforts to characterize traffic in the Internet.
These investigations profit from previous and ongoing efforts to characterize traffic in the
network. The goal is to recognize traffic flows, characteristics of single flows and also
characteristics of multiplexed traffic composed of many flows.
The set of applications presented below is representative of the contemporary overlay and
social networking and in some cases such applications were used by SmartenIT partners
for thorough investigations, based on studying application architecture, service scenario,
possibility to selectively monitor or register relevant traffic flows for immediate inspection
and also for off-line analysis. The description of an application should give a
comprehensive view of its main features, popularity, opportunity for intervention and
optimization. Defining such applications features more precisely we aim at describing its:
Category: to which service/application it belongs, e.g., online storage, search
engine, etc.
Operation: how the application works, what other cloud services it employs,
Popularity, dynamics: known data about evolution, number of users, perspectives.
Openness: open code, possibility of intervening,
Potential for SmartenIT intervention/optimization: jointly consideration of data and
details concerning application with openness of source code (or protocol)
suggestion possibility to influence the service.
Applicability for SmartenIT: expression of interest for SmartenIT with some hints
about level of relevance and the way of influencing the application performance.
Public

4.1 Video Streaming
According to Cisco traffic measurement studies [23], video traffic constitutes 51% of global
consumer IP traffic in 2011, and it is forecast to reach 55% of global IP traffic by 2016.
Moreover, according to recent studies (e.g., [23], [68]), mobile video streaming traffic
accounts for up to 50% of the aggregated traffic in mobile networks at peak hours in North
America and the Asia-Pacific region. In Europe, its share is lower with around 35% of
aggregated traffic. In all three regions, YouTube is either leading the ranking in total
downstream traffic or is among the top three traffic sources. This does not include video
exchanged through P2P file sharing. The sum of all forms of video (TV, Video on
Demand, Internet, and P2P) will be approximately 86% of global consumer traffic by 2016.
In particular, it is said that it would take over 6 million years to watch the amount of video
that will cross global IP networks each month in 2016. Every second, 1.2 million minutes
of video content will cross the network in 2016.
In this section, we focus on Internet video streaming, which was measured to generate up
to 72% of total video consumer traffic in 2011, where both short, i.e., UGC and video clips
up to 7 minutes, and long videos, i.e., generally videos exceeding 7 minutes in duration,
have been considered. Moreover, video streaming traffic will constitute up to 46% of total
consumer video traffic by 2016. Note that although video streaming traffic is forecast to
present positive CAGR, i.e., 34% for short video and 22% for long, the share w.r.t. to total
traffic will be reduced due to the fact that other types of video traffic will increase with
much higher CAGR, i.e., 45% for Internet to TV traffic, and 90% for mobile video traffic.
4.1.1 YouTube
In this section, we provide a description of the functionality, traffic characteristics and QoE
implications of YouTube, as the most popular application both for PCs and mobile
applications, which generates enormous traffic volumes in the Internet. Moreover, an in-
depth analysis of YouTube is provided in Appendix A.
YouTube is a video streaming platform and its portal (i.e., YouTube.com) is one of the
main Internet portals for Video-on-Demand (VoD), as well as one of the most popular sites
globally. Currently, only Google (i.e., Google.com) and Facebook (i.e., Facebook.com) are
more popular than YouTube according to the Alexa Ranking [24]. The video content that is
available on this platform is generated by users. This means that YouTube does not
distribute its own videos. Instead, it offers a platform where users can upload videos and
share them with their friends or the public.
This service is free of charge for the users, i.e., every user can upload and watch the
videos without any costs additional to the Internet access. Therefore, a large number of
people and companies upload short videos clips, for example product advertisements. In
contrast, currently popular movies or songs are not available at YouTube due to copyright
protections. For that purpose, other VoD portals exist, which charge a fee for their service
and for the copyright owners.
In the YouTube service, three different stakeholders are involved. These are the YouTube
platform owner, the ISPs and the users. While the ISPs and YouTube aim at increasing
their profit, the users are interested in a high QoE when watching the video. The ISPs sell
connectivity to their customers and this is their main source of revenues. However,
YouTube (or Google) try to establish peering agreements with other ISPs so that they can
exchange traffic without additional fees.
Public

The user experience when watching a video might be degraded by long startup delays or
stalling events [55]. Therefore, these two aspects are the main QoE indicators. In order to
guarantee a high QoE, the connection between YouTube servers and end-user clients
needs to support the required bandwidth. YouTube videos are mostly watched in 360p
resolution (which is the default setting) and have a bit-rate of 0.1 and 1 Mb/s. Since
YouTube is a VoD service, latency and jitter or not so important. The reason is that the
client software tries to download the parts of the video up to 60 seconds before they are
watched.
Recently, YouTube gets also more and more popular on mobile devices; in particular,
according to [21], 27.8% of all YouTube traffic is consumed on a smartphone or tablet.
There are YouTube apps for the Android operating system as well as for Apples iOS. In
such mobile scenarios, the users might accept some quality degradations compared to
watching the video on their computers at home. However, energy efficiency becomes a
major issue for the QoE since users can get dissatisfied if watching a video clip on their
smartphone drains the battery very fast.
Possible points of intervention for optimizing the YouTube service are rather limited since
the whole system is proprietary. From outside, it is neither possible to change the sending
behaviour of YouTube nor to change the distribution of videos, i.e., which videos are
stored on which servers. Furthermore, the client playback software is not open source and
cannot be modified. Hence, the only point of intervention is the delivery path of the videos.
On this path, the ISPs can intervene, e.g., by caching popular videos or by smoothing the
traffic. The latter one could also be performed by the TCP/IP stack of the operating
system but this is rather a complicated task.
Energy efficiency plays mainly a role in the YouTube data centres and on mobile user
devices. However, there is only little information available about the concrete structure of
YouTube data centres and how their energy efficiency could be improved. Therefore, it
might be difficult to investigate energy efficiency issues in the context of YouTube.
4.1.2 Netflix
Netflix is a major American streaming video provider, operating in both Americas, and a
few European countries [189]. Over 23 million users account for 29.7% of peak
downstream traffic in the United States, making it bigger than YouTube.
Netflix architecture includes data centre, services that track registration process, hold
payment information and redirect users to respective servers for other Netflix functions.
Amazon cloud handles many key functions like sign-in process, content ingestion, CDN
routing, Digital Rights Management (DRM), logging and mobile device support. For main
functionality content delivery, three CDNs are used: Akamai, Limelight and Level-3. To
receive and watch video content, a dedicated player Silverlight is used.
Data streaming is performed by Dynamic Streaming over HTTP (DASH) protocol. The
protocol encodes video content and splits it into a few second-long chunks. Each
download is measured in order to determine appropriate quality for the next fragment.
While video streaming is done by CDNs alone, the user experience is reported periodically
(once every 60+ seconds) to a control server within the Amazon cloud
Choice of a CDN is based on a ranking in a manifest file received over SSL and is
independent of content type, length or location. Preference seems to depend on user
account and can remain unchanged for days [188]. Once the CDN is selected it remains
Public

unchanged, even if a different CDN could offer a higher bandwidth, change occurs only in
case when bandwidth drops below the threshold value that can support a lowest quality
video stream. While it reduces frequent changes in network topology it seems that
bandwidth usage is sub-optimal. This can be seen as opportunity for SmartenIT solutions.
4.1.3 Hulu
Hulu is a video streaming service that operates in United States and Japan [191]. It offers
a variety of video content ranging from movies and TV shows to web series, trailers and
behind-the-scenes materials from popular TV networks. Majority of Hulu download is the
free video streaming for desktop users, but paid subscription service is also available, it
enables HD quality of videos and support of transfer to mobile devices.
Most of the video content is streamed at 480kbps and 700kbps, but some of the videos
can be streamed at 1000Kbps. HD quality is limited to subscribers of HuluPlus. The users
are assigned to one of three CDNs (Akamai, Limelight or Level3), then IP address of
server is selected via hostname and DNS. Content is delivered via encrypted Real Time
Messaging Protocol (RTMP) on port 1935 when using Level3, or tunnelled over HTTP
(RTMPT) when using Akamai, Limelight or when port 1935 is blocked.
The choice of a specific CDN depends on the manifest file, obtained from s.hulu.com,
which contains arranged preference list, server location, available bitrates and other
information. There are very frequent preference changes between three CDNs but they
are unrelated to the content, user location or time slot. It appears that the preferences are
randomized in a way that directs 25% of traffic to Limelight, 28% to Akamai and 47% to
Level3 [190]. Preferences of CDN are subject to frequent changes. Change of the CDN
takes place only when the currently used one cannot support even the lowest requested
quality.
Once a CDN is selected, a specific server must be selected. All CDNs serve desktop
users while mobile users are served only by Akamai and Level3, finally Hulu
advertisement service is served only by Akamai and Limelight. All CDNs use locality-
aware DNS resolution, so general hostname (e.g., cp39466.edgefcs.net for Akamai
hostname will return one of 1178 IP addresses depending on user location).
It is important to note that CDN infrastructure and operation involves significant costs and
necessary expertise. Thats why Hulu, like many other service providers, rely on third-party
content distribution networks. Three CDNs used by Hulu are used by other online video
services with the same streaming technologies.
Hulu is already managed to some degree, by influencing user preferences towards CDNs.
That can limit SmartenIT potential influence. However energy efficiency aspect may be of
some interest.
4.1.4 Relevance for SmartenIT
As already pointed out in the beginning of Section 4.1, video streaming traffic constitutes a
large share of total consumer traffic, and thus an excellent candidate for SmartenIT to
address. Specifically, YouTube, as the most prominent example of this category of
applications, is provided over a proprietary platform, thus a closed system, we need to
consider possible points of intervention for SmartenIT and the potential for optimization for
such traffic.
Public

Since video streaming has such a large share of traffic, the prior target for optimization is
traffic management. As stated in the previous subsections the streaming service providers
YouTube, Netflix and Hulu already use CDNs to efficiently distribute their content.
Therefore the traffic optimization potential has to be investigated for each platform.
Although the CDNs bring content already close to users, local caches at ISPs, hybrid
peer-assisted distribution of popular videos or intelligent replica placement strategies
could further reduce global video streaming traffic and reduce transit costs for ISPs.
Therefore the current Internet structure with reduced upload bandwidth of end-users and
peering agreements with access networks have to be considered.
As described in [188] Netflix directs video requests to CDNs independent of content type,
length or location. Hence, there is still a high potential for optimization, since the type of
content determines the propagation by social media and locality is not taken into account.
Both Hulu [190] and YouTube (see Section 4.1.1) already use locally aware DNS-
resolution. Hulu uses third party CDNs for content distribution, whereas YouTube uses
primarily the global CDN of Google, but also offers ISPs to integrate YouTube caches in
their premises.
The options to intervene in the video delivery of YouTube are limited, if there is no
collaboration with YouTube. Without access to the content delivery network of YouTube,
intervention is for example possible for the ISPs carrying the data or in the operating
system of the clients. For ISPs possible interventions could be to prioritize the video traffic
over file downloads, where the QoE is less sensitive to bandwidth variations. In addition,
dynamic resource management strategies might be possible. Concretely, ISPs could
monitor the YouTube traffic flows and estimate the current fill state of the playback buffer
at the client based on the knowledge about the YouTube flow control detailed in A.3. In
case that this buffer is about to become empty, the ISPs can dynamically allocate more
resources to this particular flow and avoid stalling in this way. Furthermore ISPs could
exploit information of social networks to predict the demand of specific content and
optimize the hit-rates of their local caches.
The QoE findings could be exploited in the following way. In case of high network load and
scarce bandwidth resource, the initial delay (which is less QoE critical) could be increased
in order to avoid the QoE critical stalling during the video. However, this requires the
cooperation of the YouTube player and the servers as well as the ISP. Finally, in the
operating system of the client a possible point of intervention might be to change the TCP
flow control.
4.2 Mobile P2P Video Streaming
The streaming of video data in mobile environments, in comparison to fixed network
access to this data-intensive type of content, imposes special challenges for the content
delivery process. Over the last years, the delivery of video or more general of real-time
entertainment data has seen a large adoption in this environment. As mentioned in
Section 4.1, mobile video streaming traffic accounts for up to 50% of the aggregated traffic
mobile traffic at peak hours in North America and Asia-Pacific region, and 35% in Europe.
Moreover, for the Asia-Pacific region [68] documents an additional important contributor to
mobile traffic, namely the usage of P2P-based streaming applications. Here, the upload
traffic is dominated by PPStream, while regarding the download traffic, PPStream is
beyond the top three contributors [69] (see Table 4-1). Other examples of widely used
P2P-based streaming applications are PPLive [70], CoolStreaming [71], and SopCast [72].
Public

Table 4-1: Top 10 applications in the Asia-Pacific region at peak time [68]

In this section, we focus on two prominent examples of mobile P2P video streaming which
are among the top three traffic contributors in this category, i.e., PPLive and PPStream.
4.2.1 PPLive
PPLive along with PPStream, a very similar application which is introduced in the next
subsection (see Section 4.2.2), is one of the most well-known applications for streaming
video in a P2P fashion. Both are based in Asia and are available for both streaming to PC
as well as mobile devices. The user interfaces are mostly in Chinese and so is a large part
of the delivered content. This might be two important reasons why both applications see a
rather limited adoption in other parts of the world and currently have a mostly Asian
customer base. In the following, major characteristics of, first, PPLive are discussed to
understand general requirements and implications of P2P-based streaming applications
that could be used to define according traffic measurement and management
mechanisms in the context of SmartenIT. In the next Section 4.2.2, PPStream is
discussed as it has become a major contributor to the upload traffic of mobile networks, as
introduced before (see Table 4-1).
Spoto et al. [73] conducted a measurement study of the PPLive network and showed that
during their measurements in 2008 on average 1 million users were connected to the
network every day. PPLive offered several hundred different channels at this time,
including traditional broadcasting channels and scheduled videos. For each channel of the
network, an own P2P mesh-based overlay was constructed among the active participants.
As PPLive uses a proprietary protocol, not all mechanisms of the application are
understood. In the following, the results of a number of measurements studies, i.e., [74],
[75], and [76], are used to shed some light on the internals of the application mechanisms.
For the content distribution, PPLive divides the video streams into blocks that can be
requested individually in a pull-based manner. The streams are provided by a content
server that injects the video blocks to a channels overlay and thereby acts as source of
the stream. To retrieve the video stream, each client connects to a number of other clients
that it requests from an entity that acts like a tracker similar to those known from content
distribution protocols such as BitTorrent. Some of the contacts are used as active contacts
to request video blocks from, others are kept as candidates to replace active contacts if
Public

required. The number of contacts depends on the popularity of the channel and the
resources of the client itself. With this connection behaviour, the clients form a mesh-like
topology, where, potentially, every client can become a major traffic source and act as
multiplicator of the video traffic. Managing such client initiated traffic, therefore, can
become important for network providers, especially in mobile environments where
resources are generally scarce.
Conducting active measurements studies with distributed crawlers running on PlanetLab,
Vu et al. [76] observed that clients can react very fast to the arrival or leave of other clients
and include or remove contacts to/from their neighbourhood set within less than 15
seconds. They, furthermore, identified three different classes of packets that are used by
the PPLive client. One for class with rather small packets that are used for the
management of the overlay, one for the exchange of buffer maps to announce video block
availabilities with about 800 Bytes per packet, and another one for the actual video data
transfer with more than 1000 Bytes per packet. Another measurement study, by Ali et al.,
[77], shows similar results and compares them to the messages exchanged in SopCast,
another P2P-based streaming application.
Hei et al. [78] also conducted active and passive measurements of the PPLive network
and identified several shortcomings of the application. First of all, it causes very long start-
up delays due to its initial buffering of several ten seconds to mitigate later problems in the
distribution process, making channel switching costly in terms of playback experience.
Second, clients experience playback lags of up to several minutes, meaning that the
playback of different clients watching the same channel can be far away from each other.
This usually is an important quality criterion for the streaming of live events where users
might want to be as live and in sync as possible.
4.2.2 PPStream
The previously presented characteristics of PPLive streaming traffic (see Section 4.2.1), in
general, also apply in the same way for PPStream, a second Asia-based P2P streaming
application. Like PPLive, PPStream used a mesh-pull overlay for the data dissemination
process. In contrast to PPLive recent studies (e.g., [68]) show that PPStream has become
an even more important traffic source in mobile networks. Therefore, in the following,
some key characteristics of this application are presented.
In [156], the authors present the results of a measurement study of PPStream that was
conducted during the 18 days of the 2002 Olympics in Beijing. They deployed five own
instances of the application in different networks and used passive measurements of the
generated traffic, focusing on the management traffic of the overlay application. Therefore
they captured and analyzed all transferred peer lists, buffer-maps, and data request
messages. For their investigations, the authors focus on the playback delay, the data
scheduling mechanism and its impact on playback quality, as well as the control overhead
of the application. While works such as [157] report an average number of users per
channel of 6,000 for an average day, during the opening session of the Olympics about 7
million users were observed in this study.
Furthermore, an interesting observation was made concerning the used transport protocol:
In contrast to previous measurements in [155] and observed traffic in regular channels of
the application, for the broadcast of the Olympics, it was observed that almost exclusively
UDP was used for both video and signaling traffic. In addition, a switch to smaller
scheduling units, so called small-chunks of 1KB size (in comparison to 8KB sub-chunks in
Public

regular channels) was observed. While an increase of control overhead due to the
necessary additional pull requests was measured, this also reduced the data miss ratio,
resulting in a better playback experience for the user. The video bitrate of the Olympic
channels was measured in [156] as being 391 Kbps, whereas normal channels of
PPStream are reported to be able to have bitrates up to 1024 Kbps [157].
Key findings of [156] are that there are immense playback delay differences between
clients for the same channel, even in same local network. Differences of more than 80
seconds or even several minutes were observed for the Olympic channels. A previous
study [157] reports an average of 20 seconds for the average day streaming cases.
Nevertheless, the findings of [156] show that PPStream uses a mechanism to prefer intra-
ISP overlay contacts but does not implement mechanisms for more fine-granular locality
information of peers. The authors see a potential for improvement in this case.
The findings concerning large-scale streaming events, such as the Olympics, provide
valuable inputs for the understanding of traffic characteristics of popular applications used
in fixed as well as mobile environments.
In the context of SmartenIT, it is important to understand the specific traffic characteristics
and requirements that such P2P-based streaming systems impose. As Table shows, they
cause a large amount of traffic both on the uplink as well as on the downlink of todays
mobile networks. The increased utilization of the uplink is characteristic to the application
of P2P techniques as clients not only consume but also contribute bandwidth to the
system. It has to be investigated how traffic management mechanisms can be used to
support such kind of applications. This is especially interesting for P2P-based streaming in
mobile environments where the limited resources and battery lifetime of the devices
impose additional challenges.
Liu et al. [79] showed that different mechanisms on client-side can have an impact on the
energy-efficiency of the data transmission. In the context of SmartenIT it has to be seen
how the traffic management in mobile and fixed networks can contribute to a more energy-
efficient streaming process. Here, in analogy to the client side, techniques such as traffic
aggregation on network side could be used to avoid long client-side tail states.
Furthermore, there is a great potential to improve the energy-efficiency and QoE at the
mobile clients by offloading the mobile network. This could be done, e.g., using nano data
center [192] concepts to take over parts of the expensive upload process of the mobile
device or using direct ad-hoc-based communication between clients. For the latter,
support by the mobile network side could play an important role for the discovery and
initiation of ad-hoc communications in spatial proximity of a client. In a broader sense, this
can be seen as part of the mobile traffic management process for the dissemination of
streaming data to mobile clients.
4.3 Online Storage
Online storage applications allow large data transfers using between the end-user's device
and the cloud. Generally, online storage services include file-hosting, network back-up,
and one-click downloads, however, in this section, we focus on large-scale personal back-
up for disaster recovery or remote access, e.g., Dropbox, Google Drive, etc.
Public

According to [174], a significant portion of traffic is associated with online back-up and file
storage sites. Until 2011, no single site had secured dominance, as the popularity of
services varied between regions and new competitors were still entering the market.
Additionally, it was expected that online storage and back-up usage would increase in
tandem with smartphone and tablet adoption, with users relying on these services to
access their files remotely, while it was concluded that cloud storage has the potential to
become a major source of traffic on the internet.
Moreover in [21], the by then low percentage of traffic generated by online storage
applications was noted, while in [68] this low traffic share was attributed to the fact that
normally, a large and increasing amount of this type of traffic is carried via secure tunnels,
so it is frequently indistinguishable from things like mobile banking and other encrypted
applications. In fact, on many mobile networks, a scenario was observed, in which a major
mobile OS update triggers a step-function increase in levels of SSL traffic by introducing
features like file synchronization over SSL. Nevertheless, it was forecasted that online
storage and back-up services will see dramatic growth as transparent automated back-ups
become commonplace.
Below, we describe certain popular and also some less known applications for online
storage which might be of interest to SmartenIT for different reasons, which are
highlighted respectively.
4.3.1 Dropbox
In this section, we describe the operation of Dropbox, an online personal storage system.
Additionally, we briefly overview the outcome of the measurements reported in [29],
whereas an extensive analysis of Dropbox usage on the Internet is given in Appendix B.
Dropbox [27] is a cloud-based file hosting service that offers cloud storage, file
synchronization, and the necessary client software for the aforementioned operations. It
allows users to create a designated folder on each of their devices, e.g., desktops, laptops
or smart phones. In this folder, users are enabled to drop any file, or to upload it through a
web browser. Then, Dropbox synchronizes the designated folder so that the same folder
and its sub-folders including the same files with the same content appears to the end-
user, regardless of the device that the end-user is accessing the folder from. Files placed
in this folder are accessible through a website and smartphone/tablet applications.
While Dropbox functions as a storage service, it focuses on synchronization and sharing.
In particular, it supports revision history, so that files deleted from the Dropbox folder may
be recovered from any of the synched devices. Additionally, Dropbox supports multi-user
version control, enabling several users to edit and re-port files without overwriting
versions. The version history is kept by default for 30 day for the free version, while an
unlimited version is also available for purchase.
The basic object in the Dropbox system is a chunk of data with size of up to 4MB. Files
larger than that are split into several chunks, each treated as an independent object. Each
chunk is identified by a SHA256 hash value, which is part of meta-data descriptions of
files. Dropbox reduces the amount of exchanged data by using delta encoding when
transmitting chunks. It also keeps locally in each device a database of meta-data
information (updated via incremental updates) and compresses chunks before submitting
them. In addition, the client offers the user the ability to control the maximum download
and upload speed.
Public

Dropbox also provides some form of localization; more specifically, a technology called
LANSync is employed, which allows devices on the same local area network (LAN) to
securely download files locally from each other, instead of always 'hitting' the central
servers.
Concerning technical details, Dropbox uses Amazon's S3 platform to store the files, and
Amazon's EC2 for control and synchronization handling. Synchronization is performed
with SSL transfers, while data are stored encrypted via AES-256 encryption. Both the
Dropbox server and client are written in Python. Registered by IANA ports for
communication are TCP 17500 and UDP 17500 ports.
Even though Dropbox states that files are stored encrypted on their servers, it seems that
Dropbox has access to all the files, which makes the statement about encrypted data
questionable. This finding is reported in [28], where same file of several MB was added to
two different Dropbox accounts. When the file was added to the second account, the
Dropbox client generated however only about 16 KB traffic to the storage server, which
means that the file content needs to be accessible to Dropbox. Otherwise, a comparison
would not be possible.
As with all proprietary cloud application, the control or optimization of such systems is
difficult unless we assume collaboration with the cloud provider, which is Dropbox in this
case. Drago et al. [29] proposed improvements based on their network measurements,
but only the Dropbox software developers and engineers are able to implement such
proposals. Hence, although the offered service and the used technology might be of high
interest for the SmartenIT project, we can also only propose optimizations, but we are not
able to test actual implementations.
Main findings of [29] show that the Dropbox service performance is highly impacted by the
distance between the clients and the data centres, which are currently located in the U.S.
However, geographically dispersed sources as well as longitudinal data are expected to
set the need for distribution of Dropbox's storage servers around the world. The dispersion
of Dropbox servers around the globe will though set new challenges to the Dropbox
provider, the cloud operator hosting it (i.e., offering storage and control servers) and
eventually network providers whose links will be utilized for the inter-connection of
Dropbox servers. Additionally, the usage of per-chunk acknowledgments in the client
protocol combined with the typically small chunk sizes deeply limits the effective
throughput of the service. Nevertheless, as also highlighted in [29], there is space for
Dropbox (i.e., the application provider) to optimize both performance and throughput to
end-users.
4.3.2 Google Drive
Google Drive is a cloud-based storage service integrated with Google Docs providing a
free cloud-based SaaS office suite: Documents, Spreadsheets, Presentations and
Drawings. Google Docs has its own document formats; namely, when a user uploads his
own documents to the Google Docs, those documents are converted to the Googles
format. Google Drive was released on April 24, 2012 [121].
Google offers to its customers 5 GB free cloud disk space. A user can get additional quota
for adequate monthly fee. Available disk sizes are: 25 GB, 50 GB, 100 GB, 200 GB, 400
GB, 1 TB, 2 TB, 4 TB, 8 TB, 16 TB. When user get extra disk space its Gmail quota
increases additionally from 10GB (free account) to 25 GB [122].
Public

Data on the Google Drive can be stored in three ways: user can create documents via
Google Drive web panel, user can import documents to the Google Drive via web panel or
user can upload files with the Google Drive client software. Google Drive client software
also provides files synchronizations. It is available for PCs with Windows, Macs, Android
smartphones and tablets and works on iPhones and iPads with iOS 5.0+ [123].
Documents, spreadsheets and presentations created inside Google Drive or imported
thought web interface or sent via email can be shared, opened and edited by many people
at the same time. User or owner of the document cannot be notified of changes but can
see revision history. Additionally, when users cooperate with document they can see that
another person also opened the document. Users also can communicate via integrated
chat.
Created or uploaded documents, spreadsheets and presentations have some limits.
Document can have maximum of 102400 characters and can have maximum 2MB size
after uploading and converting to the Google document format. Google spreadsheet can
have maximum of 400000 cells with maximum of 256 columns per sheet. After sending to
the server can have maximum of 20MB. Presentations can take up to 50MB (about 200
slides). There is no restriction on the drawings but other files (not converted to Googles
format) can be up to 10GB [124].
Google Drive allows user to preview documents online but only files less than 25MB.
Supported file types are listed in [125]. Because Google Drive is directly connected with
Gmail, users can in simple way (using the Google Drive viewer) preview attachments form
email. Additionally, Google Drive allows user to enable free two-factor authentication for
better security. To login, user in addition to the regular login name and password must
enter short random code sequence generated by the Google Authenticator application
from Android or iOS phone or sent via SMS to their phone.
Table 4-2: Typical energy reduction in company after migrating to Google Apps [202][128].
Source of energy
consumption
Energy consumption
today (kWh per employee
per year)
Energy consumption
after Google Apps (kWh
per employee per year)
Savings from
switching to
Google Apps
Server direct energy 18175 253 7090%
Server cooling 1888 226 7090%
Total energy consumed
per user
36263 479 7090%
Additional cloud-based
energy consumption (Google
+ networking)
-- 15 23% increase
Total required energy 36263 584 6887%
As Google reported in [202], the migration from typical IT solutions in companies can
reduce energy consumption by 68-87%. For example, migration to cloud Google Apps
(operating within Google Drive) in the U.S. General Service Administration (GSA) in 2012
has reduced the total number of servers operated by GSA for email and collaboration
(Google Docs) from 324 to 61. Therefore, total direct power reduction of GSA servers
achieved 87%. Additional cloud-based energy consumption (Google + network) increased
by 2-3%. Typical energy saving resulting from reduction of energy used directly by servers
and servers cooling are presented in Table 4-2.
Public

4.3.3 PiCsMu
PiCsMu (Platform-independent Cross Storage for Multi-Usage) [30] is a cloud storage
system implemented with a cloud-based overlay by the University of Zurich (UZH). The
built overlay is used in order to store, retrieve, and share files either private (not sharing)
or in a public fashion (sharing). In the private scope, PiCsMu users can upload files to
their own use. Just who uploaded can download the file. In the public scope, PiCsMu
users can choose if files should be available to a set of other PiCsMu users (private
share), or if everyone part of the PiCsMu network can access them (public share).
The system is divided into the overlay and the underlay. In the PiCsMu underlay, PiCsMu
users need to add Cloud Services credentials, e.g., Google Picasa, Facebook,
SoundCloud, ImageShack, Dropbox, Amazon EC2/S3, Apple iCloud, OwnCloud, or any
Cloud Service that provides an API, in order to store files. The overlay holds the index
information about files, e.g., in which Cloud Service the file was stored, if the data was
encrypted, how the file was split into several small chunks, etc.
The overlay is divided in two parts: private index and shared index. The private index is
established in a centralized server and it stores information about private files i.e., files
that are just inherent to one single PiCsMu user. The shared index is established in a P2P
network to share files with a set of PiCsMu users (requiring a PiCsMu credential), or for
the whole P2P network (thus, not requiring a PiCsMu credential). Within the shared index,
PiCsMu application uses encryption (public/private keys) to share files to a limited set of
PiCsMu users. One important aspect is that the shared index established in a P2P
network uses social capabilities (expressing friendship relations) to avoid attacks (e.g.,
flood an users) and mistrust (e.g., share a malicious resources to someone).
The PiCsMu application has an innovative manner to store data due to the capability to
fragment files into several chunks (file parts) of different file types, and de facto store
these file parts into different Cloud Services (independent of which file type is accepted by
such Cloud Service). Each file part is encrypted with a different encryption method, with a
different salt, Initialization Vector (IV), and key. Moreover, each file part is encoded using
an encoder which transforms the file part data into a specific file type e.g., data can be
encoded into an image through the means of Steganography. Several encoders are
provided in the scope of PiCsMu application.
In order to illustrate the PiCsMu system, the example of a PiCsMu user uploading a PDF
file X for private use is given. The first step is to check which Cloud Services credentials
the PiCsMu user has registered for its use. Assuming that Google Picasa, SoundCloud,
Dropbox, and Facebook credentials are added to the PiCsMu application, the
fragmentation process is able to calculate how many file parts the PDF X can be split
based on the maximum upload size of each Cloud Service. In this example, it is possible
to assume that the PDF X was divided into 6 file parts. Each of these file parts should be
first encrypted and go through the encoding process. The encoding process will consider
each Cloud Service available and check which file type is accepted to upload to the given
service. For example, Google Picasa just accepts images or videos (JPG, PNG, AVI, etc),
SoundCloud just accept audio files (MP3, WAV, OGG, etc), Facebook accepts either
images or video files, and Dropbox accepts any kind of file type. Therefore, the PDF X is
fragmented to 6 parts, in which each of these 6 parts are different encoded files (of
different file types) carrying the PDF data: 2 JPG images, 1 PNG image, 1 MP3 audio file,
1 WAV audio file, and one plain text file. As the last step, these files are stored into the
Cloud Services and the PiCsMu central server that hosts the private index is informed
about the process. When the PiCsMu user wants to download the PDF X, the PiCsMu
Public

application first contacts the central server and checks the index for the specific PDF file.
Based on the index, the PiCsMu application is aware of where the file parts are located (in
which services), how the file parts were encoded, how the file parts were encrypted, etc.,
and it is able to join the PDF X file as in its original format.
The main advantages of having the fragmentation process are:
- Distribute file parts in multiple Cloud Services around the globe, enabling a network
of entities that store data being managed in an overlay.
- A single entity (Cloud Service) is not able to reconstruct the whole file data, which is
a measure to enable a higher degree of security.
- A scheduler (PiCsMu Scheduler) can interfere in the fragmentation process in order
to split files using an error correction code (e.g., Reed Solomon). Therefore, if one
or more Cloud Services loose the data, such part can be reconstructed using the
same error correction code.
The main advantages of having the encoding process are:
- Higher security and privacy. The data is encoded into files that do not seem to carry
any information. Therefore, the encoding process adds another layer of complexity
in order to extract the original data from an encoded file. For example, data is
encrypted and encoded into image files (e.g., JPG or PNG). If a malicious person
wants to extract the data from the JPG file, first it is necessary to identify how the
data was encoded. After that, the data should be unencrypted, and then, finally, the
malicious person would have the original data. It is important to highlight that the
encoding process combined with the fragmentation process gives even a higher
degree of security, since parts of data will be encoded into files and spread into
different Cloud Services thus, diminishing the probability that a malicious person
would have the possibility to reconstruct the whole file data.
- Data can be stored anywhere, even if there are file type restrictions. This enables
the notion of homogeneity: all Cloud Services can store the data, in the same
manner, independently of file type restrictions.
As forecasted in [68], online storage is constantly gaining more popularity, whereas the
services overviewed in Section 4.3 are expected to generate a considerable large amount
of the global traffic in the future. Therefore, due to the increasing popularity of and
significant traffic volumes generated by such applications, SmartenIT should consider this
category of applications by designing traffic management mechanisms that will efficiently
address the generated traffic.
Additionally, research work related to the described applications, e.g., [29], stress out the
lack of efficiency in certain cases and the potential for improvement for such applications.
Specifically, the Dropbox application is of interest to SmartenIT due to its increasing
popularity and the very high amount of traffic created by this (so far) limited percentage of
users; these two, i.e., popularity and data volumes generated, motivate our expectations
that cloud storage systems will be among the top applications producing Internet traffic
soon. However, the key question regarding the selection of Dropbox as candidate
application to be further addressed by developing mechanisms in the context of
SmartenIT is yet to be answered; unfortunately, the proprietary nature of the application
Public

and the used infrastructure is rather dissuasive due to the low level of intervention
allowed. Moreover, as online storage applications are becoming increasingly popular also
on mobile devices [21], they should be considered by the SmartenIT traffic management
solution in respect to mobility and energy efficiency aspects of the project.
4.4 Search Engines
Although a query sent to search engine generates insignificant traffic volumes, the
enormous number of queries performed daily incurs great traffic volumes and also
significant energy consumption. In particular, according to Google [126], [127], the energy
consumed to perform one query in Googles search engine, i.e., Google Search, is about
0.0003 kWh; thus the energy to conduct 100 searches on Googles site is equivalent to a
60-watt light bulb burning for 28 minutes. Below, in Table 4-3, the 5 most popular search
engines are presented. In the sections to follow, we describe the top 3 and provide traffic
and energy characteristics, where available.
Table 4-3: Top 5 search engines [128].
Websites Total visits Visits share Rank 03/16 Rank 03/09 Rank 03/02
Google 3,047,369,237 72.79% 1 1 1
Yahoo! Search 330,764,881 7.90% 3 3 3
Bing 316,318,917 7.56% 2 2 2
Ask 137,104,317 3.27% 4 4 4
AOL 71,810,730 1.72% 5 5 5
4.4.1 Google Search
Google Search [175] is the most frequently used search engine developed by Google Inc.
Google uses thousands of low-cost computers from their server farm to provide parallel
processing. Google search engine consists of: Googlebot, indexer and query processor.
The Googlebot [176] is a special program that finds and downloads web pages and
provides their content to the Google indexer. The Googlebot can find web pages in two
ways: when user fill in special add URL form, i.e., www.google.com/addurl.html, or when
Googlebot finds links leading to the web page. Those specific web crawling robot can
send out thousands requests in the same time. To do so, Googlebot uses many of
computer requesting and fetching web pages. Googlebot makes request to the specific
web server more slowly than it can. This behaviour does not overload servers. It has
special queue for pages it will visit in the near future. In addition Googlebot must
constantly examine and compare pages already existing in the Googles index. On the
other hand, duplicates in the queue list must be removed to prevent scanning the same
page few times.
Indexed page are stored in huge Googles index database. From the Googlebot indexer
receives full text of the pages found in searching process. This database is sorted
alphabetically by search term. Each index stores a list of web pages that consist desired
term. Rapid access can be possible thanks this data structure. To improve search
performance Google omits stop words (like the, is, on, why, single letters, etc.). Googles
indexer is also not sensitive to the letter case, multiple spaces and some punctuation.
Public

The Googles query processor consists of: user interface, the engine that matches queries
to the relevant web pages and the result formatter. When a user sends a query via
Googles search box, the web server sends this query to the index servers. Next index
server is searching docs with relevant web content. Fragments of web pages are
generated to describe each search result, while they are returned to the user in fraction of
seconds.
Important part of Googles search engine is PageRank [177] that provides rank of web
pages. It scores importance of the web page using over hundred factors. This value
determines web pages popularity, the position and size of search terms, the proximity of
the search terms etc. A page with a higher PageRank is considered as more important
and it is in higher position of search results than a page with lower PageRank. PageRank
is patented algorithm improved over the years and containing many secret criteria.
4.4.2 Yahoo! Search
Yahoo! is a global Internet company founded March 1
st
1995, known mainly as a portal,
search engine and many other services, such as Yahoo! Search [129], Yahoo! Directory
[130], Yahoo! Mail [131], Yahoo! News [132], etc. It was a successful company in 90ties,
founded by two students Yang and Filo originated from Stanford University. Yahoo! has
shown rapid and spectacular growth in 90ties but around 2000 it experienced serious
difficulties related to situation after dot-com bubble. There was an attempt by Microsoft to
acquire Yahoo! in 2008. Starting from 2000 Yahoo! Search uses Google search engine;
however in 2004 it started to develop its own engine.
Yahoo! Search is composed of a set of scalable, flexible and highly-available data storage
and processing services and implementing them in a cloud model [201]. Yahoo!'s cloud
can be used to support externally-facing cloud services and some components of the
cloud as possible is open source (such as Hadoop). This will allow to build their own cloud
services. Yahoo! has been providing the set of requirements that characterize cloud like
multitenancy, elasticity, scalability, load and tenant balancing, availability, security,
operability, metering, global and simple APIs.
The main services of Yahoo!s cloud consist of three tiers: core services, messaging, and
edge services. The core services layer is divided into three groups of systems: batch
processing (manages CPU cycles), operational storage and provisioning (manages the
allocation of servers). The edge services are responsible for reducing latency and
increasing supplies to end users. The messaging tier assists to bind different services
together.
4.4.3 Bing
Bing [178] is a search engine developed by Microsoft and running in the cloud ([199],
[200]). It was launched in 2009 and steadily gains popularity. Currently Bing powers
Yahoos search, making it second most popular search engine with nearly 30% combined
query share (thus Google, Bing and Yahoo serve over 90% of all web searches) [135].
Marketing angle for Bing was to advertise it as a decision engine, that provides targeted
results, depending on input gathered from user. Currently targeted results are a norm;
however, Bing aims at returning extended information or even answer as opposed to
simply listing relevant webpages.
Public

Therefore Bing features include instant answers display of stock charts, extended travel
information, shopping advice, restaurant recommendations etc. Using WolframAlpha [179]
engine it can provide results of non-trivial mathematical problems. Since early 2012 Bing
is integrated with Facebook.
Bing currently uses four crawlers to build searchable index [134]. Bings index contains
estimated 13 billion webpages [136]. Query processing is being constantly improved;
therefore multiple indexes, partial indexes and training trees are used. They are then
evaluated by several metrics, to compare them and find ones that perform better. Those
metrics include cost to search, click-through rate (how often users follow link suggested by
given query), quick-backs (how often user hits back button in browser soon after visiting
returned webpage). Those alternative indexes along with various code changes and new
features are tested on certain group of users (either random or picked depending on their
previous activity), typically without their knowledge. Subsequently analysis of engagement
metrics evaluates necessity and effectiveness and impact of given search method or
functionality [133].

Figure 4-1: Bing ISN structure [198].
Figure 4-1 presents the structure of Bing search engine [198]. Frequently asked queries
are stored in cache. Since the large number of indexed web pages is stored and latency-
sensitive search queries is issued, the crawled data is split into multiple and independent
Index Serving Nodes (ISNs). Each ISN works in parallel; it identifies and returns most
relevant pages with their ranks to the top level machine i.e., the aggregator. The
aggregator replies to the requesting client by sending him search results containing a title,
URL and context snippets. To achieve better performance ISNs can be replicated [199].
This allows to balance incoming search queries among multiple ISNs that are responsible
for the same shard. ISNs communicate only with frontend machines the aggregators,
never with other ISN.
While web searches alone do not create significant traffic themselves, they are
fundamental functions allowing efficient usage of Internet resources by Internet users.
Two aspects seem to prevail constant evolution and result targeting.
Public

While initial indexing is a straightforward process, query processing is a complex process.
Returned results not only have to match submitted query, but are also ranked via as much
as hundred factors and metrics, ranging from accuracy and popularity of page to user
preferences and search history. Search engines constantly create new mechanisms and
functions to improve their performance, and test them on subsets of their users. Therefore
search engines must be designed in a way that allows such on the fly trial and error
approach. Since result targeting is a norm it might be reasonable to group users according
to their preferences, to optimize and strengthen targeting process.
In terms of energy efficiency aspect, it is important to realize that Google data centres
alone consume 0.01% of worlds electricity. Moreover, Google takes pride in those,
claiming theyre up to 50% more energy efficient than typical ones (with most energy
saved on facilities housing servers) [137]. This makes it an interesting topic for SmartenIT
energy-awareness considerations. An example of impact of migration to cloud on energy
savings is included above in Section 4.4.2. Moreover, end-users' queries could be crawled
to gain insight on their interests and communities. Such information can be employed to
complement and support social awareness which is also of high interest to SmartenIT.
4.5 Online Social Networks
In this section, we briefly describe main features, functionalities, implementation issues
and performance indices (if available) for fundamental concepts of social networking,
which are responsible for creating novel habits and opportunities for Internet users
changing way of communication and content exchange. Such insight will be useful
towards the design of socially-aware traffic management mechanisms.
4.5.1 Facebook
Facebook is currently the most popular online social network started in 2004 by students
from Harvard University. Facebook users are able after their registration to create their
personal profile and invite or add other people as friends. Whenever a friend changes
profiles content, notifications are sent to his friends. Besides personal profile Facebook
allows to create common-interest user groups by organizations, TV stations, local
authorities, etc. Currently Facebook has more than 1 billion registered users [138].
The interface of Facebook enables each user to create a public prole with pictures and
other personal information such as date of birth, hometown, contact information, school,
employer and lists of personal interests. Moreover, a user can establish a friendship link
with other users by sending and accepting a friendship request. A prole of user has a
wall wherein the user himself and his (or her) friends can post messages, links to
external sources (e.g., YouTube videos), photos, etc. A message is visible to everyone
who has access to the users prole. Uploading messages on the wall is the mostly used
activity in Facebook [187].
Facebook homepage (as of March 2013) can be split into six independent sections: 1) top
bar, 2) navigator area on the left side, 3) news feed in the central part, 4) advertisement
and birthday and event notifications part, 5) Ticer a real-time information about status,
friendship, photos, etc., and 6) Facebook Chat.
In accordance with [185], a single Facebook web page is retrieved from more than 15
different IP addresses corresponding to multiple servers. During homepage downloading,
Facebook creates a number of parallel TCP connections to these servers. Each
Public

homepage section is downloaded in fixed order (from 1 to 6). Each object is retrieved with
3-7 parallel streams for data transmissions. Downloaded files may be divided into 24 types
according its properties (file type, name pattern, location on the web page, source IP
address) [185]. Moreover, during FB page downloading two peak periods were observed
[185]. The first, approximately in the middle of downloading, occurs when core files,
pictures and scripts of extensions are collected, whereas the second peak appears at the
end of page loading.
Several studies of the traffic patterns generated by Facebook have been performed over
the last few years, due to Facebook rapidly increasing popularity and traffic volumes
generated. Facebooks applications and traffic are built on a mix of many IP applications
(messages, downloads of photos, video streaming, etc.) over HTTP. Below, we
summarize major results from measurements ([82], [83]) on Facebook traffic profiles
within the last years:
- Communication via Facebook is most often between users in local communities;
- Facebook relies on large replicated data centres in the USA, but most users
outside the USA experience long delays and high loss rates for Facebook requests;
- Long paths also bring unnecessary traffic load on expensive transcontinental links;
- Facebook posts generate non-negligible traffic volume, although they have a small
volume, but often are spread over large friendship communities.
A comparison [85] of the session activity of users in OSNs with users navigating via
search engines reveals that OSN visitors are more likely to stay within the origin website
compared to search engine visitors. Moreover the variety and number of domains visited
from search engines is higher than those visited from OSNs; in particular, search engines
lead their visitors to a wider variety of websites including more shopping and reference-
related websites, whereas OSNs are more likely to send their visitors to less popular
domains in comparison with search engines. The latter may also be related to the fact that
through OSNs significant amounts of content is exchanged, e.g., photos, and videos,
which is mostly user-generated (UGC), and thus, long-tail content [63].
An in-depth investigation of Facebooks traffic patterns, which are compared to respective
patterns obtained by observation of YouTube, as the application that contributes the
highest traffic volumes to global IP traffic, as well as the model used for their traffic
characterization are provided in Appendix C.
4.5.2 Twitter
Twitter [180] is an online social networking service created on March 21
st
2006 and then
launched in July 2006. The service rapidly gained worldwide popularity. Since its launch,
Twitter has become one of the ten most visited websites on the Internet.
Twitter has similar features to Facebook: The interface allows users to post short
messages (up to 140 characters) know as tweets that can be read by any other Twitter
user. Often, a Twitter user will see another users tweet and wish to rebroadcast that
message. This is traditionally accomplished with a retweet. A retweet is a re-posting of
someone else's tweet on your profile. Moreover, a user can follow other users they find
interesting in order to get real-time updates of content those users publish.
Public

Twitters privacy policy is a two option that either allows every message a user creates to
be publicly available, or allows only a users followers to see posted messages. In addition
unregistered users can only read tweets.
As official statistics show Twitter achieves over 500 million registered users as of 2012 but
only one out of four is active user, generating over 340 million tweets daily. About 43% of
Twitter users are male and only 57% are female. The most users of Twitter, almost 57%,
are in age between 26 to 45 years.
4.5.3 Google+
Google+ [181] started on June 28
th
2011 as closed social network (users can join only with
invitation). The platform is open for public from September 20
th
2011 (registration without
invitations) [31].
Google+ has similar features to Facebook or Twitter: users can share their information in
the stream (like the wall in Facebook) where all users activity is shown. Unlike to
Facebook users Google+ users friendship is unidirectional (similar to Twitter), e.g., first
user can observe the second (as a friend) public posts without reciprocate of friendship.
To protect the visibility of a user's activity, Google+ allows users to manage their contacts
access to their content by creating circles. This feature allows users to control all
information visibility to labelled groups of friends. There are two types of circles: in- and
out-circles [32]. The first one is the list of other users who contain a given user in their
circles. The second list of added users by a given user. List of circle user and circle
name are only available for circle creator. People can add other users to their circles
without confirmation. Note that default privacy settings in Google+ are public (all
information visible to anyone). In addition there are also four other options: extend circles
profile open to users that are in circles and their friends, your circles information visible
in user circles, only you and custom custom list of circles that can see user activity.

Figure 4-2: Google+ User Statistics (Dec. 2012) [33].
Google+ uses encrypted channel (i.e., SSL) throughout the connection. Like in other
Google applications, the user can enable a two-way-authentication; this functionality
propagates also to other Google applications (e.g., Gmail, Google Calendar, Google Docs,
etc. This feature improves account security by requiring a combination of two elements for
the confirmation of identity a user name and password and the special random code
(delivered as SMS, via phone call or supplied with special mobile application, e.g.,
Google Authenticator for Android smartphones). Additionally, each Google+ user has
individual ID [32], which contains a 21-digit integer, where the first digit is always 1. This
Public

protects users to identify valid user IDs by generating random number (the ID space is
very large, i.e., 10
20
). Figure 4-2 shows that Google+ achieved by Dec. 2012 500 million
total users with over 135 million active users [33].
4.5.4 LinkedIn
LinkedIn [182] is a social networking site focused on business relationships. It is currently
ranked 13th in Alexa Ranking [24] with over 175 million registered users. Professional
offers can be posted there and users can look for jobs and career opportunities. Users
can build their contact network with direct and indirect contacts. Each can construct their
resume showcasing their background, skills and experiences.
LinkedIn is focused on connections and job offers; however some content can be
published, ranging from business-related articles, market analysis, through group
discussions to user pictures, which aid identification. Other features include individual
message system, groups and LinkedIn Answers where one can ask questions or seek
answers and opinions in chosen category.
There is no charge for the basic functions of LinkedIn. Bulk of the companys revenue
comes from advertising, but in 2008 LinkedIn launched sponsored advertising as a new
functionality for users. Nevertheless, there are three tiers of service in LinkedIn. Data tier,
maintaining persistent state including user data. The service tier, implementing an API
layer, is responsible for capturing the logical operations on the website. The display tier
translates API into user interfaces (web pages and widgets on external sites). This
separation, along with fact that service and display tier are stateless allows for easy
balancing of the load, it also restricts harder problem of scaling the systems with state to
smaller number of devices.
LinkedIn employs two data storage systems, one for the rich search capabilities and one
for the simple and fast key-value access. It also uses Databus system, to capture data
changes and provide a common pipeline for transporting those changes from primary
databases to various applications. This solution is easy to scale and extend to add support
for other data sources.
Social networks produce currently considerable traffic volumes (estimated between 20% -
25%). Their characteristics are quite consistent mostly made up of small posts that are
spread over very large communities of users. Since interacting users are in most cases
geographically closely located and communication relies on distant data centres, there is
clear space for improvement and management with traffic related aspects being
addressed by SmartenIT. Additionally, due to the multitude of websites integrated with
many social networks, solutions involved in this process (traffic management) may be of
interest to SmartenIT. Some level of predictability of OSN traffic and user behaviour may
be useful to build social-aware mechanisms in WP2.
4.6 Online Gaming
We focus, in this section, on cloud gaming which is a sub-category of online gaming.
Cloud gaming is a service offered to gamers (consumers) and involves the streaming of
games to any personal device using a thin client, while storage and processing are
Public

performed in the service providers server clusters. This service has the following
advantages compared to traditional online gaming:
- Consumers are given the opportunity to stream and play a particular game on any
of their personal devices,
- Consumers are not required to have powerful and expensive machines, in order to
play recently-released high-end games, but a decent Internet connection.
Below, we provide descriptions of OnLive, Gaikai and Zynga. OnLive and Gaikai are two
major competitors in the cloud gaming market, offering high-quality game titles from well-
known game publishers, whereas Zynga represents a social-network-oriented aspect of
cloud gaming that deploys cloud services to serve millions of gamers daily.
4.6.1 Onlive
OnLive is a San Francisco-based company, founded in 2009, offering cloud gaming and
desktop services. Its cloud gaming platform is available in the US and has recently
entered the European market, in the UK and Belgium. Its partnership with over 50 game
publishers, such as Sega, Warner Bros, etc., as well as its 300 game titles, makes OnLive
one of the leading platforms in the cloud gaming industry.
Its gaming platform is available through either the OnLive client (OnLive Game System)
for certain consumer devices, e.g. PCs, Macintosh computers, iOS and Android
smartphones and tablets or the MicroConsole TV Adapter, which can be connected to
any digital TV or monitor. In addition, consumers are able to watch the services demo
videos through any web browser. OnLives games consist of 2 streams, one for gameplay,
and another for social and demo purposes, and are available in 720p format; hence an
internet connection of 5Mbit/s is recommended.
OnLives network consists of 5 data centres across the US, covering a 1000-mile radius of
users, while contracts have been made with major US tier-1 network providers to reserve
bandwidth explicitly for the service [34] and resolve the Best-Effort problem of the Internet,
aiming to achieve high Quality of Service (QoS) and QoE for the consumers gaming
experience. OnLive has also designed and developed their own video compression
algorithm, claiming to reduce latency to 1ms at servers side. A few studies have been
carried out in the recent years, measuring the services characteristics. More specifically,
total latency was measured to 135-240 ms. [35], [36], in a variety of games (from basic to
high-end), with high downstream (5 Mb/s) and low upstream (100kb/s) bitrates and frame
rates ranging from 25 to 60 fps depending on the provided capacity [37].
4.6.2 Gaikai
Gaikai is a company offering cloud-based gaming services, recently acquired by Sony
Computer Entertainment for 380 million USD. It is based in Orange County, California and
was founded in 2008, aiming to provide cloud gaming services worldwide. It serves more
than 50 million consumers per month, offering more than 200 game titles and has
partnered with major industries of the electronics and gaming sector, such as Samsung,
LG, Warner Bros, Electronic Arts, etc.
Consumers may access Gaikais gaming platform from any of their personal devices, e.g.
personal computer, smartphone, tablet and digital TV, by using their web browser (client is
embedded to websites or Facebook). Hence, on the consumers side, no extra heavy
Public

software is required, but a Java or Flash plugin and a reasonable internet connection
(Gaikai recommends an internet connection of 5Mbit/s). As also presented during the
Google I/O 2012, the service was also ported to Google Native Client, making it instantly
available to Chrome browser and Chromebooks.
As a cloud gaming platform, all the game processing, storage and video encoding is
performed on Gaikais server clusters. Gaikai has partnered with NVIDIA and has adopted
the NVIDIA GeForce GRID (PaaS) [38] cloud gaming platform, operating a worldwide
streaming network of 24 data centres, and aiming to achieve low latency/lag, high server
density, high-performance video encoding and energy efficiency. Digital Foundry [39] has
performed a research, comparing Gaikai to OnLive for a variety of games and measured
latency of 133-216ms (while OnLives latency ranged from 183 to 300 ms.) [40].
4.6.3 Zynga
Zynga is social gaming creator and provider, based in San Francisco, founded in 2007. In
the recent years, its games have become very popular due to its close collaboration with
major social networks, such as Facebook. It features more than 30 game titles and serves
300 million active users monthly in 166 countries.
Zyngas games are accessible on most consumer devices, through either a web browser
in personal computers (via Zyngas website or social networking websites, e.g. Facebook,
Google+) or native applications in iOS and Android smartphones and tablets. The
minimum requirement is an internet connection.
The massive growth of Zyngas popularity, has forced the company to investigate ways for
managing the increased traffic. Initially, Amazon Web Services (Amazon EC2 platform -
Section 3.5.1) were employed (12000 EC2 nodes in 2010), but Zynga has recently moved
parts of its service to its private cloud, called zCloud, which uses load balancing and
scaling techniques similar to EC2, and is based on Apache CloudStack. The cloud
management platform RightScale is used for management of both cloud infrastructures.
Internet gaming traffic has witnessed a steady growth over the past years, and according
to Cisco, it will continue growing by 52% annually, estimated to reach 630 PBs per month
in 2016 [20]. While most gaming services move their infrastructure and services to the
cloud, and millions of users access such services from their mobile devices or/and
personal computers, it could be important for the SmartenIT project to address the
expected traffic growth in the gaming sector and provide efficient solutions.
Generally, cloud gaming generates traffic that travels through the public Internet, while it
aims to achieve as lower latency possible to ensure the best QoE to the end users.
Although cloud game providers place their PoPs in strategically-selected locations
globally, and sign SLAs with major network providers, there are still studies that reveal
potential pitfalls of the services, such as high latency due to network delay and bandwidth,
reduced QoE in consumers side, and unreachable markets due to lack of PoPs. Hence, it
is clear that its performance relies heavily on end users distance from the closest data
centre, and the best effort nature of the Internet. SmartenIT could enforce its traffic
management mechanisms to optimize the delivery paths between game servers and end
users and reduce latency, by assigning optimal alternative routes, instead of best effort
ones, and/or allocating more resources to downstream traffic. As a result, cloud gaming
Public

services footprint could be widened to currently unreachable locations and markets, and
end users QoS and gameplay experience could be significantly improved. It is possible
that SmartenIT could cooperate with network providers to intervene in and manage cloud
games generated traffic over the public Internet, but the proprietary nature of all cloud
gaming services would impose significant obstacles.
4.7 Cloud Content Delivery Networks
As defined in Section 3.1, CDNs are distributed systems of strategically deployed servers
over the Internet, aiming to store replicas of the content to be disseminated close to end-
users and redirect users requests to the closest copy. Hence, end-users can access the
requested content, with less delay and timeouts, while traffic is reduced as content is not
delivered from the origin content server.
Nowadays, there are multiple traditional CDN providers, with many points of presence
(PoPs) worldwide, offering content distribution and delivery services to content providers.
Akamai [43] is the leader in the CDN market, with more than 100,000 servers in more than
2000 locations in 80 countries, partnering with several popular content providers, e.g.,
Youtube, Facebook, CNN [41], ESPN [42], etc. Akamais major competitors are Limelight
[44] with 6000 servers at 75 PoPs in Europe, US and Asia, Level3 [45] with 215 PoPs
globally and ChinaCache [46]. Such major CDN providers deploy and operate an
extensive and expensive network of physical and virtual edge servers worldwide; hence,
its rather unlikely that new CDN providers would afford joining the CDN market and
competing with them.
Cloud-based CDNs provide the same services as the traditional ones, but take advantage
of cloud attributes such as virtualization in order to improve their reliability and
performance, as well as reduce operational costs. Cloud-based CDNs can be classified in
two main categories.
The first one includes CDNs which i) partly use cloud resources to optimize their services,
e.g., cloud storage for content caching, and cloud computation for automatic content
replication or service management, ii) are deployed by cloud operators and are provided in
an integrated manner with other cloud services. Prominent examples of this category have
been described in Section 3.5, e.g., Amazon's CloudFront, Windows Azure CDN, and
Google's Cloud Storage.
On the other hand, the second category includes CDN-as-a-Service platforms, i.e., CDNs
that are fully deployed on the cloud and operate on a virtualized environment of servers
that cache, distribute and deliver content in a CDN-like fashion. The only example of this
category, at least to our knowledge is MetaCDN.
4.7.1 MetaCDN
MetaCDN, as aforementioned, is a fully cloud-based CDN company, which has been
founded in 2011 as a result of research that took in the University of Melbourne. The
problem that MetaCDN faces is the global presence of traditional CDNs such as Akamai,
who have invested billions in operating an extensive network of physical servers in
multiple locations around the world, making very difficult for others to enter this market. On
the other hand, existing cloud storage providers while offering less expensive services,
they only provide virtual storage space, missing all essential CDN functionalities, such as
load balancing and redirection, and automatic content replication.
Public

Figure 4-3: MetaCDN architecture [47].
MetaCDNs aim is to integrate major cloud storage services, such as Amazon S3 and
Nirvanix SDN into one low-cost, fully-functional overlay CDN. As depicted in Figure 4-3,
MetaCDN implements connectors for all supported cloud storage services, in order to
provide an abstraction and hide their requirements and complexity from its users. It offers
a website and exposes Restful web services, with which content creators and providers
are able to distribute their content to one of the supported cloud storage providers, while
end-users experience high-performance content delivery due to load balancing and
redirection mechanisms.
To achieve this, MetaCDN consists of several components, handling specific tasks; QoS
monitor measures storage services performance, Allocator selects the optimal provider
based on multiple criteria, Database and Manager components store information and
perform internal tasks, respectively, while Load Redirector handles request redirection to
the most appropriate replica. A unified namespace is also adopted to simplify content
management and routing, and make the mechanisms transparent to end-users. The
services features also include website acceleration, cloud-based video encoding and
streaming and high-performance content delivery.
Table 4-4: MetaCDN performance and Content Providers benefits [47].

MetaCDN consists of around 80 PoPs in 15 countries worldwide. A few studies were
performed regarding MetaCDNs performance [47], [48]; they focused on response time
compared to direct replica access time and the overhead was from insignificant to
Public

noticeable, depending on the clients and closest replicas location. More specifically, a
few tests were conducted in an experimental testbed, to measure response times and
throughput from multiple client locations worldwide, using 3 redirection policies, random,
geographical and utility-based; average response times varied just over 1 second due to
the proximity and distribution of Amazon and Nirvanix nodes globally, with the exception of
Beijing clients and a few timeouts when random redirection was applied (Table 4-4).
As MetaCDNs network expands and further integration of other popular storage services
is adopted, performance increases due to higher availability of more local replicas.
MetaCDN integrates multiple cloud storage services into one virtual and efficient CDN,
achieving lower, but comparable, and cost-efficient performance to traditional CDNs and
cloud storage services. Its internal architecture and mechanisms could be a basis for
SmartenIT architecture design which aims to support the federation and inter-connection
of multiple data centres and clouds. In addition, SmartenITs traffic management
mechanisms could be used to improve MetaCDNs response times towards end users,
especially in cases that cloud storage PoPs are not in close geographical proximity, while
energy efficiency and social awareness criteria could be enhanced into services
redirection policies, improving replica distribution and selection and overall service
performance.
Though due to its proprietary nature, potential for intervention in the MetaCDN overlay
might be quite low. Moreover, the impact of a mechanism addressing MetaCDN's traffic
cannot be predicted, as currently no information on traffic characteristics of the MetaCDN
was found.
4.8 Mobile P2P File-Sharing
Over the last few years, P2P file sharing traffic share over global IP traffic has declined
from 19.2% in 2010, to 12.0% in 2012 [21]. However, regardless of the share decline, P2P
file sharing still is responsible for huge amounts of data carried over the network, i.e.,
according to [23], P2P file sharing traffic (excluding P2P video streaming traffic such as
PPStream and PPLive) will increase by 2016 with 17% CAGR.
Moreover, BitTorrent is expected to experience a drop in overall ranking [21]; still though
its place will remain high as it is predicted to fall from being the second largest application
on the network, to being the fourth, after Netflix, YouTube, and HTTP web browsing.
In recent years, P2P file sharing applications have been developed to operate also on
mobile devices, e.g., smartphones and tablets. Nevertheless, mobile P2P file sharing has
not been very popular due to the fact that mobile devices participating in content sharing
communities face at least two challenges, both of which are related to mobile devices
limitations in terms of storage and energy capacity.
First, content shared over P2P networks such as BitTorrent [5] comprises mainly of video
files, whose size varies from few hundreds of MBs (e.g., standard definition (SD) video
clips, TV shows episodes, etc.) to few tens of GBs (e.g., high definition (HD) 3D movies).
Taking into account that the storage capacity of mobile devices currently is up to few tens
of GBs, it can be easily inferred that storage constitutes an important constraint to mobile
P2P file sharing. An also important constraint and more serious obstacle is the energy
consumption of the mobile device. The progress on battery technology is steady but slow
Public

so spending as little energy as possible on different operations is likely to remain an
important design constraint for mobile solutions.
Hence, in the sections below, we overview applications for mobile P2P file sharing, briefly
describe their operation and address their relevance w.r.t. SmartenIT's objectives.
Moreover, an overview of P2P file sharing applications mainly addressing desktop devices
such as PCs and laptops, and not mobile devices such as smartphones and tablets, is
provided in [183].
4.8.1 BitTorrent
In this section, we briefly present most notable P2P torrent-based file sharing applications
for mobile phones.
Generally, P2P clients for mobile devices address certain platforms and OSs. For
instance, SymTorrent [86] is an open source BitTorrent client for the 2nd and 3rd
generation of the Symbian OS, which supports downloading of multiple files
simultaneously and resuming after restarting the application. Transmission [89] is a torrent
client directed at Mac OS X, Linux, FreeBSD and Solaris, which is claimed to can also be
ised on a mobile phone by the use of a web user interface named "Clutch". MobTorrent
[90] is the first Bittorrent client for Java Micro-Edition (ME) based mobile phones, e.g.,
Nokia S40, Nokia 6500, and supports all Java ME capable phones with the proper JSRs.
Moreover, wmTorrent [93] is available for Microsoft's Windows Mobile, respectively.
Particularly for the Android OS, there are plenty of BitTorrent clients such as tTorrent [91],
aTorrent [92], BitTorrent for Android [94], and uTorrent for Android [95], with plenty of
features, some of which include: download of multiple torrents, queuing search for
torrents, use of both Wifi and WiMAX mode, ability to limit upload/download speed, web
browser integration, multiple parallel downloading, trackerless torrent (DHT) support, IP
filtering support, proxy support, encryption, option to pause downloads when external
power supply is not connected, etc.
Apart from BitTorrent clients that download the content file to the mobile device, there are
also several clients such as uTorrent Remote [97] and BitTorrent remote [96] that
practically provide remote access to a PC or laptop running the actual BitTorrent
application so as to monitor downloading, add, remove, stop and resume torrents.
Last but not least, another BitTorrent client, highly interesting w.r.t. to SmartenIT's targets,
is CloudTorrent, which is an extension to SymTorrent. CloudTorrent [88] is an alternative
mobile P2P file sharing architecture addressing energy efficiency. In particular, the
CloudTorrent architecture consists of two main parts: a phone application communicating
with the cloud and a cloud server (an Amazon EC2 instance with 10 Mbps upload capacity
was used in the evaluations) hosting the remote (regular) BitTorrent client. Evaluation
results have shown that CloudTorrent is able to reach much higher transfer speed, i.e.,
88% faster than SymTorrent. Additionally, CloudTorrent consumes less energy compared
to SymTorrent. According [88], CloudTorrent can be extended to incorporate also users'
PCs as servers; practically, that would lead to a hybrid solution which would achieve
higher decentralization of traffic, and lower energy consumption in the cloud.
Cisco [23] predicts that global mobile P2P traffic volume will increase from 46 PBs (i.e.,
10
6
TBs) in 2011 to 194 PBs by 2016, which implies a CAGR of 33%. Though, taking into
Public

account the fact that global mobile traffic is expected to reach up to 10.8 EBs, this implies
that mobile P2P traffic will constitute only 1% of total mobile IP traffic. Note that the
aforementioned traffic volume does not include P2P video streaming traffic, as all kinds of
video streaming traffic is considered to be a consolidated category.
The development and adoption of cloud-based solutions such as CloudTorrent may affect
this trend, increasing the popularity of mobile P2P applications, and thus the overall traffic
volume generated by such applications.
From SmartenIT's point-of-view, mobile P2P file sharing applications are of interest as
they seem to gain popularity and therefore are expected to generate higher traffic volumes
on the network in the near future. Additionally, cloud-based mobile P2P applications such
as CloudTorrent might be interesting, as the decentralized approach incorporating also
users' PCs as servers, e.g., in order to run the actual BitTorrent client, could be extended
to provide also other high-volume, or bandwidth-intensive mobile applications, e.g., video
streaming or gaming, in an energy efficient manner. Moreover, the cloud-based P2P file
sharing applications can constitute a nice example of service mobility as content chunks
exchanged over the P2P network can become simultaneously accessible through different
devices of the same end-user.
4.9 Mobile Instant Messaging and Voice-over-IP
With the increasing usage of mobile devices and in particular smartphones, a family of
special applications has been published that targets the communication needs of users in
mobile environment. These applications are not cloud-based; though they might be of
interest to SmartenIT as they might constitute a source for obtaining social meta-
information on mobile users. Well known examples are WhatsApp [49], Viper [50], HeyTell
[51], and Skype [52]. Their motivation is to provide a rich platform for communication,
going beyond the traditional services of mobile providers. They are available for all big
players in the mobile operating system market, such as Android, iOS, and Windows
Phone. In contrast to traditional mobile services, such as telephony and SMS, the apps
provide their services Over-The-Top (OTT) using the mobile data connection, without any
explicit cooperation with mobile service providers. Thereby, they largely interfere with the
traditional service providers business models [53].
According to Sandvine [53], the providers revenue is already drastically impacted by the
users switch to OTT messaging services such as WhatsApp. While for SMS services, the
network providers revenue is estimated to be as high as 30,000 USD per delivered GB
text data, for OTT delivery, e.g., using WhatsApp, it is only about 10 USD for the same
amount of data. Sandvine therefore expects that SMS delivery as a very important
segment for the providers revenue becomes less important, while the operation costs
remain the same and conclude that the missing revenue has to be replaced, most
probably by increasing prices for all other services (i.e., voice and data services). This
shows that, from a business model point of view, these special applications and their
adoption for mobile devices are highly relevant for mobile providers.
From the technical point of view, it is important to understand the mobile data traffic that is
generated by such communication services. In 2011, WhatsApp announced that, globally,
they deliver one billion messages per day [53]. In comparison to traditional SMS, it is
characteristic for WhatsApp messages that they include a high percentage of media
objects generated by the user itself. This includes pictures, videos, and sound recordings
that users share with single contacts or a whole group. WhatsApp is most prominently
Public

seen in the Asia-Pacific region, where, during peak hours, it occurs in the top ten services
that are contributing to the upstream traffic in mobile networks. According to another
report by Sandvine [21], it accounts for up to 2% of the overall uploaded data during these
hours. In an example mobile network with 1 million subscribers, about 7% of them were
using WhatsApp, sending in average 12 messages each hour during an average day.
Another kind of applications, such as Viper and Skype, targets real-time voice
communication. In comparison to WhatsApp, they produce longer lasting flows of data
with rather small bitrates. For voice traffic, quality of service aspects such as low delays
and jitter are very relevant, whereas for sharing media objects, e.g. via WhatsApp, bursty
traffic with no tight timing requirements is the reality.
4.9.1 Viber
Viber [50] is a VoIP and Instant Messaging (IM) service for mobile users. The introduction
of the service took place on December 2010. The user needs to install the Viber client in
his mobile device and register in to the Viber network. After the registration process the
user can invite other users, and/or search which of his contacts are already registered in
to the Viber network. After the registration completeness the user has access to a variety
of services like IM, group IM, IM delivery report, full-duplex (FDX) voice calls, images
exchange, and geo-location information sharing.
The registration process demands from the user a valid Mobile Subscriber Integrated
Services Digital Network-Number (MSISDN). The user provides to the network his
MSISDN through the client. Then, the user receives an SMS with a verification code. After
entering the verification code the user is granted access to the network. Only one client at
a time is allowed to be paired with an MSISDN. However, the client can run in a different
device than the Subscriber Identification Module (SIM) card is installed.
Currently the Viber client is available for the majority of the Operating Systems (OSs) used
by the majority of the mobile devices vendors (i.e., iOS, Android, Windows Phone,
Blackberry, Nokia, and Bada). The client is available in ten languages and the services
provided to the Viber network users are free. To our knowledge, there is no significant
research, neither a technical traffic evaluation concerning the Viber protocol, nor
information concerning the call-accounting schema if any.
4.9.2 HeyTell
HeyTell [51] is an instant voice messaging service for mobile users. HeyTell is a Half-
DupleX (HDX) communication application. The user need to install the client, which is
currently available for iOS, Android and Windows Mobile devices, register to the network
and start using services like instant voice messaging, group instant voice messaging, geo-
location sharing, and voice transformation effects. The client as well as the service is
provided for free. However, some add-on services like group instant messaging capability
and voice transformation effects are charged. The user can register to the network either
by his MSISDN, or his e-mail address. Then the user needs to invite all the users that he
wish to establish a communication with. After both users accept the invitation they can
exchange instant voice messages. The client can equally transmit messages over any
available Internet connection. Since the communication is HDX the quality of the sound is
not affected from by quality of the network (e.g., bandwidth, latency). The only relation to
the network quality is the maximum time needed for a message to be transmitted.
Similarly to Viber, on our knowledge there is no significant research, or traffic analysis of
Public

the protocols used in HeyTell, neither any service related accounting information, nor any
Mobile Network Operator (MNO) that blocks this specific service.
4.9.3 AbaCUS
AbaCUS is an application that integrates VoIP solutions in order to support the Auction-
based Charging and User-centric System described at [54], while it can also act as a VoIP
client. The subscribers of every MNO might act as a caller, as well as a callee.
Concerning the caller role, the MNOs subscribers will define their QoS demand per call,
and their maximum cost tolerance for the requested service. Furthermore, the application
will consult third party providers, such as social networks, in order to retrieve all the
necessary information for the AbaCUS system, like the callees IP address, his home
MNO, his current location, the caller preferences for potential multimedia, or
advertisement content while the service is being initiated, etc. Finally, the initiation of the
call in the auction-based environment will be handled by the application. However, the
respective steps defined by the AbaCUS protocol will not be transparent for the caller,
since that could negatively affect the overall call procedure experience.
The MNOs subscribers, concerning the callee role will have to set, and update when is
needed, their home MNO and provide access to the application to switch between
different MNOs. Thus, the AbaCUS application will handle the temporary MNO switching
for the callee. Last but not least, the AbaCUS application both on the caller and the callee
side, acts (1) as a QoS measuring mechanism, and (2) as a reporting system when a QoS
agreement is violated. The information collected during the QoS measuring process will be
available to the stakeholders.
The presented special applications for mobile devices are important to be considered in
the analysis of todays content delivery and communication applications. As presented,
WhatsApp, Viber and HeyTell have a large user basis and contribute a small but,
nevertheless, relevant part to the overall upload traffic share in mobile networks. AbaCUS
is using charging as a provider selection solution for the mobile voice services, thus
relating also to management of this traffic. The importance of this type of traffic
management is mainly due to the fact of the significantly higher cost, compared to the
mobile data traffic, for the end users. Furthermore, the aforementioned applications can
be valuable source for the analysis of social interdependencies between users as they
have detailed knowledge on the communication pattern and the social graph of active
users. It should be evaluated if and how such information could be accessed to enhance
content delivery mechanisms.
Besides their use as source for social information, the management of traffic generated by
mobile instant messaging and VoIP applications is out of the scope of SmartenIT due to
the fact mainly that these applications are not cloud-based. Other limitations include the
fact that they generate extremely low amounts of traffic, whereas in comparison to, e.g.,
online social networks where often the content distributed is meant to be radically spread
among very large populations, i.e., viral effects, for content that is sent via applications like
WhatsApp in most cases this is not the case.
Public

4.10 Summary of Overlay Applications Characteristics
In Chapter 4, we have overview 9 categories of overlay applications. The large majority of
these categories, e.g., video streaming, P2P video streaming and file sharing, online
storage, online social networks, cloud CDNs and online gaming refer to cloud-based
applications which generate significant traffic volumes. Moreover, applications belonging
to the aforementioned categories are responsible for increased energy consumption either
in data centres, e.g., video streaming, online storage and search engines, or in end-users'
mobile devices, e.g., mobile video streaming, mobile P2P video streaming and mobile P2P
file sharing.
On the other hand, one category, namely mobile instant messaging and VoIP, may not be
generating high traffic volumes or energy consumption, though they might be of some
interest to our investigations, as they may reveal social relationships between users; thus,
information extracted by them can potential serve as input to socially aware traffic
management mechanisms to be designed, implemented and evaluated by SmartenIT.
Moreover, apart from well-established applications, we have also addressed some
emerging ones. The reason for addressing also some less known applications is the fact
that they can constitute nice examples for evaluating traffic management mechanisms
where intervention potential is higher than other well established proprietary applications
where intervention may not be applicable.
Summarizing the investigation conducted in this chapter, we have provided description of
the aforementioned applications from multiple viewpoints, in terms of their operation,
traffic characteristics, energy consumption, and end-user's device. The identified
characteristics will play fundamental role and will provide input to the classification of cloud
services, which will be performed by Task T1.1 and will be described in D1.2, where also
final decisions on the selection of applications to be addressed by SmartenIT are to be
made.
Finally, we highlighted relevant to SmartenIT characteristics of new overlay applications,
i.e., related to the main concepts and scenarios of applications, as well as we have
discussed potential for intervention and optimization by traffic mechanisms to be designed
by SmartenIT in WP2.
Table 4-5 summarizes the characteristics of the overlay applications categories described
in this chapter taking into account the traffic characteristics and energy consumption
considerations for cloud services discussed in Section 3.6 and Section 3.7, respectively.
Public

Table 4-5: Summary of cloud-based applications' characteristics.
Type of
Application
Video streaming
Mobile P2P
streaming
Online
Storage
Search
Engines
Online Social
Networks
Online
Gaming
Cloud CDNs
Mobile P2P
File Sharing
Traffic
generated
Very high
High; expected
to increase
Medium;
expected to
increase
Low to medium
(due to the
magnitude of
queries)
Medium to high High Medium to high
Low to medium;
expected to
increase
Energy
consumption
Very high; both in
data centres and
mobile devices
High; mainly for
mobile devices
High; mainly in
data centres
High; mainly in
data centres
Medium to high;
mainly in data
centres
No data;
expected to be
high mainly in
data centres
Expected to be
high mainly in
data centres as
for traditional
CDNs
High; in mobile
devices
QoE concerns
Long startup
times and
stalling; longer
startup could
reduce stalling
(critical to QoE)
Long startup
times and
stalling; longer
startup could
reduce stalling
High latency
Accuracy of
results, high
response times
Increased latency
Increased
latency; critical
to QoE
Higher
response times
than traditional
CDNs
High download
times
Potential for
intervention
Low to medium;
e.g., alternative
streaming client;
incentives need
to be addressed
Low to medium;
e.g., alternative
P2P client;
incentives need
to be
addressed
Medium; incenti
ves need to be
addressed
Low;
proprietary
solutions
integrated with
other services
by the same
operator
Medium to high;
depending on
available social
information;
incentives need
to be addressed
Medium (or low
to medium);
incentives need
to be
addressed
Medium (or low
to medium);
incentives need
to be
addressed
High; multitude
of solutions for
P2P file
sharing can be
extended
Potential for
optimization
High; unburden
data servers in
terms of energy
consumption,
traffic
localization,
improved QoE
High; reduce
energy
consumption
High; improve
QoE for end-
users, reduce
energy
consumption
Medium to low;
reduce energy
consumption in
data centres
Medium to high;
traffic
localization,
improve QoE,
reduce energy
consumption
High; traffic
localization,
improved QoE,
reduction of
energy
consumption
High; traffic
localization,
improved QoE;
reduce energy
consumption
High; traffic
localization,
improved QoE

Public

5 Stakeholders Characterization
In this chapter, we identify major stakeholders involved in cloud services provisioning
scenarios, as well as to investigate their (possibly conflicting) interests, and analyse their
interactions and potential for collaboration among them, their technical dependencies and
business relationships. By the term stakeholder designates an entity which supervises or
makes decisions that affect how the Internet ecosystem operates and evolves. A same
entity often plays multiple roles, for example ISPs offer connectivity services but at the
same time can provide entertainment content services.
As mentioned above, stakeholders characterization depends on specific scenarios. In this
chapter, we indeed describe four scenarios of interest: (i) inter data centre / inter cloud
communications, (ii) collaboration between data centres for energy efficiency, (iii) global
service mobility, and (iv) exploitation of social awareness for content delivery. Three of
these scenarios, in particular, (i), (iii) and (iv), were also discussed in SmartenIT's DoW
[204], whereas the fourth one, i.e., (ii) was introduced in the SmartenIT kick-off meeting.
Note that we chose to present the latest identified scenario right after the inter data centre
/ inter cloud communication, as they are more closely related.
All four scenarios reflect the main objectives and challenges faced by the project, i.e.,
economic considerations and incentives for collaboration, energy efficiency, social and
QoE awareness. Nevertheless, the description of scenarios in this chapter is not final and
will be further evolved by Task T1.4, while the final scenarios description is to be provided
in D1.2. The stakeholders characterization, as well as the technical dependencies and
business relationships between them, will be taken into account both by WP1, especially
in the decision making for the selection of applications to be addressed, and WP2, in the
definition of relevant use-cases by Task T2.2, and mainly the specification of traffic
management mechanisms by Task T2.3. Specifically, the traffic management
mechanisms will be designed so as to address the involved stakeholders' incentives and
lead the system to a stable outcome [105]. Moreover, the business relationships and
technical dependencies will also assist designers, i.e., WP3, to define the interfaces in the
inter-cloud, or ISP-to-cloud, etc. communications.
5.1 Basic Definitions
In this section, we define significant terms for the stakeholder characterization to be
performed in the next sections.
First of all, in order to perform stakeholder characterization, we need to define specific
scenarios and then identify involved stakeholders and analyse their relationships within it.
A scenario is an overall outline of real world actions and events occurring during an
interaction with the underlying SmartenIT system. It describes the general behaviour of all
system mechanisms and enables the identification of challenges and problems to be
solved in order to achieve desired functionality. Stakeholders are entities, individuals or
organizations, supervising or making decisions that affect how the Internet ecosystem
operates and evolves. The stakeholder analysis comprises of the identification and
assessment of the major stakeholders related to an activity based on their influence and
importance, while the stakeholder relationship analysis identifies the types of inter-
dependence between stakeholders and how these dependencies can be affected towards
some specific outcome.
Public

Stakeholders are considered to be rational, self-interested players; following these
interests they perform certain actions towards the satisfaction of their interests. In
complex, multi-stakeholder environments, it can be expected that different stakeholders
involved in a specific activity may result having conflicting interests. To describe the
competitive behaviour due to conflicting interests of stakeholders in the Internet, the term
tussle was introduced by Clark et al. [105]. A tussle is a process in which stakeholders
inter-actions usually lead to contention. Reasons for tussles to arise in our scenarios of
interest are manifold, e.g., overlay / cloud traffic management and routing decisions
between autonomous systems constitute a typical example for tussle space. Thus, with
the ongoing success of the Internet and with the assumption of a future Internet being a
competitive marketplace with a growing number of stakeholders such as users, service
providers, network providers, and cloud providers, tussle analysis becomes an important
approach to assess the impact of stakeholder behaviour.
To address an up-rising tussle among Internet stakeholders, incentives, as an economic
mechanism to motivate an entity to perform certain actions or to pursue certain goals, can
be the tool to motivate stakeholders towards an incentive-compatible behaviour,. In
SmartenIT, we aim to design mechanisms that address incentives of the various involved
stakeholders, so as to avoid potential tussles among them. We focus on two types of
incentives: monetary incentives, e.g., revenues for providing a specific service, and
performance incentives, e.g., enjoying high(er) QoE or experiencing low(er) congestion.
In the next sections, we describe tools and methodologies proposed in literature to
perform stakeholder relationship analysis, and finally, we combine these tools to analyse
the interests and types of relationships among identified stakeholders.
5.2 Tools and Methodologies for Stakeholders Characterization
In order to identify significant stakeholders involved in scenarios of interest, we use tools
such as Value Network Configurations (VNCs), and employ methodologies proposed and
used before in literature such as the Business Model Reference Framework (BMRF)
presented in [108].
Moreover, in this section, we briefly introduce the Tussle Analysis Framework (TAF) [106]
proposed by SESERV [107], despite the fact that to this end it only partly employed in our
investigations. In particular, the stakeholders identification which is the first step of our
investigation is based on TAF's stakeholders identification map.
Besides that, another reason for introducing TAF is that it is a useful tool for analysing
stakeholders' relationships and interactions, and qualitatively assessing the adoption and
evolution of specific network mechanisms or architectures in a long-term time scale. Thus,
TAF is aimed to be employed by Task T2.4 to specific scenarios and use-cases, after
initial versions of traffic management mechanisms are considered by Task T2.3. Then, the
outcome of the theoretical investigation by means of TAF will provide feedback to Task
T2.3 on the specification of the mechanisms.
5.2.1 Value Network Configuration
A Value Network (VN) [113] is a business analysis perspective that describes social and
technical resources within and between businesses. The nodes in a value network
represent stakeholders or actors. The nodes are connected by edges which represent
interactions, and tangible / intangible deliverables. These deliverables take the form of
Public

knowledge or other intangibles and/or financial value. Value networks exhibit
interdependence.
The VNC method has been proposed in [112] and constitutes a particularly useful tool for
stakeholder relationship analysis as it separates the stakeholders (or actors) from the
functional roles the actors can take, thus allowing to analyse multiple role combinations
instead of limiting to a single value network. Additionally, the VNC method emphasizes
technologies role in defining the possible value networks and identifies the technical
components and technical interfaces between them. Thus, the method improves our
understanding of the relationship between the technical architecture, i.e., a set of technical
components linked to each other with technical interfaces, such as protocols, and the
value network configuration, i.e., role division and related business interfaces among
actors.
In Figure 5-1, the basic notation of the VNC method is illustrated. In particular, the larger
rectangle with rounded acnes represents a stakeholder (or actor), while the smaller one
inside it a role played by that stakeholder; note that multiple roles can be played by the
same stakeholder, e.g., a cloud operator may provide both IaaS and PaaS services. The
bold arrow is single-directional and represents a product or service delivered from a
stakeholder to another one, where the one receiving the service / product is located on the
side that the arrow is directed to. On the other hand, the medium and light arrows are bi-
directional and represent a business and a technical interface between two stakeholders,
respectively.
Stakeholder/Actor
Technical interface
Business interface
Product/service
delivery
Role

Figure 5-1: VNC notation.
The VNC method can be nicely combined with two methodologies for business
relationship analysis, i.e., BMRF, which partially incorporates VNs, as well as TAF [113].
5.2.2 Business Model Reference Framework
BMRF is an adaptation/customization of the framework introduced by Ghezzi in [109] and
refined in [108]. It is based on the conceptual framework originally developed by [114],
who claims that the business model components broadly refers to the underlying concepts
of 'control', i.e., the inter-firm or VN relationships that the firm is involved in and controls
over, 'value', i.e., the way a firm creates actual benefits to its customers and to itself
through its value proposition and financial configuration.
The proposed reference framework attempts to shed light on the core Business Model
(BM) design parameters for stakeholders in the SmartenIT scenarios of interest. Taking
also into account the related literature, according with a value proposition / revenue
Business Model, a set of three core building blocks / design themes was identified: Value
Proposition (VP), Value Network (VN) (interconnection) and Financial Configuration (FC).
This means that the BM starts with the VP, describing the customers benefits and
explaining how they are achieved by the architecture of the value chain and how they are
Public

monetized by the revenue model, i.e., FC. BM provides the required understanding of the
way the business entities transform their input to monetizable output. This is necessarily
linked to the value chain structure, which provides the means of performing an external
business ecosystem analysis (rather than focusing to just one actor) from both a business
and technology perspective. The VN model depicts the network of relationships and inter-
dependencies among firms that can be structured on several levels or layers. To this end,
a set of key layers and variables are identified and then employed to perform the value
network analysis of scenarios of interest, thus identifying open issues, stumbling blocks
and both business and technical opportunities. The definition of the key variables and
roles characteristics is complemented with a description of stakeholders (seller-buyer)
relationships, the direction of services provisioning and money apportionment along the
value chain. The latter must inevitably take into account (even implicitly) the actors
incurred costs, so it is important to be able to identify them per actor in scenarios where
multiple actors compose services.
To this end, the proposed frameworks three design themes have initially been divided into
13 parameters. The value range of each of the parameters has been adapted (where
needed) with respect to the initial proposal of [108] so as to capture the SmartenIT needs
and focus, while one of the 13 parameters (i.e., Key Players) has been omitted (but
mentioned below for completeness reasons) since it results in information repetition as
this is included in the SmartenIT scenario template. Therefore, the SmartenIT business
model framework consists of three design themes, each pertaining to four key parameters.
The value range illustrates the main feasible strategic decisions of each of the value chain
actors under a specific scenario. The reference framework is depicted in Table 5-1.
Table 5-1: Business Model Reference Framework.
Business Model
Parameter
Value Range Value Network
Value Proposition Parameters
Product/Service
Delivered
Basic connectivity
SmartenIT Service
Content
Basic Connectivity and Content Provisioning are already
provisioned in the current VN. However, the presence of the
coordination function performed by CDN Providers together with
the establishment of a SmartenIT Intermediation with roles of
matching between interconnection demand and offer, brokering
and advertising of interconnection agreements, QoS and QoE
Management, SLA definition, monitoring and enforcing.
Target Customer Content Provider
End user
Network Provider
Data Centre
OTT Provider
The VN provides a clear picture of how different types of actors
are interconnected with each other, and particularly the
interconnection resulting from a producer-consumer relation.
Moreover, through the interconnection designed, the possibility for
Content Provider and End User to be target by different
stakeholders is disclosed.
Customer Value Basic connectivity
Packaged Service
Content
With regard to the service/product delivered, as described above,
the VN is conceived to disclose how different players could
actually exploit their customer value.
Resources &
Competencies
Technology-oriented
Content-oriented
Both resource and competence are enclosed in the VN picture,
according with the activities and functionalities carried out by each
stakeholder.
Value Network Parameters
Public

Vertical Integration Infrastructure Layer
coverage
Internet Service Layer
coverage
Complete Integration
According to the choice made by each actor within the value
chain, whether focus on one or more service layer provisioning,
the specific stakeholder will cover different activities
encompassed in the VN structure.
Moving from the bottom side up, the activities belonging to the
infrastructure layer shift to the service layers one.
Customer
Ownership and
relationship
Intermediated
Direct
Though the direct option, is the preferred one, and is highly
considered in the VN picture, the introduction of an Intermediation
(SmartenIT / Marketplace) with roles of matching between
demand and supply could reduce the dependence on actors
directly managing the offer and revenue flow from/to the
customer.
Interconnection
Modality
Transit prevalence
Peering prevalence
Marketplace
Abstract/Cloud
The network interconnection modality configuration. The VN
proposed, comprises this opportunity for new network products
with also the already existing transit and peering interconnections,
which are relevant in the current marketplace at wholesale level.
Key partners

CP, OTT, Data Centre
Marketplace /
Intermediary
Vendors
The VN embraces all these possible alternatives, indeed the
activities performed by Content Application Provider, Content
Distribution Network and Network Service Provider, are
connected one another in order to represent all these
opportunities.
Content-Data
delivery model
Client-server
Cloud
P2P
Content Delivery
Network (CDN)
To the traditional Client-Server model, the VN with the integration
of the distributed configuration adding the technical solutions
regarding the Cloud Computing and the Content Delivery Network
present all the possible delivery model and all its combinations
increasing the interconnection the service management systems
optimization, scalability and flexibility.
Financial Configuration
Revenue Model Usage-based fee
Subscription
Free
Bundled
According to this parameter, the VN does not imply the adoption
of a specific model, i.e. the revenue model will vary according to
the strategic decision of the specific stakeholder rather than the
position on the VN position.
Revenue Sharing
Model
Present
Absent
As mentioned above, the introduction of the Intermediation
Marketplace Community with roles of matching between
interconnection demand and offer, brokering and advertising of
interconnection agreements, QoS and QoE Management, Service
Level Agreement definition, monitoring and enforcing, open up the
opportunity of implement new forms of revenue model, such as
the revenue sharing model.
Traffic Charging
scheme
Receiving Party Pays
Sender Party Pays
Congestion Charging
Absent
VN network structure does not provide an indication on how
stakeholders should choose between different Traffic Charging
Scheme, however the model proposed seems to be more suitable
to the Sender Party Pays and Congestion Charging schemes.
Cost Model Concentrated
Investment
Joint Investment
As for Revenue Model, the introduction of the Intermediation
Marketplace with the roles of matching between demand and
supply, brokering and advertising of SmartenIT agreements, QoS
and QoE Management, Service Level Agreement definition,
monitoring and enforcing, could incentivise the choice of joint
investments.
A main feature of this framework is that entails an attempt to cross Business Model
Parameters with Value Network Characteristics, thus providing a complete picture of the
actors and the nature of their relation under a service scenario. Key actors can utilize this
framework while mapping their service-specific VN and BMs of the key stakeholders
involved. The association of the different combinations of BM parameters, with each
relative variable value, and the currently models adopted, should determine the refinement
process through which, the different scenario will undergo, shaping the business models
Public

of the key stakeholders, mapping the activities and the functionalities carried out within the
provision of a specific service, taking into account the relationships and interconnections
described in the VN framework.
5.2.3 Tussle Analysis Framework
In [115], a framework is described that extends the methodology proposed in [106] for
identifying and assessing socio-economic tussles in large, highly dynamic, multi-player
environments such as scenarios of cloud services provision. The approach proposed in
[115] is a top-down one starting from generic functionalities and studying related tussles,
while the approach in [106] was more of a bottom-up: starting from tussles and then
consolidating them.

Figure 5-2: Tussle analysis methodology - The SESERV project [107].
The tussle analysis methodology as described in [115] is composed of three steps (see
Figure 5-2) and can be executed recursively; the three steps are as follows:
- Identify main Internet and cloud computing functionalities and potential bottlenecks,
i.e., critical control points and/or scarce resources (step 0),
- Identify major stakeholders for each networking functionality and cloud computing
functionality and their interests (step 1),
- Identify tussles (step 2) and provide scenarios of their evolution by examining
whether technologies designed by those research projects meet major
stakeholders interests (step 3).
Concerning SmartenIT's investigation in this chapter, we focus here on the first part of
step 1, namely the stakeholder identification. In [115], an indicative taxonomy of Internet
stakeholder roles has been formed, by studying a number of European research projects
and identifying missing roles from previous attempts and taking into account interactions
with the FISE community. Figure 5-3 depicts the full stakeholders map where seven major
categories can be seen; these seven categories are broken down to multiple types of
stakeholders. Next, the second part of step 1, namely the stakeholders interest
identification is a procedure that depends highly on the specific scenario which the tussles
analysis methodology is applied to.
Public

Figure 5-3: Stakeholders map - The SESERV project [115].
5.3 Inter Data Centre / Inter-Cloud communication
In this section we focus on the inter data centre/inter-cloud communication scenario. In the
analysis to follow, we deliberately focus on the inter data centre communication scenario,
as the most concrete one between the two. This choice is motivated by the need to focus
on a concrete scenario, the existing market needs (also identified in Section 3.6 of this
deliverable) and the related research works.
5.3.1 Scenario
A concrete instance of the latter is the NetStitcher system [116] that employs a network of
commodity storage nodes that can be used by data centre operators to stitch together
their unutilized bandwidth, whenever and wherever it is available. Such storage nodes can
be co-located with the data centres or available at other parts of the network. It is clear
that this research work is one concrete instance of the SmartenIT inter data centre
communication scenario (and potentially service). For this service, some store-and-
forward algorithm is envisioned for the chunks of bulk data that need to be transmitted
from source to sink data centre(s), as depicted in Figure 5-4. Scheduling should take into
account the availability of leftover bandwidth at different parts of the network as well as
storage and bottleneck links constraints. This way, an optimal store-and-forward schedule
is constructed and the system maximized the utilization of leftover resources.
Therefore, this scenario depicts the need to efficiently and in-time carry traffic among data
centres that are located in different parts of the network so as to comply with certain
market-driven requirements and deadlines. Below we provide two concrete use cases
Public

envisioned under the inter data centre communication scenario. Note that energy
efficiency is not included here since it is handled in Section 5.4.
- Fault tolerance services for data centres: Data of a data centre are periodically
(e.g. every 24 hours) replicated to a backup facility located at a remote location of
the network, thus increasing redundancy and security.
- Customer experience enhancement: Replication of data across the network is
crucial for large businesses (e.g. CDNs, social networks such as Facebook) in
order to push the content or service closer to the end user (corporate or residential)
who consumes them. To this end, a SmartenIT inter data centre communication
service could serve as a backend service for supporting the data replication.

Figure 5-4: A source data centre uses available bandwidth to perform bulk backup to sink
data centre [116].
In order to better illustrate the ideas and apply the relevant tools, we now provide a
concrete example for the Fault Tolerance services use case, taken from [116], which is
also in the scope of SmartenIT: Assume that a data centre in New York wants to back up
its data to Palo Alto. Due to the different load and thus storage availability in the two data
centres, direct transfer of the data between them may not be feasible in a common time
window. However, taking advantage of the availability of intermediate storage nodes and
the available capacity of the backbone links over different time zones, this can be feasible
if the intermediate data centres/storage nodes of Boston, Chicago, Phoenix, Denver and
Seattle are used. Note that these data centres may be in general served by multiple ISPs;
their interconnection is possible due to the Transit ISPs that sell global connectivity
network service. This transit service is seamless to both the data centres and the end
customers who are purchasing Internet access from their Edge ISPs.
Note that the above scenario is an indicative case of inter data centre communication.
Simpler scenarios such as the communication of two data centres for the exchange of
data needed to perform computing operations over massive datasets (e.g. using a medical
data database in the cloud) are also possible. However, this example comprises a simple
yet illustrative use case of this type of scenarios that is representative enough to perform
Public

stakeholder analysis and gain insight on the market needs and solutions SmartenIT is
suitable for.
The big picture capturing both the aforementioned use cases (depending on the
placement of data centres/caches is depicted in Figure 5-5. Note that it is possible that
Source and Destination ISP are direct neighbours (there is a direct link between the two
AS Border Routers) or interconnected via one or more Transit Connectivity Providers. This
difference will only impact the number of network connectivity providers that must
intervene in the service and will be present in the respective value network configurations
and the respective business models and money flows.

Figure 5-5: The scenario use cases and network placement of the data centres.
5.3.2 Stakeholders, Roles, Interests
We employ the stakeholders taxonomy described in Section 0 to identify all involved
stakeholders and to describe their role. Table 5-2 includes the identified stakeholders,
their role description and a preliminary identification of their interests.
Table 5-2: Stakeholder identification for inter data centre / inter-cloud communication.
Stakeholder Role(s) Interests
Connectivity providers
Edge ISP Provides connectivity to end-users
Reduce / keep low inter-connection
costs; avoid network congestion
Transit ISP
Provides inter-connectivity between the
various service providers or edge ISPs;
provides inter-connectivity between the PoPs
of one service provider
Sell transit to Edge ISPs or large
business customers (CDNs, Internet
giants), peer with other Transit ISPs;
avoid or limit links congestion, apply
hot potato routing when possible
Information / service providers
Data Centre
Provides the data centre services, i.e.,
storage of large amounts of data and possibly
replication to multiple locations.
Provide high QoE to end-users;
reduce inter-connection costs;
reduce storage overhead and costs;
reduce energy consumption across
his own infrastructure
Application
Provider
Provides the high-level service of inter data
centre communication management (e.g. the
NetStitcher) service, i.e. the schedule and
Provide high QoE to end-users for a
competitive price; reduce network
costs; utilize network resources
Public

traffic management, so that customer
objectives are met; this service can be
provisioned under the SaaS model; may own
and control data centres or storage nodes;
decides on the data replication schedule, thus
operating on top of the network and data
centre infrastructure. (This stakeholder may
be integrated with the data centre role.)
efficiently and achieve multiplexing
gains for his customers
Infrastructure providers
Cloud operator
Provides IaaS (servers, links) to Application
providers and Data centres (as well as end
users) so that E2E data transfer from source
to sink data centre is made feasible; owns
and controls several (intermediate) storage
servers across the globe; owns and controls
links (even multiple ASes) among his servers,
or uses Transit ISP for interconnection.
Provide high QoE to service
providers; reduce / keep low network
costs; improve performance and
scheduling of customers tasks;
improve / limit energy consumption
by his servers
Users
End-user
The customer of the data centre, typically a
business user (e.g. a multinational company
forwarding its data all over the globe to data
centres close to its PoPs)
Security, redundancy and fast
access to data; Enjoy high QoE in
terms of low latency, and high
availability, fault tolerance, low
network costs
We now provide the value network configuration for this scenario. Depending on the level
of (vertical or horizontal) integration in the market, there can be a variety of possible value
network configurations. This is due to the fact that the roles envisioned in the market and
their separation or integration by stakeholders may vary.
In order to highlight this, we begin the presentation with the simplest possible configuration
where the (SmartenIT) inter data centre communication service is a new service offered
by and integrated in the data centre stakeholders business. This could be a valid scenario
since such an extension of the data centre functionality is likely due to the foreseen
benefits in its business and thus the potential for enhancing its competitiveness in the
business, as well as attracting additional revenue to the richer services provided. The
resulting VNC is provided as Figure 5-6. Note that under this VNC there is no separation
between the Application Provider and the Data Centre stakeholders identified in Table 5-2.
Similarly, it is typical in the European market for certain large providers to undertake both
the Edge and the Transit roles. This pertains to providers who both serve their national
markets but also have strong presence in other countries (including America and Asia).
This allows these providers to sell transit (or partial transit) connectivity services. Thus,
such provider integrate both those roles, and though they typically maintain different AS
numbers for access and transit, those services are provided by (different divisions of) the
same carrier.
Thus, the VNC of Figure 5-6 is the simplest one in terms of number of stakeholders due to
the fact that certain stakeholders integrate in their business new roles, both a) in the
network layer, where the Connectivity Provider is both Transit and Edge ISP and b) in the
application layer, where both the SmartenIT inter data communication service and the
data centre services are undertaken by the Data Centre stakeholder.
Public

Figure 5-6: Basic VNC model.
A more generic configuration, as opposed to that of Figure 5-6, where there is a clear
separation between the stakeholders that provide Data centre and inter data centre
communication services is depicted in Figure 5-7. Under this VNC, there is a clear
separate Application Provider who is offering the inter data centre communication service.
Both the network layer stakeholders and the data centres focus solely on their domain of
expertise are separate entities that conduct business with this new Application Provider.
Hence, under this configuration the SmartenIT inter data centre communication service
justifies the introduction of an OTT service provider, namely the SmartenIT Application
Provider, who under the SaaS model provisions this new service in a similar fashion that
CDNs like Akamai nowadays push content close to the networks end users.
Next, we provide as Figure 5-8 the most generic VNC where each role in both the network
and the application layer is undertaken by a separate stakeholder. Thus, there is no
integration, vertical or horizontal, in the market.

Figure 5-7: Modified generic VNC model.
Public

Note that there are several other alternatives, such as having integration of the Data
centre and SmartenIT Application Provider roles without having integration of the Edge
and Transit roles by the same Connectivity Provider stakeholder, or even have full vertical
integration in the market by an actor who undertakes the Data centre, SmartenIT and ISP
roles. Though these scenarios are valid, we omit to provide the respective VNC by means
of graphics since these variations add little or no additional insight regarding the
SmartenIT scope and stakeholders collaboration compared to the three aforementioned
VNCs. However, for completeness reasons these configurations are also briefly discussed
in the Business Modeling subsection of this analysis, which follows.

Figure 5-8: Generic VNC model.
5.3.4 Business Modeling
We now apply the business modeling reference framework to this scenario. In particular,
we select the appropriate values for the key business model parameters described in the
template so as to provide insight to the way business is conducted.
We begin our work with the first design theme, the Value Proposition, and we apply its
basic parameters to this market.
Product/Service Delivered: The SmartenIT inter data centre communication service is the
end-to-end product delivered. This comprises also custom managed network services
products (e.g. standard or premium transit agreements) that support the transfer of the
inter data centre communication traffic from the source data centre AS to the
destinations. The data centre that originates the data transfer compensates the
connectivity provider for the traffic it injects by purchasing this managed network
service. Similar interconnection products need to be purchased from the involved
networks in cases of inter-domain data centre communication. At the retail level, we
can also envision micropayments coming from the customer layer e.g. via (business)
customer subscription to the data centres for accounts that allow them to host and
replicate a certain amount of data.
Public

Target Customer: The end-to-end service target customer is the data centre who is
providing data replication/backup/mirroring/caching services to end customers (via
some portal where users subscribe). This could mostly include business customers,
including content providers.
Customer Value: The Packaged Service model prevails: Guarantees on the efficient traffic
delivery for the data centres, predictability and management of heavy traffic for the
connectivity (network) providers.
Resources and Competencies: The stakeholders utilize their resources and competencies
in their core business, i.e. is technology-oriented.
Next, the four parameters of the Value Network design theme are provided.
Vertical Integration: It is likely that large data centres that already own or are affiliated with
some connectivity provider will perform vertical integration by aggregating the roles of
Application Provider, Data centre and Connectivity Provider so as to offer advanced
data replication services to its customers.
Customer Ownership and Relationship: There is mediation of the Application Provider with
the data centres and networks involved for the efficient provisioning of the service.
Thus, there is an intermediated relationship and indirect revenue flows across the
Infrastructure and Internet Service Layer, with the exception of the scenario where
vertical integration is present in the market. In the latter case, there is direct customer
ownership, since the integrated Data Centre / Application provider has ownership and
direct interaction with the end customer.
Interconnection Modality-Business Agreements: For the agreements among the Data
Centres (and the Application Provider, if present) and the connectivity providers
involved in the service provisioning, wholesale transit or custom transit-like agreements
are envisioned. Thus, we have transit prevalence.
Content-Data Delivery Model: The CDN model is the model closest to this scenario. In
particular, the content delivery of this scenario can be seen as the evolution of existing
Akamai-like solution, with some more advanced features regarding the customer
requirements for the service.
Last, the parameters of the Financial Configuration theme are investigated.
Revenue model and revenue sharing: The services are purchased from data centres and
subsequently by business users who subscribe to those services e.g. via a Dropbox-like
portal. Thus, there are two distinct charging layers with separate/co-existing money
flows: the aggregate level traffic charging at the network layer (inter-ISP charging) and
the per-session based charging at the application layer (end user payments). In the
wholesale layer, revenue sharing schemes could be present in the cases of vertical
integration and or business alliances among data centres and connectivity providers in
the market.
Traffic Charging Scheme: For the management of the aggregate traffic flows, it is
expected that the purchaser of the involved traffic contracts will have to compensate the
involved networks (if any) for carrying this traffic on top of their network, according to
the Sending Party Network Pays principle. This means that the connectivity provider is
compensated by the source data centre and then might need to compensate additional
connectivity networks that may be involved for the delivery of the traffic at the sink(s),
proportionally to the quality attained and the volume of traffic handled.
Public

Cost Model: Concentrated Investment since the stakeholders must invest to compete for
and manage the data centre-generated traffic, whereas no Joint Investment scenario is
likely.
5.3.5 Potential for Collaboration and Relevance for SmartenIT
Inter data centre / inter-cloud communication is clearly an interesting scenario for
SmartenIT. Its scope is in the core of the projects interests. Also, a traffic management
solution for inter data centre communications is considered as an attractive solution for the
market, serving an actual market need. Moreover, the inter data centre communication
scenario is less vague than the inter-cloud communication scenario and allows us to
assume that the positioning of the data centres is known. This fact can also help the
project to promote its added value more effectively by working on this more concrete
scenario. This way, SmartenIT can make a realistic solution/value proposal to the market,
increasing its impact and visibility.
Furthermore, this scenario applies to an advanced service that has to meet different
stakeholders requirements at various levels. The network and the application layer
requirements and interests are coupled and should both be considered for providing a
realistic solution. To this end, it is clearly depicted that collaboration is needed among the
stakeholders involved in the end-to-end service. This way, the different stakeholders can
thus express their interests and constraints and find some incentive-compatible TM
solution to be applied.
Note also that this inevitable collaboration can be a major driving force towards integration
of roles and the respective product offerings and value propositions in the market. The
inter data centre communication service can be seen as an attractive niche market
solution that is also a generalization of the services already provided in this market.
Connectivity Providers, Data Centre owners, or even Application Providers, either existing
or emerging, could try to perform bundling of this new service with existing ones so as to
dominate the market and strengthen their position in the markets where they are active.
Clearly the additional offering of such a sophisticated service would be of high value to the
Data Centre's customers; thus the Data Centre owner is the stakeholder who is identified
as the most likely one to integrate this role in his business. This in turn could create mixed
incentives for collaboration and competition with the other data centres, and could also
have adverse implications on the agreements with the network layer providers, i.e., ISPs.
Thus, there is an interesting tussle and different incentives regarding who should
undertake the provisioning of this service and under what terms and with what scope.
Analysis of this issue is beyond the scope of this deliverable and is left for future work.
This scenario analysis also highlights the potential nature of the SmartenIT solution as a
whole and the placement of the SmartenIT functionality required to support this solution in
the inter data communication service scenario and also beyond. In particular, this
SmartenIT solution could be seen as a set of architecture/functional elements placed
within the aforementioned stakeholders so as to coordinate, exchange information and
enforce the schedule for the inter data centres communication. As both the
aforementioned analysis and the NetStitcher example [116] indicate, the control, market
offering and orchestration of the end-to-end inter data centres communication service
could be performed either by the Data Centre owners, which would integrate this
SmartenIT functionality in its own product offerings, or alternatively by a new Over-The-
Top provider, i.e., an Aggregator. The latter can be seen as an extension of the Akamai
model that is also differentiated with respect to that of Akamai in order to cater for the
Public

emerging Overlay Traffic needs and the more complex requirements of advanced services
(beyond caching) such as the one of this scenario: Thus, the SmartenIT OTT stakeholder
can be envisioned, serving as the meeting point/clearing-house for networks and data
centres who use its smart scheduling service in order to ensure that their interests are
considered in the service provisioning and the way traffic is managed. In both options
however, the alignment of a critical mass of the key stakeholders incentives regarding the
scheduling and management of traffic is required for the adoption of this service by the
market. This critical set of stakeholders could promote this new business role and by
taking advantage of their existing customer base and business agreements with key
market players enforce the desired solution as a de facto standard in the market. Other
players in the market would either buy this service if it matches their business needs or try
to offer similar solutions in order to be able to remain competitive in the market. This
would further strengthen the added value and sustainability of the envisioned SmartenIT
Aggregator service for inter data centre communication.
5.4 Collaboration for Energy Efficiency
In this section, we perform stakeholder relationship analysis for two scenarios where
energy efficiency is pursued through collaboration among stakeholders: in the first case,
collaboration is sought between data centres, i.e., cloud operators, while in the second
one, collaboration is considered between the data centre and the ISP, and consequently
the end-users. Note that apart from fundamental stakeholders identified in cloud service
scenarios (as in Section 5.3) such as Cloud Operator, ISP, Application provider, we also
consider here a new entity/stakeholder, namely that of the Energy Provider, and address
his role(s) and interests in Section 5.3.2.
5.4.1 Scenarios
Below, we describe two scenarios where energy efficiency is pursued through
collaboration among different stakeholders. Both scenarios are considered to be of high
interest to SmartenIT; the first one is an extension of the scenario described in Section
5.3.1, while the second one is inspired by [192].
5.4.1.1 Collaboration between Data Centres
Federation of data centres and live migration of virtual machines offer new effective ways
to manage and optimize energy consumption. This allows establishing new energy saving
policies and procedures dealing with energy demand patterns of data centres and
collaboration with energy producers. Apart from energy saving, also the exploitation of
renewable energy sources may have high priority.
Federated data centres create a decentralized energy ecosystem where the workload can
be exchanged to address energy shaping requests from the energy providers, economical
demands (lower costs of energy in certain locations) or prioritizing access to stable
renewable energy sources.
Advanced adaptive power consumption requires close collaboration between data centres
and energy providers [117]. Normally the data centres are treated as common business
customers expecting high level of energy but to achieve valuable benefits this relation has
to be enriched. There are already efforts introducing interfaces between those two sides to
share the information and dynamically adjust power consumption.
Public

The information about energy consumption and related implications can be included in
SLAs between service users and service providers. Possible and measurable limitations
could be compensated with discounts or other benefits in order to achieve high
optimization and thus lower energy consumption. On the other hand users could require
from service providers to use substantial level of green energy (or steady increase of it).
Also relevant OLAs including energy consumption aspects could be established between
data centres for workload migration and access to distributed energy sources.
Figure 5-9 presents an example with two federations where virtual machines can be
located in a federated data centre according to the optimized decisions.

Figure 5-9: An example of two data centre federations.
5.4.1.2 Collaboration between Data Centre and ISP
In [192], the issue of energy efficiency in data centres is addressed in an alternative way
by a distributed service platform, called Nano Data Centres or NaDa, based on tiny (i.e.,
nano) managed servers located at the edges of the network, i.e., users' premises. In
NaDa, both the nano servers and access bandwidth to those servers are controlled and
managed by a single entity, typically an ISP who is also providing connectivity to end-
users. The role of the nano servers can be played by ISP-owned devices like Triple-Play
gateways and DSL/cable modems that sit behind standard broadband accesses.
According to [192], such gateways can potentially form the core of the NaDa platform and,
in theory, can host many of the Internet services hosted in today's data centres, including
the increasing popularity video streaming services.
The proposed approach follows a P2P philosophy, but is coordinated and managed by an
ISP that installs and runs the gateways that act as nano servers, as well as the network
that interconnects them. Additionally, ISPs can easily implement nano servers by
providing new customers with slightly over dimensioned gateways, whose extra storage
and bandwidth resources would be used to implement services like video hosting, all of
that will be totally isolated from the end-user via virtualization technologies.
The architecture proposed in [192] is depicted in Figure 5-10 and consists of: Gateways
which provide storage, and bandwidth resources, whereas in the case of VoD service,
they store full or partial replicas of video content objects, and provide uplink bandwidth to
deliver these objects to other gateways. Additionally, a Tracker coordinates all VoD-
related activities, i.e., it monitors the availability and content possession of gateways, and
Public

answers requests for content by lists of gateways holding the desired content. Then, it is
up to the requesting gateway to download movie parts from other gateways. Finally,
Content Servers provide the content from legacy data centres or caches. They can belong
to the ISP, or to content providers, whereas their primary role is to pre-load gateways in
offline mode with content that they can subsequently re-distribute. Content servers can
also serve content requests online if no gateway can treat the request.

Figure 5-10: The NaDa architecture [192].
According to the description of the scenarios for collaboration between data centres or
data centre and ISP so as to achieve energy efficiency, we employ the stakeholders
taxonomy described in Section 0 to identify all involved stakeholders and to describe their
role. Table 5-3 integrates the identified stakeholders, their role description and a
preliminary identification of their interests.
Table 5-3: Integrated stakeholder identification for collaboration for energy efficiency.
Connectivity provider
ISP
Provides connectivity and inter-
connectivity between the cloud operators
and the application providers, provides
connectivity to end-users.
Increase revenues from inter-
connection of cloud providers; low
network cost; optimized network
bandwidth utilization (expected traffic
patterns)
Cloud Operators
IaaS Provider
Provides IaaS (servers, links) to users
(i.e., PaaS provider, application provider
or end-user); owns and controls servers
and links inter-connecting his servers, or
his servers with other PoPs; establishes
OLA with other IaaS or PaaS providers,
and SLAs with energy providers
Provide high QoE to users; reduce
OPEX (i.e., inter-connection costs,
energy costs); improve his servers
utilization; avoid / limit congestion on
his links; limit energy consumption by
his servers
PaaS Provider
Provides PaaS to users (i.e., application
provider or end-users); establishes and
manages OLAs with IaaS and other PaaS
providers, and SLAs with users and
energy providers
Provide high QoE to users; reduce
OPEX (i.e., IaaS costs); avoid / limit
overload on his platform
Public

Energy Provider
Energy Provider
Provides the cloud operators with energy
(power); may own and control power plant
(producer), or be a re-seller of energy
(transmitter)
Meet energy demand, eliminate
outages; reduce OPEX including
transmission costs; increase revenue
Users
Cloud User
Consumer of IaaS or PaaS offered by the
cloud operators, e.g., application provider,
content provider or end-user
Enjoy high QoE, availability, etc. as
dictated by the established SLA;
reduce OPEX (i.e., cost for IaaS/PaaS
service)
End-User
Consumer of cloud and connectivity
services, buys and owns equipment for
Internet access
Enjoy high QoE as dictated by the
established SLA; enjoy high
bandwidth and reduce energy cost
In the following sections, we construct certain VNCs which depict possible interactions
between major stakeholders of the scenarios under study. Initially, we consider
collaboration between federated data centres / cloud operators so as to achieve reduction
of the overall energy consumption among data centres. Practically, this means that in
order for the cloud operators to cooperate towards overall energy efficiency, they must be
provided incentives such as their individual energy and cost reduction. The
ISP/Connectivity Provider stakeholder is in purpose omitted in the VNCs for the first
scenario as its role is not primary. However, in the second scenario, the ISPs role
becomes central as his collaboration is fundamental to achieve energy consumption
reduction in the data centres.
We provide first the value network configuration for the collaboration between data centres
depicting the technical and business dependencies, as well as the products and services
provided by one stakeholder to another. Depending on the level of (vertical or horizontal)
integration in the market or the insertion of new entities, there can be a variety of possible
value network configurations. This is due to the fact that the roles envisioned in the market
and their separation or integration by stakeholders may vary.
In order to highlight this, we begin the presentation with a more generic configuration
where there is a clear separation between the stakeholders who provide cloud services
focusing solely on their domain of expertise without undertaking additional roles. The
resulting VNC is provided as Figure 5-11. Note that under this configuration, IaaS and
PaaS Providers are considered to be separate entities, thus no vertical integration applies.
There applies though some horizontal integration, in the sense that multiple roles may be
integrated under one stakeholder.
Note that the generic VNC model of Figure 5-11 assumes no horizontal or vertical
integration in the market. Globally, this specific setup is met quite often, e.g., IRT offers
IaaS services to PaaS providers. However, another setup that often appears is the
scenario where the IaaS and PaaS are provided by one stakeholder in an integrated
fashion, e.g., Amazon or Google. Figure 5-12 depicts such an integrated setup. The main
difference to the previous VNC is that OLAs are now required between different cloud
operators, i.e., different instances of IaaS/PaaS provider. Still, a hybrid scenario may also
occur, e.g., integrated IaaS/PaaS operators may collaborate with IaaS or PaaS only
providers so as to achieve energy efficiency.
Public

SLA Mgmt Energy
Transmission
Energy
Production
Power Plant
Ownership
SLA Mgmt
SLA Mgmt OLA Mgmt Platform
providion
Infrastructure
Ownership
Infrastructure
Provision
OLA Mgmt
Cloud
Service
Consumption
Energy Provider
IaaS Provider
PaaS Provider Cloud Service User
SLA Mgmt
SLA
OLA
SLA

Figure 5-11: Generic VNC for separation of IaaS - PaaS Providers.
SLA Mgmt Energy
Transmission
Energy
Production
Power Plant
Ownership
OLA Mgmt
Infrastructure
Ownership
Infrastructure
Provision
SLA Mgmt
Cloud
Service
Consumption
Energy Provider
IaaS/PaaS Provider Cloud Service User
Platform
Provision
SLA Mgmt
SLA
SLA

Figure 5-12: Updated VNC for integration of IaaS - PaaS providers.
Next, we consider an alternative VNC where though a new entity is inserted called Energy
Broker (see Figure 5-13). The Energy Broker is responsible for orchestrating the
collaboration between cloud providers so as to achieve energy efficiency. Thus, a new role
is introduced here, as so far collaboration was achieved so far in a distributed manner
between the federated data centres. The Energy Broker can be controlled by either a
member of the federation, or by a trusted third-party.
Public

SLA Mgmt Energy
Transmission
Energy
Production
Power Plant
Ownership
Energy
Orchestration
Infrastructure
Provision
Cloud
Service
Consumption
Energy Provider
IaaS/PaaS Provider Cloud Service User
Platform
Provision
SLA Mgmt
Energy Broker
OLA Mgmt
SLA Mgmt
OLA
SLA
SLA

Figure 5-13: Extended VNC to depict the insertion of the Energy Broker.
Finally, in Figure 5-14, the VNC for the collaboration between the data centre and the ISP
is provided. It can be seen that part of the cloud service is migrated to the End-Users.
Additionally, in this VNC, we also illustrate the delivery of energy (power) to End-User by
the Energy provider, as the energy consumed is significantly affected due to the cloud
service migration at the last mile.
SLA Mgmt Energy
Transmission
Energy
Production
Power Plant
Ownership
Cloud
Service
Consumption
Energy Provider
End-user
SLA Mgmt
SLA
Application
provider / VoD
service
Inter-
Connectivity/
Connectivity
OLA Mgmt
ISP
Cloud/ISP
service
consumption
Infrastructure
Ownership /
Sharing
Cloud operator
Infrastructure
Ownership
Infrastructure
Provision
Platform
provision
SLA Mgmt
SLA
SLA
OLA Mgmt
OLA

Figure 5-14: Extended VNC to depict the distributed approach (NaDa).
We apply the BMRF presented in Section 5.2.2, to build the business model for the
collaboration between data centres for energy efficiency scenario. In particular, we select
the appropriate values for the key business model parameters described in the template
Public

so as to provide insight to the way business is conducted. We begin our investigation with
the first design theme, the Value Proposition, and we apply its basic parameters to this
market.
Product/Service Delivered: The energy (i.e., power) is the main product delivered from the
Energy provider to the Cloud Providers, the ISP and the Users. By using this energy,
the IaaS Provider caters its servers and data centres, and provides storage, system
backups, etc. Additionally, the PaaS Provider and OTT/Application Provider use energy
to power their platform, and provide computation, VMs, etc. to its customers, e.g., End-
Users. Moreover, the End-Users are delivered in a retail level energy by the Energy
provider. All of the stakeholders provisioned with energy compensate the Energy
provider for the amount of energy consumed.
Secondary products in this scenario are the cloud services provided by the Cloud
operators to their Users.
Target Customer: The Energy provider addresses mainly the IaaS and PaaS Providers,
i.e., the Cloud Providers; additionally, he targets the Connectivity provider, i.e., the ISP,
the Application Provider, e.g., VoD platform, and the End-Users.
Customer Value: The customer value is guaranteed by means of OLAs between Cloud
Operators, and SLAs between the Cloud Operator and the User of the cloud service
(e.g., OTT provider or end-user), or between the Cloud Operator and the Energy
Provider that caters the first.
Resources and Competencies: All three types of stakeholders, i.e., Energy Provider,
Cloud operators, Connectivity provider and OTT / Application provider utilize their
resources, and competencies in their core business, thus the business model is
technology-oriented.
Vertical Integration: It is likely that large IaaS providers that already own or are affiliated
with some PaaS provider will perform vertical integration by aggregating the roles of
IaaS/PaaS provider.
Customer Ownership and Relationship: There can be an inter-mediated relationship
between the Energy provider and the various federated Cloud providers. In such a
case, the mediating entity is the Energy Broker and the value flows across the
stakeholders through him. Of course, a non-mediated relationship is also possible,
where the Cloud operator directly communicates with other Cloud providers and
ultimately, with the Energy provider.
Interconnection Modality-Business Agreements: There are several agreements
established in the considered scenario; the most important of which include the OLAs
among the Cloud Providers, and between the Cloud Providers and the Connectivity
Providers, and SLAs between the Cloud Operators and the Energy Provider, where the
Energy provider is penalized if violation of the SLA occurs, whereas SLAs between the
Cloud operators and the Application provider are also established, where though the
Cloud operators are the ones responsible for meeting the SLA terms. Finally, an SLA is
necessary to be established between the Cloud operator and the ISP, by which the ISP
is committed to offer a certain level of QoS in his connectivity services towards the
Cloud provider.
Content-Data Delivery Model: The model closest to this scenario is the peering ISPs
model, i.e., ISPs who establish peering agreements so as to achieve reduction of their
Public

inter-connection traffic, i.e., transit costs. The case of the Energy Broker is similar to the
P2P case when the tracker coordinates the content delivery among peers, e..g, the
federated data centres.
Last but not least, we address the parameters of the Financial Configuration theme.
Revenue model, revenue sharing & money flows: The energy provided by the Energy
provider to its customers can be charged on e.g., Time-of-Use (ToU) pricing, where
customers pay different prices at different times of the day, while on-peak prices are
higher and off-peak prices are lower than a standard rate; another pricing scheme for
energy consumption which is typically combined with ToU is Critical Peak Pricing
(CPP), where very high critical peak prices are assessed for certain hours on event
days (10-15 yearly). Specifically for large customers, such as Cloud operators, a CPP
scheme may be followed, namely the Cloud operator will pay a certain, relatively low
price, until a threshold is reached. Then, the Cloud operator will be charged with a
much higher price, e.g., penalty, for over-utilization and for exceeding the pre-defined
threshold. Finally, in the case depicted in Figure 5-13, if the Energy Broker inserted is
owned by the federation members, no money flows are foreseen. However, if the
Energy Broker belongs to a third-party, then some compensation should be provided to
him for the orchestration of the federated data centres activity.
As secondary products, cloud services will then be charged based on either on a pay-
per-use basis, or on flat rate. Finally, the Application Provider may charge the end-user
for the use of a premium version of the application with a subscription fee, and do not
charge him at all for the basic version, e.g., like YouTube, Dropbox and Google Drive
do. Alternatively, the Application provider may not charge the end-user, but follow the
ad-driven road.
Traffic Charging Scheme: Not applicable in this scenario.
Cost Model: The energy cost can be mitigated by the establishment of a federation among
cloud operators through a long-term agreement. Additionally, Cloud Operators can
make an investment so as to develop a joint platform and interfaces between their
clouds / data centres, e.g., the Energy Broker, in order to efficiently and fairly allocate
energy to servers aiming to simultaneously decrease energy consumption by all
operators in the federation.
Regarding the scenario for collaboration between the data centre and the ISP, which is
based on the scenario described in [192], the business model is similar to the one
provided for the collaboration between data centres. The main differences w.r.t. the Value
Network Design are highlighted below:
Vertical Integration: Vertical integration is possible between an IaaS and a Connectivity
Provider so as to distribute efficiently the cost of energy to the entire network, and
specifically, the gateways in End-Users' premises.
Customer Ownership and Relationship: Here, a mediated relationship is necessary. In
particular, as the cloud resources, i.e., capacity in gateways, located in End-Users'
premises are organized in a P2P manner, some orchestration is required. Thus, a
Tracker coordinates all application-related activities, i.e., it monitors the availability and
data possession of gateways, and answers requests for content by lists of gateways
holding the desired content.
Interconnection Modality-Business Agreements: The SLA between the Cloud operator and
the ISP needs to be extended to include not only ISP's commitments to offer a certain
Public

level of QoS in his connectivity services towards the Cloud provider, but also the
amount of data/traffic offloaded by the Cloud operator to the ISP's (and consequently
his End-Users') premises, the patterns of the traffic, and the related compensation for
the ISP. Moreover, a business agreement needs to be established between the ISP
and the End-Users; practically, an extension of the End-User's contract, to describe the
amount of storage and bandwidth shared from the User's gateway and the
compensation for him, e.g., monetary benefit like a reduced price for similar QoE
Internet access.
Content-Data Delivery Model: The model closest to this scenario is clearly a P2P-like
model, e.g., BitTorrent.
Regarding the Financial Configuration, the following changes apply.
Revenue model, revenue sharing & money flows: Similar revenue model and sharing, and
money flows to the previous case. Though, the End-User must be compensated for
sharing his bandwidth, storage and computational power, and increasing his energy
consumption, due to the higher power consumption by the enhanced gateways.
Compensation could be either monetary, e.g., reduction of price charged for
connectivity services, or performance-related, e.g., upgrade of the User's access
speed/package, or service-related, e.g., provision of an extra service such as free
access to the IPTV/VoD platform.
Traffic Charging Scheme: A volume-based or congestion-based scheme can be employed
by the ISP to charge the Cloud Operator for the extra traffic flowing over his network;
which has also direct impact to the energy consumption by the ISP's networking
equipment.
Cost Model: The energy cost will be mitigated for the Cloud operators by the
establishment of the collaboration with the ISP. Though, the energy cost will increase
for End-Users participating in the P2P architecture by obtained an enhanced gateway.
Additionally, the ISP will deal increased energy cost due to the extra traffic trespassing
his networking equipment. An investment can be made so as to develop a joint platform
and interfaces between the End-Users, the ISP and the Cloud provider, e.g., the
Tracker, so as to efficiently and fairly allocate energy to data servers and gateways.
Pursuing collaboration between stakeholders to achieve energy efficiency by employing
incentives is one of SmartenIT's key goals. The considered VNCs for the scenario of
collaboration for energy efficiency may constitute the basis to develop attractive solutions,
while their analysis provides some insight on what expectations each stakeholder will likely
express, and also gives some directions on how to address / meet these expectations.
Specifically, we have considered four cases. In the first one, the IaaS and PaaS
stakeholders are separate entities, e.g., a cloud provider as IRT and one of its customers,
and one where they are assumed to be an integrated one, i.e., vertical integration, e.g.,
Google. As the various blocks representing stakeholders in the VNC of Section 5.4.3
represent multiple instances of a specific stakeholder, we need to clarify that there is also
the possibility of interaction between an integrated IaaS/PaaS Operator to other non-
integrated ones IaaS and PaaS Providers. Thus, in second case, a server - customer
relationship is considered between the IaaS and PaaS providers. In the aforementioned
setups, collaboration to achieve energy efficiency takes place in a distributed manner.
Public

In the third case, the introduction of a new entity called Energy Broker is assumed. This
entity is responsible for orchestrating the collaboration between cloud providers so as to
achieve energy efficiency, and can be controlled by either a member of the federation, or
by a trusted third-party. However, in order for the Energy Broker to be controlled by one
member of the federation, then it would be necessary to achieve a win situation not only
for the entire federation, but also for each member of it in the long term; otherwise
negatively affected members of the federation will desert. Clearly, in such a setup a
centralized management of the federated data centres / Cloud Operators takes place to
achieve energy efficiency. Then, we consider a fourth case where the Cloud Operators are
not burdened, i.e., energy consumption decreases, as part of the storage/computational
service migrates to the End-Users' premises. In this setup, the Energy Broker
stakeholder's role may be played by either the ISP (Connectivity Provider) as in [117], or a
Cloud Operator, or a third-party.
Depending on the considered setup, various inter-connection agreements need to be
established between the various Cloud Operators, Connectivity Providers, the Energy
Broker and the End-Users, whereas different money flows will be generated.
A SmartenIT mechanism could be seen as a set of architecture/functional elements
embodied within the aforementioned stakeholders, or placed as an extra entity among
them, so as to coordinate and enforce the collaboration for the efficient energy
consumption behaviour among all players. In the first case, the SmartenIT point-of-
intervention would be a module or "box" within the entity controlling the data centre, which
would be responsible for coordinating with other similar "boxes" within other data centres,
and ultimately, communicating with the Energy provider. Alternatively, in the Energy
Broker case or the P2P-like one, the SmartenIT point-of-intervention is exactly this broker
or tracker orchestrating the energy consumption by the involved parties, e.g., data centres
and gateways. Of course, even in such a case, specific "boxes"/modules are necessary
within the data centres, which will constitute the interfaces for the collaboration with the
broker/tracker. In all cases, alignment of the participating stakeholders' incentives
regarding the energy consumption as described in Section 5.4.4 is required for the
adoption of such a solution by the market, at least in certain cease where all of them can
benefit.
Last but not least, a tussle analysis is needed to take place in order to identify possible
conflicts between stakeholders and to investigate the adoption of potential traffic
management mechanisms in this scenario. The application of tussle analysis however
presumes the (initial) specification of traffic management mechanisms; thus, it is left as
future work to be performed within WP2.
5.5 Global Service Mobility
Global Service Mobility relates to the services ability to follow users location. That could
refer to the ability of providing services, or delivering content as close to the user as
possible. In more detail, the Global Service Mobility goal is to serve content to possibly
moving end-users with high availability and high performance levels, by the means of,
e.g., the use of coordinated CDNs, different Cloud Providers, or distributed private data
centres.
In order to enable a dynamic service and content reallocation, services should detect
when users move to a different location. The word location means a different
Autonomous System (AS), which may, or may not, represent a different geographic
Public

location. There are well-known techniques to detect users mobility. The most used and
trivial one is to associate the users IP address to the corresponding AS, and check if the
user is accessing the service in an AS different than usual. However, nowadays, ASs can
present Virtual links to different other ASs, which makes such technique not fully realistic
to detect users location. Therefore, other methods can be combined to reach a desired
level of accuracy. One possible example would be to combine services QoE, response
time of most used services actions, traceroute-based measurement results, and AS
discovery. The need to combine several techniques always depends on the desired level
of accuracy. Moreover, accuracy also depends on how the service is architected and what
are the desired levels of performance to be reached.
There are two approaches considering the Global Service Mobility topic: (1) one-time data
move, and (2) gradual data move. Even though both approaches are different strategies,
they can co-exist as different options to enable service mobility.
In the one-time data move, the service should fully move its resources to the new location
at one action. Important aspects like how to technically migrate, where to migrate (e.g.,
other Cloud Providers), and when to migrate, should be considered.
In the gradual data move, the service should not fully adapt to the user new location
straight after users move gets detected. Instead, the service should observe (considering
a time-frame) if the user remains in the new location. E.g., when the user is temporarily
out of its usual location, gradually moves the data closer to the new location considering
most frequently accessed content. As the time passes, and considering user interaction
with the service, more data should be moved to the new user location. However, trade-offs
between long-term and short-term move should be investigated. The gradual data move
can be considered a set of smaller one-time data moves but depending on policies (e.g.,
based on most accessed data, most triggered actions in the service, etc.).

Figure 5-15: Global Service Mobility high-level approaches for moving data resources.
Public

Table 5-4 presents the stakeholders and their interests identification for the Global Service
Mobility scenario.
Table 5-4: Stakeholder identification for global service mobility.
Edge ISP Provides connectivity to end-users
Transit ISP
various service providers or edge ISPs;
provides inter-connectivity between the
PoPs of one service provider
connection of service providers due to
content delivery provision; avoid or
limit links congestion
Cloud Provider
Provides IaaS (servers, links), PaaS, or
SaaS; owns and controls several servers
across the globe; inter-connects his servers
with other PoPs
Provide high QoE and QoS to end-
users; reduce / keep low inter-
connection costs; improve his servers
utilization; avoid / limit congestion on
his links; improve / limit energy
consumption by his servers
Co-location Provider
Provides co-location services (servers,
storage, links, etc.) with a certain
commitment, differently from Cloud
Providers; owns and controls several data
centres to place co-located servers, disk
arrays, and internet links
Provide a continuous service to its
customers, respecting an SLA; Protect
co-located servers, links, storage disks
from any hazard; Co-location
Providers customers can be either
Cloud Providers or Application
Providers
Users
Application Provider
Contracts services from Cloud Providers or
Co-location Providers to enable its
businesses. E.g., Cloud Customer can be a
game developing company which contracts
IaaS from Cloud Providers in order to
provide a game service to end-users
Provide high QoE to end-users; high
availability and low service response
times; interested to motivate end-
users to consume more services
End-user
Participates in the online social network;
generates / consumes content, i.e., uploads
/ downloads videos, files, etc.
Enjoy high QoE in terms of low latency
and availability, independently of its
location; seamlessness
In this section, we describe the relationship between the identified stakeholders in Section
5.5.1. Three use cases under the Global Service Mobility scenario are presented: the first
exploring Global Service Mobility without the presence of a Cloud Provider, the second
considering a Cloud Provider, and the third assuming the Cloud Providers use the nano
data centers concept (within Edge ISPs) instead of Co-location Providers.
Nevertheless, the use cases description for the Global Service mobility scenario are
preliminary to be refined and finalized at a later stage of the project, i.e., scenario
definition in Task T1.4, use cases definition in Task T2.2 and complete business analysis
in Task T2.4. The first use case description is used to show the relations of cloud
providers in comparison to the second use case. The second and third use cases are
Public

intended to study QoS, QoE, social awareness, and especially for the third use case:
energy efficiency.
5.5.2.1 Application Provider not using Cloud to enable Global Service Mobility
Global service mobility can be achieved without the use of Cloud Providers. However, to
achieve the mobility property, the Application Provider should manage users data on its
own, contracting a Co-location Provider to place its own servers and storage disks in a
third-party data centre facility. Without the use of a Cloud Provider, the Application
Provider should detect where end-users are located, detect users moving from/to different
locations, and therefore move their data (or resources) from/to another data centre
possibly in a different Co-location Provider.
Connectivity
Provision
DataCenter
Location 1
DataCenter
Location n
Inter-
connectivity
Provision 1
Content
Provision
Content
Consumption
End-User
Datacenter (Co-location provider)
Transit ISP
Edge ISP
Inter-
connectivity
Provision n

Figure 5-16: Application Provider without using Cloud to enable Global Service Mobility.
Figure 5-16 depicts an Application Provider which should move end-users data from data
centre 1 to data centre 2, in order to enable Global Service Mobility and, therefore, provide
a high QoS and QoE. Note that data centres are contracted in a co-location fashion (within
a Co-location Provider), where the Application Provider has physical servers in a data
centre, having the responsibility to not only manage end-users data but also to manage
the whole IT infrastructure solution from the hardware to the application. In this case, the
Co-location provider just has an SLA agreed with the Application Provider, to ensure the
access to space, power, cooling, and physical security for the Application Provider
servers, storage, and networking equipment.
Public

5.5.2.2 Application Providers using Cloud Providers to enable Global Service
Mobility (UZH)
In this case, Application Providers use Cloud Providers to enable Global Service Mobility.
With the use of Cloud Providers, Application Providers do not need to manage the whole
IT infrastructure solution in a Co-location Provider and, in order to move end-users
data/resources back and forth, benefit from the elasticity and on-demand properties
inherent to cloud computing. Therefore, it is possible to manage end-users data/resources
focusing just on the application layer and moving policies (e.g., when to move based on
most triggered actions, most accessed data, etc.).
Figure 5-17 depicts the scenario, showing the relations between the Application Provider
and the Cloud Provider. It is important to highlight that, with the use of Cloud Providers
and optimizations made within cloud environments, the Global Service Mobility can be
enabled also considering energy savings since the management part of clouds can be
optimized, e.g., to share physical nodes to mirror data around the globe, or to process
data resources faster in a pool of servers.
Connectivity
Provision
IaaS
Provisioning 1
IaaS
Provisioning n
Inter-
connectivity
Provision 1
Content
Provision
Content
Consumption
End-User
Cloud Provider
Transit ISP
Edge ISP
Inter-
connectivity
Provision n
Content
Cache

Figure 5-17: Application Provider using Cloud Providers to enable Global Service Mobility.
5.5.2.3 Application Providers using NaDa to enable Global Service Mobility
Application Providers can use nano data centres to enable Global Service Mobility. Nano
data centres have the goal to create a distributed service platform on tiny servers locate at
the edges of the network (Edge ISPs). In NaDa, the nano servers and the link access to
these servers are managed by the Edge ISPs. Other use cases are possible where end-
users manage nano servers that would enable Application Providers, which are aware of
nano servers to form a Private Cloud.
Public

In this case, we focus on the NaDas managed by ISPs. Therefore, we assume the
following scenario described in Figure 5-18. Edge ISPs has the responsibility to either
provide connectivity to end-users as well as to provide the infrastructure to Cloud
Providers. In this manner, Edge ISPs act like Co-location Providers, but providing servers
as close to end-users as possible.
When Cloud Providers detect end-users moving to another location, their resources will be
moved as close to where they are located. Basically, this is done contacting the Edge ISP
and therefore getting access to the nano servers in order to move users resources.
Connectivity
Provision
IaaS
Provisioning 1
IaaS
Provisioning n
Inter-
connectivity
Provision 1
Content
Provision
Content
Consumption
End-User
Cloud Provider
Transit ISP
Edge ISP
Inter-
connectivity
Provision n
Content
Cache

Figure 5-18: Application Provider using Cloud with NaDas to enable Global Service
Mobility.
5.5.3 Business Model Analysis
We apply the BMRF to the second case described in Section 5.5.2 and is depicted in
Figure 5-17, in order to build the business model for the collaboration between Application
Providers and a Cloud Provider. We focus on these two stakeholders, as they are relevant
for SmartenIT and while Co-location Providers in the first case described in Section 5.5.2
and Private Cloud Providers in third one share functionalities with Cloud Providers as well,
they do not provide a single unified access pattern for Application Providers. Cloud
Providers will enable the Global Service Mobility scenario. In particular, we select the
appropriate values for the key business model parameters described in the template, in
order to provide insight to the way business is conducted. We begin our investigation with
the first design theme, the Value Proposition, and we apply its basic parameters to this
market.
Product/Service Delivered: The capability to move resources close to where end-users are
located, without disrupting the service delivered, is the main product from Cloud
Providers to Application Providers. Cloud Providers should be aware where to move the
Public

resources and, therefore, should have bilateral agreements with Transit ISPs and
possibly with other Cloud Providers. The Cloud Provider that wants to provide global
service mobility should have agreements with Transit ISPs due to the amount of traffic
generated, and also due to traffic prioritization. Agreements with other Cloud Providers
should be made to enable a dynamic and on-demand utilization of other data centres
which are located closer to where the end-user moved.
Target Customer: The Global Mobility Service, which is offered by Cloud Providers, mainly
targets Application Providers. The Application Provider is mainly interested to contract
such service to give a better QoS and QoE to its end-users.
Customer Value: End-users' customer value, i.e., improved response time for their service
requests, is guaranteed by means of SLAs between the Application provider and the
Cloud Providers.
Resources and Competencies: All the stakeholders, i.e., Application Provider, Cloud
Providers, Transit ISPs, Co-location Providers, and Edge ISPs, use their resources and
competencies in their core business. Therefore, the business model is technology-
oriented.
Vertical Integration: Cloud Providers will integrate several levels of technology. Most
important, Cloud Providers will integrate the roles of Co-location Providers and data
transfers between Transit ISPs to enable the Global Service Mobility.
Customer Ownership and Relationship: The relationship between Application Providers
and Cloud Providers is direct. Cloud Providers offer a service (in this case, the Global
Service Mobility) and Application Providers contracts it considering some agreements.
Moreover, Cloud Providers should have direct agreements with other Cloud Providers
and Transit ISPs to establish a viable way to move resources close to end-users.
Interconnection Modality-Business Agreements: There are several agreements
established in the considered scenario; one of the most important is between
Application Providers and Cloud Providers. This contract is fundamental and should
present parameters related to QoE and QoS. Another very important agreement is
between Cloud Providers: in this case, Cloud Providers should agree on prices to have
resources on-demand, in specific data centres. Contracts between Cloud Providers and
Transit ISPs should consider the amount of traffic generated to other Cloud Providers
and, most important, traffic prioritization, since QoE and QoS are the key factors to
offer in Global Service Mobility.
Content-Data Delivery Model: The model closest to this scenario is the Cloud model.
Application Providers contract Global Service Mobility with Cloud Providers related to
their application which is already in the Cloud. Therefore, the Cloud Provider has the
responsibility to interface to the application to know where the end-users are located, if
end-users moved, and then to move resources close to them. All this work is done in a
transparent fashion and, on demand (just if necessary).
Revenue model, revenue sharing & money flows: Cloud Providers have several options to
monetize the Global Service Mobility. First, the Global Service Mobility can be done as
a subscription. The Application Provider pays a fixed amount to the Cloud Provider to
enable such service, world-wide. The Cloud Provider can calculate the price based on,
Public

for example, number of end-users using the application, or amount of data usually
generated out of the Cloud Provider. A refined calculation should be made combining
several factors of the application. Second, pay-per-use should also be an option. Cloud
Providers will just charge Application Providers depending on how many end-users
moved and, therefore, based on how many resources had to be re-allocated to different
locations. In both options, the Cloud Provider should perform a refined estimative of
what exactly to charge, also assessing the risk of not providing Global Service Mobility
like agreed beforehand therefore, the contracts between other Cloud Providers and
Transit ISPs should reflect such decision.
Traffic Charging Scheme: It is expected that Cloud Providers and Transit ISPs charge
based on bandwidth and QoS parameters.
Cost Model: The cost of employing Global Service Mobility can be mitigated by the
establishment of a federation among Cloud Providers and Transit ISPs through a long-
term agreement.
Global Service Mobility is an interesting scenario for SmartenIT. The project's interests in
respect to the traffic management partially overlap with the global service mobility VNCs
presented in this section. In more detail in the first case, when the Application Provider is
not using cloud in order to enable Global Service Mobility, the Application Provider should
manage the traffic generated by his users with own resources. Furthermore, the
Application Provider on its own should handle the end user location detection. Thus, an
SLA agreement with a Co-location provider should be established in order to provide the
necessary computational resources.
In the scope of SmartenIT, which addressed cloud-based applications/service, clearly the
two latter cases presented in Section 5.5.2 are of high interest. In particular, we identified
that when the Application Provider is using a Cloud Provider in order to move users
data/resources closer to the user, the former stakeholder may focus on the application
layer, and the data moving policies. Last but not least, Global Service Mobility could be
enabled with the use of NaDas. NaDas might be managed either by the Edge ISPs or by
the end-users enabling the Application Providers to form a private cloud. Furthermore, the
energy efficiency aspect of the NaDas is of high relevance to the SmartenIT project.
Nevertheless, the definition of the Global Service Mobility scenario and related use-cases
is an ongoing process and it is yet to be investigated within Task T1.4 and Task T2.2.
Thus, SmartenIT's deeper investigation into these use-cases will be elaborated within
D1.2 and D2.2 respectively, once the on-going discussion in the project will be finalized
and solid use-cases will be decided.
5.6 Social-Awareness for Content Delivery
In this section, we perform stakeholder relationship analysis for two scenarios where
social awareness is employed to achieve efficient content delivery. In both scenarios,
similar social information is sought and exploited; though, their main difference relies on
the architecture deployed for the content delivery itself, i.e., centralized vs. distributed,
provider-driven vs. peer-assisted content delivery.
Public

5.6.1 Scenarios
Below, we describe two scenarios where social information is employed to achieve
efficiency in content delivery, in terms of either content placement or pre-fetching. Both
scenarios are considered to be of high interest to SmartenIT; the first one is inspired by
the evaluation scenario described in [63], while the second one is on scenarios considered
in [149] and [150].
5.6.1.1 Exploitation of Social Information by a Centralized Content Delivery Platform
We consider an online social network (e.g., Facebook) having users around the globe who
upload user-generated content (e.g., home-made videos) to an online video streaming
platform (e.g., YouTube). This content can be viewed by their online friends, their friends'
friends, etc. through the Friend-of-Friend (FoF) relationship.
In order to cater the users of the video streaming platform, the video platform is operated
on a geo-diverse system comprising multiple points-of-presence (PoPs) distributed
globally. These PoPs are connected to each other by links, which can either be owned by
the entity that also owns the POPs, or be leased from network providers.
Each user is assigned and served out of their nearest (geographically) PoP, for all of his
requests. Placing data close to the users is an approach followed by most CDNs.
Therefore all content uploaded by a user A is first uploaded to the nearest respective
PoP
A
. When content is requested by another user B, the nearest PoP to B, i.e., PoP
B
, is
contacted and if the content is available there, the request is served. The content can be
already present at PoP
B
, if content was first uploaded there or was brought there by an
earlier request. If the content is not available in PoP
B
, then a request is made to PoP
A
and
the content is brought to the PoP
B
.

Figure 5-19: Content dissemination of UGC over an OSN.
The scenario considered in this section is illustrated in Figure 5-20, and as
aforementioned, is inspired by the evaluation scenario described in [63]. In such as setup,
social relationships between the users of the OSN can be taken into account to to predict
where a piece of content will be requested next, i.e., by which PoP. For instance, it is
expected that due to their social relationship, users of the online social network will
request for a piece of content that a friend of them, i.e., user A, has uploaded to the video
platform with higher probability than users that have no social relationship with A. The so-
Public

called social awareness involves the exploitation of such social information (e.g., social
graph, social relationships among users), and meta-information (e.g., behaviour patterns,
login time, time spent in the OSN) in order to predict where and by whom a video
uploaded by a user will be consumed. Such predictions can be employed, also in the
context of SmartenIT, to develop social-aware mechanisms such as TailGate proposed in
[63] that will enable pre-fetching of the uploaded video in various locations (e.g., POPs).
5.6.1.2 Exploitation of Social Information by a Distributed Content Delivery Platform
We consider again an OSN such as Facebook, whose users scattered around the globe
and upload videos, i.e., UGC to an online video streaming platform like YouTube. This
content similarly to the scenario described in Section 5.6.1.1 can be viewed by their
friends, their friends' friends, etc.
End-users, also called peers, download parts of the file, e.g., chunks, and are considered
to be able to re-upload them to another peer. Additionally, a proxy server is considered to
participate in the content dissemination, to which a video can be uploaded by a peer, and
as well as video originated from the content provider, e.g., a music video clip.
Consequently, a proxy server is connected to a content provider, e.g., a TV channel.
Moreover, multiple proxy servers are considered to be also distributed globally and each
one of them to have a specific domain of influence, e.g., an ISP's domain, an Autonomous
System (AS). The initial peer uploading a video to the proxy server (in the case of UGC),
the proxy server itself and the peers participating in the dissemination of that particular
video are considered a swarm. Furthermore, the video parts exchange among peers is
performed based on some specific peer and chunk selection policy.
As aforementioned, placing video chunks close to the end-users is an approach followed
by most CDNs as it leads to lower latency and stall time, and thus high QoE for end-users.
Therefore, social information is extracted from OSN by the video platform owner, so as to
predict by whom a video uploaded to the proxy server will be viewed. According to [149]
and [150], direct friends (1-hop friends) and friends of friends (2-hops friends) of a user C
have high probability, i.e., more than 80%, to watch a video uploaded or posted by C.
Additionally, social meta-information such as user's interests, e.g., sports, music, prove to
be also important, as users having a FoF relationship with C and also the same interests
are highly possible to view the video uploaded by C. Figure 5-20 illustrates the case,
where content is disseminated in a P2P fashion among OSN's users, taken from [149].

Figure 5-20: Structure of a socially aware P2P scheme for content delivery [149].
Public

According to the description of the social awareness scenario, we employ the
stakeholders taxonomy described in Section 0 to identify all involved stakeholders and to
describe their role. Table 5-5 includes the identified stakeholders, their role description
and a preliminary identification of their interests.
Table 5-5: Stakeholder identification for social-awareness for content delivery.
Edge ISP
Provides connectivity to end-users; may
own a proxy server assisting video
dissemination among his customers, i.e.,
End-Users
Transit ISP
PoPs/proxy servers of a service provider
through leased lines; sells transit to
Edge ISPs
connection of service providers or
selling transit to Edge ISPs
Information / service providers
Content Distribution
Network / Video
Streaming Platform
Owns and controls the video delivery
platform; may own PoPs/proxy servers
or rent them from a Cloud Provider; buys
inter-connectivity between PoPs/proxy
servers through leased lines by the
transit ISP; establishes SLAs with transit
ISPs; decides on the content replication
among PoPs/proxy servers; may charge
the End-User for access to content, i.e.,
performs Authentication, Authorization,
Accounting (AAA)
Provide high QoE to End-Users;
reduce inter-connection costs; reduce
management overhead and costs;
reduce energy consumption across his
own infrastructure
Application Provider /
Online Social Network
Provides the online social network
service; may own and control proxy
servers across the globe or rent them
from a Cloud Provider
reduce inter-connection costs; reduce
energy consumption across his own
infrastructure; increase revenues from
advertisements on the OSN webpage
Content Provider
Provides popular content to the CDN;
may charge the End-User for access to
content, i.e., performs AAA
increase revenues from content
delivery
Cloud Provider / Cloud
Operator / IaaS Provider
Provides IaaS (i.e., storage,
computation) to service providers, i.e.,
CDNs and OSNs; may own links
interconnecting his data centres or buy
inter-connection from Transit ISP
Provide high QoE to service providers;
meet SLA requirements; reduce / keep
low inter-connection costs; improve his
infrastructure utilization; avoid / limit
congestion on his links or data
centres; improve / limit energy
consumption by his servers
Users
End-User
Participates actively in the OSN, e.g.,
establish friendships, post and comment
videos uploaded by friends or friends of
friends , related to his interests;
generates and uploads content to the
CDN, i.e., downloads and consumes
Enjoy high QoE in terms of low latency
and stall time; additionally, high
service and content availability, access
bandwidth and download speed along
with low connectivity cost
Public

videos from the CDN
In the following sections, we construct certain VNCs which depict possible interactions
between major stakeholders of the scenarios under study. Nevertheless, there can be a
multitude of possible value network configurations, though in this section, we reside to
three highly relevant to SmartenITs scope. Note also that some of the roles which have
been identified in Table 5-5, e.g., AAA, are not considered in the following analysis
intentionally. The reason for this is the fact that the focus is here on roles directly related
to the extraction and use of the social information, thus the aforementioned roles
considered to be of secondary importance to our investigation below.
We begin to build the value network configuration for the exploitation of social awareness
to provide efficient content delivery through a centralized architecture where the OSN
Provider consents to offer social information to the CDN. In particular, we consider a CDN
Provider that controls the video delivery platform and offers both popular content by the
Content Provider, and UGC by End-Users. Moreover, the CDN makes decision on where
the content, i.e., videos, is being stored based on social information and meta-information.
Additionally, a Content Provider is considered who provides the popular, non-UGC content
to the CDN. Additionally, we consider an OSN Provider that owns and controls the OSN
platform and delivers OSN information and meta-information to the CDN. Both the CDN
and the OSN Providers are buying IaaS from a consolidated Cloud Operator.
Moreover, we assume an ISP providing access connectivity to End-Users, and a Transit
ISP who inter-connects the data centres of the Cloud Provider. Apart from his basic role
as connectivity provider, the (edge) ISP is also assumed to have installed and control a
local cache/proxy server, which is made available to the CDN provider to reduce distance
to the End-Users and improve content delivery performance. Finally, the End-User is
practically a service consumer; he consumes content, i.e., watches videos, and he is also
using the OSN platform connecting to friends, posting content, sending messages, etc.
The produced VNC is illustrated in Figure 5-21.
Connectivity
Provision
DC
Ownership
IaaS
Provision
Content
Cache
Ownership
Inter-
connectivity
Provision
Content
Provision
Content
Placement
OSN Service
Provision
OSN Client/
Activity
Content
Consumption
End-User
Video
Delivery
OSN provider
Cloud Operator/IaaS provider
Transit ISP
ISP
CDN/Video Platform Service Provider
OSN (Meta)-
Information
Mgmt
Content Provider

Figure 5-21: Basic VNC for Exploitation of Social Awareness for Content Delivery
Centralized; Collaborative.
Public

An alternative setup is one, where the OSN is considered not to consent to provide social
information to the CND to perform content placement. In such a setup, the CDN, or the
ISP can deploy an OSN crawler extracting as much as possible social information by the
social graph and End-Users interests, as expressed through their posts, public messages,
etc. If the CDN employs the crawler, the information extracted can be directly be used by
the Content Placement module to perform its role. Such a setup is depicted in Figure 5-22;
the differences to the previous VNC are mainly found in the upper right side of the VNC
figure.
Connectivity
Provision
DC
Ownership
IaaS
Provision
Content
Cache
Ownership
Inter-
connectivity
Provision
Content
Provision
Content
Placement
OSN Service
Provision
OSN Client/
Activity
Content
Consumption
End-User
Video
Delivery
OSN provider
Transit ISP
ISP
Content Provider
OSN (Meta)-
Information
Crawler

Centralized; Non-Collaborative; CDN-driven.
On the other hand, if the ISP employs the crawler, he can combine the social information
with underlay information derived by his network, e.g., network condition (such as level of
congestion), physical location of End-Users, latency, delay, etc. and offer an integrated
information service to the CDN. The respective VNC is illustrated in Figure 5-23.
Integrated
Information
Provision
Connectivity
Provision
DC
Ownership
IaaS
Provision
Content
Cache
Ownership
Inter-
connectivity
Provision
Content
Provision
Content
Placement
OSN Service
Provision
OSN Client/
Activity
Content
Consumption
End-User
Video
Delivery
OSN provider
Transit ISP
ISP
Content Provider
OSN (Meta)-
Information
Crawler
Underlay
Information
Monitoring

Centralized; Non-Collaborative; ISP-driven.
Public

So far, the three VNCs which have been described are referring to a centralized
architecture for content delivery, based on the scenario described in Section 5.6.1.1. Next,
we extend the setup depicted in Figure 5-22 to address the scenario discussed in Section
5.6.1.2. Therefore, in this setup, we consider a more distributed architecture employing
peer-assisted content delivery. Specifically, we assume that storage space in End-Users
premises is used and orchestrated by the CND to improve performance of content delivery
(see Figure 5-24). Nevertheless, the exploitation of End-Users capacity, both in terms of
storage and bandwidth, is not pre-assumed, as incentives must be provided to them to
coincide. Ultimately, the hybrid content delivery architecture can also be combined with
the local cache/proxy server in ISPs premises considered in all three previous VNCs.

Content Mini-
Cache
Ownership
DC
Ownership
IaaS
Provision
Connectivity
Provision
Inter-
connectivity
Provision
Content
Provision
Content
Placement
OSN Service
Provision
OSN Client/
Activity
Content
Consumption
End-User
Video
Delivery
OSN provider
Transit ISP
ISP
Content Provider
OSN (Meta)-
Information
Crawler

Figure 5-24: Extended VNC for Exploitation of Social Awareness for Content Delivery
Distributed; Non-Collaborative.
We apply also for this scenario the BMRF presented in Section 5.2.2, to build the
business model for the exploitation of social information for content delivery. In particular,
we select the appropriate values for the key business model parameters described in the
template so as to provide insight to the way business is conducted. We begin our
investigation with the first design theme, the Value Proposition, and we apply its basic
parameters to this market.
Product/Service Delivered: The main product delivered from the Content Provider or the
End-Users (depending of the content's nature) to (other) End-Users over the CDN, i.e.,
video delivery platform is video. Additionally, another important product in the scenarios
described in Section 5.6.1 is the OSN service through which the End-Users make
online friends, interact with them and their friends' friends, comment, post, upload and
download a video.
Target Customer: The CDN Provider addresses the Content Provider and the End-Users;
i.e., the CDN distributes through his network the Content Providers data, while he also
delivers/distributes content requested/uploaded by the End-Users. The Cloud Operator
addressed the CND and the OSN Providers offering them IaaS; the ISP and OSN
Provider address the End-Users providing them connectivity and OSN service,
Public

respectively; finally, the Transit ISP addresses the (edge) ISP and the Cloud Operator
inter-connecting their AS and data centres, respectively.
Customer Value: The customer value is guaranteed by means QoS requirements for
content delivery and OSN service (e.g., low latency). Additionally, the customer value
between the Cloud Operator and his customers, i.e., CDN and OSN Providers, is
guaranteed by means of SLAs.
Resources and Competencies: The Cloud Operator, CDN Provider, OSN Provider, and
ISPs utilize their resources, and competencies in their core business, thus the business
model is technology-oriented.
Vertical Integration: It is likely that large CDN Providers deploy their own data centres and
thus, integrate also the role of the Cloud Operator. Additionally, Cloud Operators
usually also own links inter-connecting their data centres, thus aggregating also the role
of Transit ISP.
Customer Ownership and Relationship: There can be an inter-mediated relationship
between the CDN Provider and the End-Users. In such a case, the mediating entity can
be a cache or proxy server located in ISP's premises.
Interconnection Modality-Business Agreements: There are various agreements
established in the considered scenario; the most important of which include the SLAs
among different Cloud Providers, and between the Cloud Providers and the CDN
Providers, or the Cloud Providers and the OSN Providers. If a Cloud Operator does not
meet his SLA performance requirements towards the CDN, OSN Provider or another
Cloud Operator, he is penalized. Additionally, an SLA is possible to be established
between the CDN Provider and the ISP, by which the content stored in ISP's
cache/proxy is controlled by the CDN provider. Moreover, an SLA can be established
between the Transit ISP and the Cloud Operator; in such a case, the Transit ISP is
committed to provide connectivity of certain QoS to inter-connect the data centres of
the Cloud Provider. Nevertheless, an SLA agreement is also necessary between the
CDN and the Content Provider, whose content is distributed through the CDN's
infrastructure.
Content-Data Delivery Model: The CDN model is the dominant one.
Revenue model, revenue sharing & money flows: Usually, the CDN Provider is paid by the
Content Provider to replicate the latter's content in multiple servers close to the End-
Users. The CDN Provider, then, can pay a Cloud Operator to buy storage capacity in
the latter's data centres, while he can also pay an ISP to employ the ISP's cache/proxy
server to diminish the distance to his End-Users. Similarly to the CDN case, the OSN
provider also pays a Cloud Operator to buy both storage and computation. Moreover, a
Content Provider can be paid by an End-User; specifically, the End-User may login
using his unique credentials in the Content Provider's portal and rent movies from it
paying with a credit card, e.g., the Netflix and Hulu cases. In a socially aware context,
the CDN may pay the OSN Provider, i.e., provide him monetary incentives, so as to
share with the former one, social information and meta-information about his End-
Users.

Traffic Charging Scheme: Not applicable in this scenario.
Public

Cost Model: Inter-connection costs may be imposed by the Transit ISP to the Cloud
Operator for connecting his data centres. Additional costs may also be incurred for the
CDN Provider due to the provision of local cache/proxy server by the ISP, and the
social information potentially made available by the OSN provider. In the peer-assisted
case, an extra cost may be inferred for the CDN as he may be obliged to compensate
the End-Users for providing their resources for the delivery of non-UG content.
To address the scenario for exploitation of social awareness for peer-assisted content
delivery, which is based on the scenario described in [149], we extend the business model
described to include also the P2P characteristics of the hybrid, centralized and distributed,
overlay for content delivery.
Vertical Integration: Vertical integration is also possible between the CDN provider and the
End-Users, in the sense that the End-Users may install and use an application client in
their premises which provides them the capability to interact with the CDNs servers
(rented by the Cloud Operator) and mainly to be orchestrated by the Content
Placement module of the CDN.
Customer Ownership and Relationship: In the peer-assisted case, the mediating entities
can also be mini-caches in other End-Users' premises, e.g., [149].
Interconnection Modality-Business Agreements: An agreement needs to be established
between the End-Users and the CDN; practically, the agreement will dictate how the
End-Users will be compensated for the resources they offer to assist content delivery;
the compensation may be monetary, e.g., lower subscription fee, or performance-
related, e.g., better QoS.
Content-Data Delivery Model: The (hybrid) peer-assisted CDN model is the dominant one.
Regarding the Financial Configuration, the following changes apply.
Revenue model, revenue sharing & money flows: Similar revenue model and sharing, and
money flows to the previous case. Nevertheless, the End-User must be compensated
for sharing his bandwidth, storage and computational power. Compensation could be
either monetary, e.g., reduction of price charged for connectivity services, or
performance-related, e.g., upgrade of the User's access speed/package, or service-
related, e.g., provision of an extra service such as free access to the IPTV/VoD
platform.
Traffic Charging Scheme: A volume-based or congestion-based scheme can be employed
by the ISP to charge the CDN Provider for the extra traffic in his network due to the
peer-assisted content delivery.
Cost Model: The transit costs will be mitigated both for the CDN Provider and the edge
ISP by the peer-assisted content delivery. Though, OPEX will increase for the ISP due
to the extra traffic in his network.
The exploitation of social awareness to employ efficient content placement and therefore
content delivery is an upcoming trend which has proven to achieve significant benefit for
some of involved stakeholders, e.g., the CDN, the ISP and the End-Users, whereas, on
the other hand, a Transit ISP may see his revenue from selling transit reduced. Moreover,
the OSN Provider should be given incentives to provide the required information,
otherwise the entity employing a socially aware traffic management mechanism should be
Public

in place to perform extensive crawling of the OSN. Nevertheless, the latter choice may not
reveal as much social information as a collaboration with the OSN would, but it is a viable
solution that still brings significant performance improvements according to work in
literature.
A major question related to the social awareness scenario is who will apply the
mechanism, e.g., the CDN or the ISP? In Section 5.6.3, both cases have been described;
however, different constraints must be taken into account and interactions take place in
each one of the setup, which need to be considered when designing a socially aware
traffic management mechanism.
Exploiting social awareness to perform traffic management is one of the main objectives of
SmartenIT, whereas collaborative approaches employing incentives to persuade the
various stakeholders to contend are also highly relevant to the project's scope. The
considered VNCs for the scenario of social awareness for content delivery may constitute
attractive solutions, while their analysis provides some insight on what expectations each
stakeholder will likely express, and also gives some directions on how to address/meet
these expectations.
A SmartenIT mechanism could be seen so far as a set of architecture/functional elements
embodied within the aforementioned stakeholders. Alternatively, such a module could be
employed by a new entity, e.g., an OTT Provider, and placed among the existing
stakeholders, so as to collect social information and provide it to the CDN or the ISP, or
the End-Users themselves in the peer-assisted setup. Thus, the SmartenIT point-of-
intervention can be either a module or "box" within the entity employing the mechanism,
which would be responsible for coordinating with other similar "boxes" within other entities,
and ultimately, communicating with the Content Provider. Alternatively, in the OTT
Provider case or the peer-assisted one, the SmartenIT point-of-intervention is the role
played by the OTT or a tracker orchestrating the P2P overlay. Of course, even in such a
case, specific "boxes"/modules are necessary within the CDN and ISP, which will
constitute the interfaces for the orchestration with the OTT/tracker. In all cases, alignment
of the stakeholders' incentives regarding the exploitation of social awareness for content
delivery as described in Section 5.6.1 is required for the adoption of such a solution by the
market.
As in the previous 4 scenarios, a tussle analysis is needed to take place in order to
investigate the adoption of potential traffic management mechanisms in the long term and
to identify possible conflicts between stakeholders. The application of tussle analysis
however presumes the (initial) specification of traffic management mechanisms; thus, it is
left as future work to be performed within WP2.

Public

6 Summary and Conclusions
The purpose of this deliverable was two-fold. The two main target were stated in Section
2.1, and are repeated below:
- Target 1 was to identify major characteristics of cloud-based overlay applications
and services such as traffic volume generated, traffic patterns, energy
consumption, QoE aspects, and end-user's devices implications. Additionally, the
potential for intervention and optimization w.r.t. traffic, energy and QoE need to be
addressed as the ultimate target of this investigation is to provide input to
Deliverable D1.2 and the selection process of the overlay applications to be
addressed by the SmartenIT project.
- Target 2 was to describe basic scenarios of cloud-service provisioning in which
The next sections summarize key outcomes and lessons learnt from our investigations
towards the two targets, and outline the next steps and open issues to be addressed in
the next phases of SmartenIT.
6.1 Key Outcomes & Lessons Learnt
Our study initially presents the definition of crucial terms related to SmartenIT, which
is provided in Section 3.1, that gives our understanding of: what cloud computing stands
for practically, the differences between cloud computing and grid computing, cloud and
CDN, or cloud and data centre. Deployment models, and economic and QoS aspects of
cloud computing are briefly addressed in Sections 3.2, 3.3, and 3.4, to develop a common
background on the impact of cloud computing on the various Internet stakeholders.
Regarding our Target 1, Section 3.5 describes a variety of cloud services offered by well-
known major cloud operators, as the so-called Internet giants, such as Amazon and
Google. Most cloud-based applications are built on Internet giant cloud services, e.g.,
Dropbox employs both Amazon's EC2 and S3 to provide its personal online storage
service. This description allows us to understand the architecture and operation of the
cloud-based overlay applications described in Chapter 4, to define realistic scenarios of
cloud service provision, and last to identify possible stakeholders corresponding to real
operators described in Chapter 5.
6.1.1 Traffic Characteristics
An overview of various sources of traffic generated by data centres and cloud is
provided in Section 3.6, both as a share of global IP traffic, and in terms of absolute traffic
volumes, popularity and future trends. The main conclusions and future projections are as
follows: Global data centre IP traffic will nearly quadruple, while it will grow by 31% CAGR
over the next 5 years. On the other hand, global cloud IP traffic will increase 6 times and
grow by 44% CAGR over the next 5 years, while global cloud IP traffic will account for
Public

nearly two-thirds of total data centre traffic by 2016. Moreover, the rapidly growing number
of mobile devices (e.g., smartphones and tablets) used to access new overlay applications
constitutes a key driver of the cloud traffic increase. Specifically, it has an important
influence on how application ecosystems are implemented nowadays, namely,
applications 'run' in the cloud (i.e., computation, data storage), while only a 'shell' providing
an interface to access the application remains at the mobile device.
Concerning the efficient management of cloud traffic, a main challenge lies in an
optimized distribution of data over traditional and cloud data centres with regard to short
transport paths from data centre to the end-user and between data centres themselves so
as to achieve energy efficiency. Additionally, energy efficiency is highly important for
mobile devices due to their limited power resources. Characteristics of energy
consumption by data centres, cloud, ISPs' networks and end-users' mobile devices have
been derived from a survey described in Section 3.7.
Our major conclusions for data centres are as follows: Although the IP traffic was
expected to grow by 46% per year in the last 5 years, the energy consumption for data
centres did not increase as predicted. In particular, it has been estimated to have
increased by up to 36% in the US, due to the use of virtualization that leads to more
efficient hardware utilization. Moreover, currently, only, 30% of total energy is delivered to
IT equipment for critical operations such as hard disk drive operations, networking,
memory, CPU, motherboards, fans, etc., implying PUE equal to 3.3. On the other hand,
the remaining 70% of total data centre energy consumption splits to 45% for cooling
infrastructure and 25% for power delivery units. Although virtualization is a key driver for
energy efficiency in the cloud, it does not provide for energy efficiency in the network
domain. SmartenIT will design mechanisms, which will manage cloud traffic by live VMs
migration from data centre to data centre, migration that will be strictly tied to data centre
resources orchestration and energy efficiency achievements.
bandwidth growth. Fibre optic links and equipment consume more energy than older
equipment for switching and routing, while caching infrastructure within ISPs, which is
increasingly adopted to shorten traffic paths in content delivery, leads to installation and
operation of small data centres and server resources in ISP's premises, which again
consume energy. Moreover, energy saving by switching off parts of the network equipment
in phases of lower load and reactivating them on demand is often not feasible currently
due to long set up periods of networking equipment.
On the other hand, considering energy implications on users' mobile devices of all
user/smartphone interactions have been recorded as being related to communication, a)
email, SMS, instant messaging and voice calls, nearly 50%, b) browsing, i.e., 12%, and c)
media consumption, i.e., 9%. Proposed approaches that aim to optimize the consumed
energy utilize the trade-off of transmission range and energy consumption. For instance,
terms of energy, whereas low range technologies like WiFi or Bluetooth have a relatively
low energy-footprint and can provide higher data rates.
The investigation in Section 3.6 allows us to foresee that applications such as video and
audio on demand/streaming are strong factors in cloud traffic growth, while emerging
services such as online storage e.g., Dropbox, are also gaining in popularity. This
investigation was continued in Chapter 4, and will provide input to Task T1.1 and
specifically Deliverable D1.2, where cloud service classification is to be completed.
Public

Additionally, the investigation of energy implications by cloud traffic in Section 3.7 showed
that although cloud improved energy consumption w.r.t. traffic growth; yet significant
inefficiencies still exist either in data centres and ISPs' networks, or in end users' mobile
devices. Such inefficiencies provide an opportunity for SmartenIT for intervention and
optimization.
6.1.2 Cloud-based Applications Characteristics
An extensive overview of a wide selection of cloud based overlay applications is
provided in Chapter 4. In particular, we have provided a description of such applications
from multiple viewpoints, namely in terms of their operation, traffic characteristics, energy
consumption, and end-user's device. The identified characteristics will play important role
and provide input to the classification of cloud services, as well as in the selection of
applications to be addressed by SmartenIT. Additionally, we highlighted characteristics of
new overlay applications that are relevant to SmartenIT, and discussed particularly the
potential for intervention and optimization by traffic mechanisms to be developed within
SmartenIT.
A comprehensive summary of the characteristics of the considered overlay application
categories is provided in Table 4-5 taking also into account the traffic characteristics and
energy consumption considerations for cloud services discussed in Section 3.6 and
Section 3.7. The main conclusions of this investigation are as follows:
- Video streaming, P2P video streaming and file sharing, online storage, cloud CDNs
and online gaming generate high or very high traffic volumes, while online social
networks, search engines and mobile instant messaging and VoIP generate
medium to low volumes,
- All categories of applications, except the mobile ones, cause a significant increase
of energy consumption in data centres, whereas mobile applications increase the
energy consumption in mobile devices, which is higher for video streaming,
- Considering QoE aspects, all of these applications require low latency, which may
not be always achieved, as their performance is highly dependent on network
conditions such as congestion.
Regardless of the type of application, the potential for intervention by a SmartenIT traffic
management mechanism is expected to be higher for non-proprietary solutions. The
investigation of stakeholders' incentives (as performed in Chapter 5) can help identify
intervention potential in specific setups, and enable collaboration between any entities
employing (either individually or collaboratively) a traffic management mechanism, e.g.,
ISPs, cloud IaaS operators, etc., and all other involved stakeholders. Moreover, potential
for optimization is considered to be directly dependent both on generated traffic volumes,
energy implications and QoE requirements. Thus, potential for optimization is considered
to be higher for video-related applications, e.g., video streaming and online gaming, where
there is higher potential for a more visible impact by SmartenIT. Potential for intervention
and optimization will be severely considered as selection criteria by SmartenIT, when the
final selection of applications, whose traffic is to be addressed by SmartenIT's
mechanisms designed in WP2, will take place in WP1 and specifically Deliverable D1.2.

Public

6.1.3 Stakeholders Characterization
Target 2 was to identify stakeholders and characterize their relationships in four basic
scenarios of cloud-service provisioning in which SmartenIT can add value. First, a survey
of selected methodologies and tools, i.e., tussle analysis framework, value network
configuration and business modeling framework, to perform the stakeholder relationship
analysis is provided in Chapter 5.
Tussle analysis was not fully applied in this deliverable; only its first step, i.e., the
stakeholders identification based on the stakeholders map generated by the SESERV
project was employed to identify involved stakeholders and their interests in the four
scenarios of interest. A complete tussle analysis can be the subject of theoretical
investigations in WP2 to qualitatively assess the impact of the designed mechanisms in
the scenarios of interest, potential conflicts and consequences in the long term. As a next
step in this deliverable, we used the business agreement framework to analyse
relationships and dependencies among the identified stakeholders, while as part of the
business analysis we employed value network configuration to illustrate these
relationships and inter-dependencies. Below, we summarize the main outcomes of the
stakeholder characterization performed for each of the four scenarios:
The inter data centre communication scenario depicts the need to efficiently and timely
carry traffic among data centres that are located in different parts of the network so as to
comply with certain market-driven requirements and deadlines. ISPs, Data Centre
Providers, or even Application Providers, either existing or emerging, may try to perform
bundling of the inter data centre communication service as provided to either other
(possibly smaller) Data Centre, or Application Providers and End-Users with existing
services, e.g., storage, computation, so as to dominate the market and strengthen their
position in the markets where they are active. Clearly the additional offering of such a
sophisticated service would be of high value to the data centre customers, whereas this in
turn could create mixed incentives for collaboration and competition with the other data
centres and also have adverse implications on the agreements with the ISPs. Moreover,
the control, market offer and orchestration of the end-to-end inter data centre
communication service could be performed either by the Data Centre Provider, which
would integrate this SmartenIT functionality in its own product offers, or alternatively by a
SmartenIT Application Provider, i.e., a new OTT provider.
Collaboration between cloud operators to achieve energy efficiency can be
performed either in a distributed or centralized manner. In the first case, a SmartenIT
mechanism could be seen as a set of architecture/functional elements embodied within
the aforementioned stakeholders, while in the second one, the SmartenIT mechanism
could be developed or placed as an extra entity among them, so as to coordinate and
enforce the collaboration for the efficient energy consumption behaviour among all
players. Such an entity would be responsible for orchestrating the collaboration between
cloud providers, and can be controlled by either a member of the federation, or by a
trusted third-party. However, in order for the Energy Broker to be controlled by one
member of the federation, then it would be necessary to achieve a win situation not only
for the entire federation, but also for each member of it (due to the attained energy
efficiency), at least in the long term; otherwise, negatively affected members of the
federation will either leave from it, or even decide not to join at all. Nevertheless,
depending on the considered setup, various inter-connection agreements need to be
established between the various Cloud Operators, Connectivity Providers, the Energy
Broker and the End-Users, whereas different money flows will be generated then.
Public

Global Service Mobility involves the capability to move resources between data centres
or clouds, close to where the end-users are located, without disrupting the service
delivered. In order to achieve Global Service Mobility, the user's location, i.e., AS, needs
to be detected; then, service-related data need to be moved to the new user location. For
the latter, two major approaches were identified, i.e., one time data move, and gradual
data move. The two approaches serve different purposes and applications, while they can
also be combined. The stakeholders characterization conducted in Section 5.5 revealed
that when the Application Provider is using a Cloud Provider in order to move users
data/resources closer to the user, then the former stakeholder may focus on the
application layer and on the data-moving policies, which are subsequently implemented by
the Cloud Provider based on the latter's optimization criteria and constraints, e.g., one-
time data move, or gradual data move. On the other hand, Global Service Mobility could
be achieved by a distributed approach involving also ISPs and end-users; e.g., the NaDa
solution could be extended to address service mobility apart from content mobility. The
stakeholders and business analysis for this scenario is to be further investigated within
WP2following the complete definition of concrete use-cases within Task T2.2.
Finally, the exploitation of social awareness to perform efficient content delivery can
be employed both in a centralized manner and in a more decentralized one, i.e., forming a
P2P overlay/swarm where also end-users participate by offering the storage and
computation resources. A major question related to the social awareness scenario is who
will apply the mechanism, e.g., the CDN or the ISP; in any case, though, the social
awareness scenario within SmartenIT is considered to be a provider-driven solution, i.e.,
initiated or deployed by an ISP, cloud operator, etc.. Moreover, a SmartenIT mechanism
could be seen either as a set of functional elements embodied within the involved
stakeholders, e.g., CDN provider, ISP, end-users' clients, or alternatively as a module that
could be employed by a new entity, e.g., an OTT Provider, placed among the existing
stakeholders, so as to collect social information and provide it to the CDN, or the ISP.
Alternatively, in the peer-assisted setup, the SmartenIT point-of-intervention could be:
either a functional module within the ISP, or a new OTT entity introduced, or the
enhancement of the overlay tracker orchestrating the P2P overlay to gather and utilize
social information.
Generally, collaborative approaches employing incentives to persuade the various
stakeholders to collaborate in each one of the four aforementioned scenarios are highly
relevant to the project's scope and will drive the design and specification of the traffic
management mechanisms within WP2. The analysis in this deliverable has revealed that
there is indeed considerable potential for collaboration in a variety of cases.
6.2 Next steps
This deliverable, D1.1, provides a categorization of cloud-based applications based on the
services offered to the end-user in order to address the issues related to the main
characteristics of the categories they belong to, e.g., traffic volumes generated, or QoS
requirements. Though, a complete classification of the cloud-based applications is
planned in D1.2, where also Task T1.1 "Cloud Services Classification" is to be concluded.
Additionally, in D1.1, describes the four major scenarios of high interest to SmartenIT and
employs stakeholders characterization so as to gain insight on their interests, relationships
under different setups and potential for collaboration among them. Nevertheless, D1.1
described four scenarios of interest to SmartenIT based on which the stakeholders
Public

characterization has been performed. However, the complete and final definition of the
SmartenIT's scenarios will be provided in the upcoming deliverable D1.2.
The outcome of D1.1 will also serve as input to several other activities within the project.
The traffic and energy consumption characteristics of the various cloud-based applications
and inter data centre communications will be used by WP1 to identify the potential for
optimization in terms of traffic management and/or energy efficiency, as well as to decide
whether SmartenIT should rather address a few, high impact applications (and which
ones) with tailor-made solutions, or a larger set of applications by designing more general
solutions with broader applicability. Moreover, the cloud applications categories described
in Chapter 4 will be used by T1.1 and D1.2 to employ cloud services classification, while
the four scenarios described and analysed in Chapter 5 will be extended by T1.4 and
D1.2, as well as the stakeholders characterization is to be finalized within D1.2.
Furthermore, the identification of stakeholders and their interests, as well as the analysis
and characterization of their relationship in four interesting cloud service scenarios will
allow SmartenIT, and particularly WP2, to design traffic management mechanisms that will
successfully address these stakeholders' incentives, promoting collaboration among them
to the extent possible, towards a mutually beneficial situation. In particular, the
identification of involved stakeholders' and their interests are already used by T2.1 in
Deliverable D2.1 (which has the same delivery date as D1.1) to define requirements of the
various stakeholders in cloud service scenarios in order to qualitatively evaluate traffic
management approaches in the literature, based on whether they address these
requirements or not. Moreover, stakeholders' relationships characterization and potential
for collaboration will serve as input to T2.3 and D2.3 to design economic-aware traffic
management mechanisms addressing the incentives of the various stakeholders, as
identified in D1.1. Ultimately, the business analysis performed in Chapter 5 will be
extended within T2.4 in order to investigate theoretical issues related to pricing schemes
and incentive mechanisms.
Public

7 SMART objectives addressed
Through this document, two overall (O1 and O2, see Table 7-1) and four specific (O1.4,
O2.3, O3.3 and O3.4, see Table 7-2) SmartenIT SMART objectives defined in Section
B1.1.2.4 of the SmartenIT DoW [204] have been partially addressed.
The overall Objective 1 is defined in DoW as following:
Objective 1 SmartenIT will lay the basis for its traffic management mechanisms for traffic
characterization by the definition of key stakeholders, which require
mechanisms with incentive compatibility, QoE.
The specific research area covered by this deliverable is: the extensive investigation of
traffic, energy and QoE implications of cloud services and applications (Chapter 3 and
Chapter 4). Moreover, we have addressed the identification of key stakeholders and
analysed their interests and relationships in four interesting scenarios of cloud service
provisioning (Chapter 5). Note that the scenarios are to be finalized in the next deliverable
of WP1, i.e., D1.2 (end of M12).
The work performed in Chapters 3 and 4 will provide input to the next deliverable of WP1,
i.e., Deliverable D1.2 (end of M12). On the other hand, the analysis of Chapter 5 will be
the basis for further analysis of stakeholders relationships, business analysis and
identification of possible tussles within WP2 and specifically Task T2.4 "Theoretical
Investigations", while it will also provide input to Deliverables D2.2 (end of M12) and D2.4
(end of M24).
Table 7-1: Overall SMART objectives addressed.
Ob-
jective
No.
Specific
Measurable
Achievable Relevant
Timely
Deliverable Number Mile Stone Number
O1
Traffic characterization D1.1 Study, design Basic MS1.1, MS1.2
Stakeholders D1.1 Study, design Basic MS1.1, MS1.2
QoE and social
awareness
D1.1 Study, design Advanced MS1.1, MS1.2
O2
Theory design for traffic
management
mechanisms
D1.1, D2.4, D4.3
Design,
simulation
Advanced
MS1.1, MS1.2, MS2.4,
MS4.3, MS4.5
Table 7-2: Specific SMART objectives addressed.

ID
Specific
Measurable
Achievable Relevant
Timely
Metric
Project
Month
O1.4
Are decentralized traffic
management schemes
superior to traditional
schemes; if yes, why? If not,
what is the associated
efficiency loss?
Number of identified and
tested scenarios for the
comparison of the different
traffic management schemes
Design,
simulation
T1.2, T2.1,
T2.5
Highly relevant
output of
relevance for
providers
M12
(design),
M24
(simulatio
n)
O2.3
Which of the designed
mechanisms should be
selected for implementation
Number of metrics identified
for the selection process,
number of possible
Design,
simulation
Extremely
relevant output
of relevance for
M12
Public

and field tests within the
project?
mechanisms to choose from T1.2, T2.3 providers and
users
O3.3
Which feedback loop and
what kind of communication
protocol should be used to
retrieve and evaluate users
perceived QoE?
Number of metrics identified
for the selection process,
number of possible
mechanisms to choose from
Design,
simulation,
prototyping
T1.3
Extremely
relevant output
of relevance for
providers and
users
M12
O3.4
How to monitor energy
efficiency and take
appropriate coordinated
actions?
Number of options identified
to monitor energy
consumption on networking
elements and end users
mobile devices, investigation
on which options perform
best (yes/no)
Design,
simulation,
prototyping
T1.3, T2.3,
T4.1, T4.2,
T4.4
Highly relevant
output of
relevance for
users
M36
This deliverable contributes to answering the following four specific questions:
1. Objective O1.4: Are decentralized traffic management schemes superior to
traditional schemes; if yes, why? If not, what is the associated efficiency loss??
Stakeholders relationship analysis has been applied in cases where either
centralized or distributed approaches, e.g., P2P approaches as NaDa, were
employed (Chapter 5). The outcome of this analysis is that indeed it may be harder
to address stakeholders' incentives in distributed schemes; however, if their
incentives are successfully addressed then these schemes are much more stable.
Moreover, sometimes distributed schemes even bring higher improvements
compared to centralized ones, though at an extra cost, i.e., the overhead to
orchestrate the multiple roles/stakeholders. Nevertheless, the identification of end
users incentives so as to convince them to provide their local capacity (storage,
computation, bandwidth) to others is yet to be performed; it will be the subject of
further studies within WP2 and specifically, Task T2.4; therefore, we can claim that
O1.4 has been partly addressed by D1.1.
2. Objective O2.3: Which of the designed mechanisms should be selected for
implementation and field tests within the project?
As reported in Chapter 4 and concluded also in the lessons learnt section (Section
6.1.2), video and audio streaming, e.g., Netflix, constitutes the hot application in the
Internet currently, as it is generating the largest share of total IP traffic. Another
significant category of applications is also online storage, e.g. Dropbox, which
although depicts lower share of total IP traffic, is forecast to become rapidly much
more popular within the next 3-5 years. Final decision on applications to be
addressed by SmartenIT's mechanisms is to be made in Deliverable D1.2 (M12).
Nevertheless, final decisions on mechanisms to be selected for implementation must
handle traffic generated by the selected applications and successfully address
stakeholders' incentives which have been identified in Chapter 5 (see discussion for
Objective O1.4 above). Thus, Objective O2.3 has been partly addressed by D1.1.
3. Objective O3.3: Which feedback loop and what kind of communication protocol
should be used to retrieve and evaluate users perceived QoE?
In Chapter 4, a multitude of cloud-based applications have been described and
related QoS/QoE metrics for each one of them have been also visited, e.g., video
streaming stalling times, online storage latency or slow download bandwidth, etc.
Public

Depending on the decision on which applications to address by D1.2, respective
QoE metrics will be considered. The identification of the feedback loop and metrics
to be monitored will be concluded within Task T1.3 (end of M12) and will be reported
in D1.2; thus, Objective O3.3 has been partly addressed in D1.1.
4. Objective O3.4: How to monitor energy efficiency and take appropriate
coordinated actions?
In Chapter 3, we have investigated energy efficiency aspects and overviewed studies
performing measurements of energy consumption in data centres, ISPs' networks
and end-users' mobile devices, while we also overviewed methodologies for
measurement of energy efficiency in mobile devices. Clearly, different energy
efficiency metrics need to be monitored for each one of them. This activity will be
concluded within Task T1.3 (end of M12) and will be reported in D1.2; thus,
Objective O3.4 has been partly addressed in D1.1.
According to the SMART objectives set within SmartenIT's DoW [204], those ones of
relevance for Deliverable D1.1 and the respective tasks within WP1, i.e., T1.2, T1.3 and
T1.4, the targeted for objectives have been met.

Public

8 References
[1] The NIST Definition of Cloud Computing, National Institute of Standards and
Technology, US Department of Commerce, Special Publication 800-145, P. Mell,
T. Grance, September 2011
[2] Akamai: http://www.akamai.com/
[3] Limelight: http://www.limelight.com/
[4] Peer-to-Peer networks: http://en.wikipedia.org/wiki/Peer-to-peer
[5] Bram Cohen, Incentives Build Robustness in BitTorrent, In Proceedings of the 1st
Workshop on Economics of Peer-to-Peer Systems (2003)
[6] KaZaA: http://www.kazaa.com/
[7] Skype: http://www.skype.com/
[8] Virtual private networks: http://en.wikipedia.org/wiki/Virtual_private_network
[9] Ian Foster, Yong Zhao, Ioan Raicu, Shiyong Lu, Cloud Computing and Grid
Computing 360-Degree Compared, Grid Computing Environments Workshop,
2008. GCE'08. IEEE, 2008.
[10] Frost & Sullivan, Comparing CDN Performance: Amazon CloudFronts Last Mile
Testing Results, available online at:
http://media.amazonwebservices.com/FS_WP_AWS_CDN_CloudFront.pdf
[11] The OpenStack project. URL: http://www.openstack.org/
[12] The OpenNebula project. URL: http://opennebula.org/
[13] The BonFIRE project funded by the EC FP7. URL: http://www.bonfire-project.eu/
[14] The GEANT BoD (Bandwidth on Demand). URL: http://bod.geant.net/
[15] http://www.markwilson.co.uk/blog/2011/06/microsofts-windows-azure-data
centres-some-statistics.htm
[16] http://www.data centreknowledge.com/archives/2011/04/25/microsoft-reveals-its-
specialty-servers-racks/
[17] Bundesnetzagentur, Annual reports on regulatory activities (in German), 2012:
www.bundesnetzagentur.de
[18] Australian Bureau of Statistics, pages on Internet activity (2012)
http://www.abs.gov.au/ausstats/abs@.nsf/Lookup/8153.0Chapter7Dec%202011
[19] Office of the Telecommunications Authority (OFTA) of the Hong Kong Special
Administrative Region, Statistics of Internet Traffic Volume (2012)
<tel_archives.ofca.gov.hk/en/tele-lic/operator-licensees/opr-isp/s2.html>
[20] Cisco Systems, Cisco Global Cloud Index: Forecast and Methodology 2011 -
2016, White paper series, 2012
[21] Sandvine, Rich Communication Services and Revenue Replacement - Spring
2012 Global Internet Phenomena Report, Tech. Rep., 2012
[22] G. Halinger and F. Hartleb: Content Delivery and Caching from a Network
Providers Perspective, Special Issue on Internet based Content Delivery,
Computer Networks 55 (2011) 3991-4006
Public

[23] Cisco Systems, Visual networking index, forecast and methodology, White paper
series (2012)
[24] Alexa Ranking: http://www.alexa.com/topsites
[25] Tinypic: http://tinypic.com/
[26] Pixlr: http://pixlr.com/
[27] Dropbox: http://www.dropbox.com/
[28] "How Dropbox sacrifices user privacy for cost savngs",
http://paranoia.dubfire.net/2011/04/how-dropbox-sacrifices-user-privacy-for.html
[29] I. Drago, M. Mellia, M. M. Munafo, A. Sperotto, R. Sadre, A. Pras, Inside Dropbox:
Understanding Personal Cloud Storage Services, IMC12, November 14-16, 2012,
Boston, Massachusetts, USA
[30] PiCsMu Website: Platform-Independent Cross Storage for Multi-Usage.
Available at: http://www.pics.mu. Last visited on: January, 2013
[31] Roberto Gonzalez, Ruben Cuevas, Reza Rejaie, ngel Cuevas: Google+ or
Google-?: Examining the Popularity of the new OSN, CoRR, Volume
abs/1205.5662, 2012
[32] Shah Mahmood, Yvo Desmedt: Preliminary Analysis of Google+s Privacy, Poster,
Proceedings of the 18th ACM conference on Computer and communications
security, October 17 - 21, 2011, Chicago, IL, USA
[33] "Google+ crosses 500 million total users with over 135 million active users",
http://google-plus.com/8430/google-crosses-500-million-total-users-with-over-135-
million-active-users/, Google+ official statistics, available in Dec. 2012
[34] "The Process of Invention: OnLive Video Game Service",
http://tv.seas.columbia.edu/videos/545/60/79
[35] Kuan-Ta Chen, Yu-Chun Chang, Po-Han Tseng, Chun-Ying Huang, and Chin-
Laung Lei, Measuring The Latency of Cloud Gaming Systems, MM11, November
28December 1, 2011, Scottsdale, Arizona, USA
[36] Domagoj Barievi, Divyashree Hassan RavindraKumar and Manasa
Chandrashekar, GameOn: Analysis and Implementation of Cloud Gaming,
available online at: http://www.cs.ucsb.edu/~manasa/cs276.pdf
[37] Mark Claypool, David Finkel, Alexander Grant and Michael Solano, Thin to Win?
Network Performance Analysis of the OnLive Thin Client Game System, In
Proceedings of the 11th ACM Network and System Support for Games
(NetGames), Venice, Italy, November 2012
[38] The Gaikai case study: http://www.nvidia.com/content/PDF/NVIDIA-GeForce-Grid-
Gaikai-Case-Study-HR.pdf
[39] Digital Foundry: http://www.digitalfoundry.org/
[40] Gaikai vs. Onlive: http://www.eurogamer.net/articles/digitalfoundry-face-off-gaikai-
vs-onlive
[41] CNN: http://www.cnn.com/
[42] ESPN: http://espn.go.com/
[43] Akamai: http://www.akamai.com/
Public

[44] Limelight: http://www.limelight.com/
[45] Level3: http://www.level3.com/
[46] ChinaCache: http://en.chinacache.com/
[47] Mukaddim Pathan, James Broberg, and Rajkumar Buyya, Maximizing Utility for
Content Delivery Clouds, Proceedings of the 10th International Conference on
Web Information Systems Engineering (WISE '09)
[48] James Broberg , Rajkumar Buyya , Zahir Tari, MetaCDN: Harnessing Storage
Clouds for high performance content delivery, Journal of Network and Computer
Applications archive, Volume 32 Issue 5, September, 2009, pp. 1012-1022
[49] Whatsapp: http://www.whatsapp.com/
[50] Viber: http://viber.com/
[51] HeyTell: http://heytell.com/
[52] Skype: http://www.skype.com/
[53] Sandvine, Whats Up with WhatsApp? - Fall 2011 Global Internet Phenomena
Report, Tech. Rep., 2011.
[54] C. Tsiaras, B. Stiller, Challenging the Monopoly of Mobile Termination Charges
with an Auction-based Charging and User-centric System (AbaCUS) NetSys
2013, Stuttgart, Germany
[55] Tobias Hofeld, Raimund Schatz, Ernst Biersack, Louis Plissonneau, Internet
Video Delivery in YouTube: From Traffic Measurements to Quality of
Experience in Data Traffic Monitoring and Analysis: From measurement,
classification and anomaly detection to Quality of experience. Editor(s):Ernst
Biersack, Christian Callegari, Maja Matijasevic, Springers Computer
Communications and Networks series , 2012
[56] Ashwin Rao, Arnaud Legout, Yeon-sup Lim, Don Towsley, Chadi Barakat, Walid
Dabbous. Network Characteristics of Video Streaming Traffic. Proc. CoNEXT
2011.
[57] Alessandro Finamore, Marco Mellia, Maurizio Munafo, Ruben Torres, and Sanjay
Rao. YouTube Everywhere: Impact of Device and Infrastructure Synergies on
User Experience. In IMC, 2011
[58] Leonidas Kontothanassis. Content Delivery Considerations for Different Types of
Internet Video. Keynote at ACM Multimedia Systems (MMSys) 2012
[59] Plissonneau, Louis; Biersack, Ernst W; Juluri, Parikshit. Analyzing the Impact of
YouTube Delivery Policies on User Experience. Proc. International Teletraffic
Congress (ITC 24), September 4-7, 2012, Krakow, Poland
[60] Shane Alcock, Richard Nelson. Application Flow Control in YouTube Video
Streams. ACM SIGCOMM Computer Communications Review 41(2). 2011
[61] Vijay Kumar Adhikari, Sourabh Jain, Yingying Chen, Zhi-Li Zhang. Vivisecting
YouTube: An Active Measurement Study. Proc. Infocom 2012
[62] Ruben Torres, Alessandro Finamore, Jin Ryong Kim, Marco Mellia, Maurizio M.
Munafo, Sanjay Rao. Dissecting Video Server Selection Strategies in the
YouTube CDN. Proc. 31st International Conference on Distributed Computing
Systems (ICDCS). 2011
Public

[63] S. Traverso, K. Huguenin, I. Trestian, V. Erramilli, N. Laoutaris, K. Papagiannaki,
TailGate: Handling Long-Tail Content with a Little Help from Friends, WWW'12,
April 16-20, 2012, Lyon, France
[64] Monia Ghobadi, Yuchung Cheng, Ankur Jain, Matt Mathis. Trickle: Rate Limiting
YouTube Video Streaming. Proc. USENIX Annual Technical Conference, 2012
[65] Tobias Hofeld, Raimund Schatz, Michael Seufert, Matthias Hirth, Thomas
Zinner, Phuoc Tran-Gia. Quantification of YouTube QoE via Crowdsourcing. IEEE
International Workshop on Multimedia Quality of Experience - Modeling,
Evaluation, and Directions (MQoE 2011), Dana Point, CA, USA, December 2011.
[66] Tobias Hofeld, Sebastian Egger, Raimund Schatz, Markus Fiedler, Kathrin
Masuch, Charlott Lorentzen. Initial Delay vs. Interruptions: Between the Devil and
the Deep Blue Sea. QoMEX 2012, Yarra Valley, Australia, July 2012
[67] Tobias Hofeld, Raimund Schatz, Martin Varela, Christian Timmerer. Challenges
of QoE Management for Cloud Applications. IEEE Communications Magazine,
April issue, 2012
[68] Sandvine, Fall 2012 Global Internet Phenomena Report, Tech. Rep., 2012
[69] PPStream: http://www.pps.tv/
[70] PPLive: http://www.pptv.com/
[71] Susu Xie, Bo Li, Gabriel Y. Keung, and Xinyan Zhang. Coolstreaming: Design,
Theory, and Practice. IEEE Transactions on Multimedia, 9, 2007
[72] SopCast: http://www.sopcast.com/
[73] Salvatore Spoto, Rossano Gaeta, Marco Grangetto, and Matteo Sereno. Analysis
of PPLive through active and passive measurements. In IEEE International
Symposium on Parallel & Distributed Processing, 2009
[74] Susu Xie, Gabriel Y. Keung, and Bo Li. A Measurement of a Large-scale Peer-to-
Peer Live Video Streaming System. In Packet Video 2007, 2007.
[75] Salvatore Spoto, Rossano Gaeta, Marco Grangetto, and Matteo Sereno. Analysis
of PPLive through active and passive measurements. In IEEE International
Symposium on Parallel & Distributed Processing, 2009.
[76] Long Vu, Indranil Gupta, Jin Liang, and Klara Nahrstedt. Mapping the PPLive
Network: Studying the Impacts of Media Streaming on P2P Overlays. Technical
Report UIUCDCS-R-2006-2758, University of Illinois at Urbana-Champaign IL
61801 USA, 2006
[77] Shahzad Ali, Anket Mathur, and Hui Zhang. Measurement of Commercial Peer-to-
Peer Live Video Streaming. In Workshop in Recent Advances in Peer-to-Peer
Streaming, 2006
[78] Hei, X., Liang, C., Liang, J., Liu, Y., & Ross, K. W. A Measurement Study of a
Large-Scale P2P IPTV System. In IEEE Transactions on Multimedia, Vol. 9, Issue
8, p. 16721687, 2007
[79] Liu, Y., Li, F., Guo, L., & Chen, S. A measurement study of resource utilization in
internet mobile streaming. Proceedings of the International Workshop on Network
and Operating Systems Support for Digital Audio and Video (NOSSDAV), 2011
[80] http://paranoia.dubfire.net/2011/04/how-dropbox-sacrifices-user-privacy-for.html
Public

[81] A. Finamore, M. Mellia, M. Meo, M. M. Munafo, and D. Rossi. Experiences of
Internet Traffic Monitoring with Tstat. IEEE Network, 25(3):814, 2011
[82] M.P. Wittie et al., Exploiting Locality of Interest in Online Social Networks, Proc.
ACM CoNEXT, Philadelphia, USA (2010)
[83] Y. Zhang , A. Arvidsson, Understanding the Characteristics of Cellular Data
Traffic, Proc. ACM CellNet, Helsinki, Finland (2012) 14-19
[84] G. Halinger, A. Schwahn and F. Hartleb, 2-state (semi-)Markov processes
beyond Gilbert-Elliot: Traffic and channel models based on 2. order statistics, to
appear in Proc. IEEE Infocom, Turin, Italy (2013)
[85] C.W. Dunn, M. Gupta, A. Gerber and O. Spatscheck, Navigation Characteristics
of Online Social Networks and Search Engines, Proc. WOSN12, Helsinki,
Finnland (2012).
[86] SymTorrent: http://amorg.aut.bme.hu/projects/symtorrent
[87] Torrent: http://www.utorrent.com/
[88] Kelenyi, I., Nurminen, J.K., CloudTorrent Energy-Efficient BitTorrent Content
Sharing for Mobile Devices via Cloud Services, short paper, 7th IEEE Consumer
Communications & Networking Conference CCNC2010, Las Vegas, Nevada,
January 2010
[89] Transmission: http://www.transmissionbt.com/
[90] MobTorrent: http://amorg.aut.bme.hu/projects/mobtorrent
[91] tTorrent Lite:
https://play.google.com/store/apps/details?id=hu.tagsoft.ttorrent.lite&hl=en
[92] aTorrent:
https://play.google.com/store/apps/details?id=com.mobilityflow.torrent&hl=en
[93] WmTorrent: http://www.wmtorrent.com/home_index.html
[94] BitTorrent for Android: http://www.bittorrent.com/bittorrent-android
[95] uTorrent for Android: http://www.utorrent.com/utorrent-android
[96] BitTorrent Remote: https://remote.bittorrent.com/
[97] Torrent: https://remote.utorrent.com/
[98] Shye, A., Scholbrock, B., and Memik, G.: Into the Wild: Studying Real User
Activity Patters to Guide Power Optimizations for Mobile Architectures. IEEE/ACM
International Symposium on Microarchitecture (MICRO), 2009.
[99] Zhang, L., Tiwana, B., Qian, Z., Wang, Z., Dick, R. P., Mao, Z. M., and Yang, L.:
Accurate Online Power Estimation and Automatic Battery Behavior Based Power
Model Generation for Smartphones. IEEE/ACM/IFIP International Conference on
Hardware/Software Codesign and System Synthesis (CODES+ISSS), 2010.
[100] Pathak, A., Hu, Y. C., and Zhang, M.: Where is the Energy Spent Inside my App?
Fine Grained Energy Accounting on Smartphones with Eprof. ACM European
Conference on Computer Systems (EuroSys), 2012.
[101] Pathak, A., Hu, Y. C., Zhang, M., Bahl, P., and Wang, Y.-M.: Fine-Grained Power
Modeling for Smartphones Using System Call Tracing. Conference on Computer
Systems (EuroSys), 2011.
Public

[102] Wichtlhuber, M., Rckert, J., Stingl, D., Schulz, and M., Hausheer, D.: Energy-
Efficient Mobile P2P Video Streaming. International Conference on Peer-to-Peer
Computing (IEEE P2P), 2012.
[103] Balasubramanian, N., Balasubramanian, A., and Venkatarami, A.: Energy
Consumption in Mobile Phones: A Measurement Study and Implications for
Network Applications. ACM Internet measurement Conference (ICM)
[104] Lee, K., Lee, J., Yi, Y., Rhee, I., and Chong, S.: Mobile Data Offloading. ACM
Conference on Emerging Networking Experiments and Technologies (CoNEXT),
2010.
[105] D.D. Clark, J. Wroclawski, K.R. Sollins, R. Braden, Tussle in Cyberspace: Defining
Tomorrows Internet. IEEE/ ACM Trans. Networking 13, 3, pp. 462-475, June
2005.
[106] C. Kalogiros, C. Courcoubetis, G. D. Stamoulis, M. Boniface, E. T. Meyer, M.
Wald- burger, D. Field, and B. Stiller, The future internet, ch. An approach to
investi- gating socio-economic tussles arising from building the Future Internet, pp.
145159, Berlin, Heidelberg: Springer-Verlag, 2011
[107] EU FP7 ICT SESERV Project: http://www.seserv.org/
[108] The ETICS Project, Deliverable D3.5: Final Business Models Analysis, January
2013
[109] Ghezzi, A.: A proposal of Business Model Design parameters for Future Internet
Carriers. NETWORKING 2012 WORKSHOPS. v. 7291, pp. 72-79. 2012.
Available: <http://www.springerlink.com/content/17u5274303204917/fulltext.pdf>
[110] EU FP7 ICT ETICS Project: https://www.ict-etics.eu/
[111] http://en.wikipedia.org/wiki/Value_network
[112] T. Casey, T. Smura, A. Sorri, Value Network Configurations in wireless local area
access. 9th Conference of Telecommunication, Media and Internet, 2010.
[113] A. Kostopoulos, I. Papafili, C. Kalogiros, T. Leva, N. Zhang, D. Trossen, A Tussle
Analysis for Information-Centric Networking Architectures, in 4th FIABook Future
Internet From Technological Promises to Reality, Editor(s): F. Alvarez, et al.,
ISO Press Books, May 2012
[114] Ballon, P.: Business modeling revisited: the configuration of control and value
Info, Vol. 9 No. 5, pp. 619, 2007.
[115] The SESERV project, D2.2 Economic Future Internet Coordination Activities,
August 2012
[116] Nikolaos Laoutaris, Michael Sirivianos, Xiaoyuan Yang, and Pablo Rodriguez:
Inter data centre Bulk Data Transfers with NetStitcher. ACM Conference on
Applications, Technologies, Architectures and Protocols for Computer
Communication (SIGCOMM) 2011
[117] All4Green - EU funded project. URL http://www.all4green-project.eu/
[118] Microsoft's Cloud Services:
http://www.microsoft.com/oem/en/products/other/Pages/cloud_services.aspx#fbid
=ZR4sCkErH9Q (last visited 22.03.2013)
[119] Windows Azure: http://www.windowsazure.com/en-us/home/features/what-is-
windows-azure/
Public

[120] Getting Started with the Windows Azure: http://msdn.microsoft.com/en-
us/library/windowsazure/ff919705.aspx
[121] Introducing Google Drive... yes, really, official Google Blog,
http://googleblog.blogspot.in/2012/04/introducing-google-drive-yes-really.html
[122] Google: Buy Storage, https://www.google.com/settings/storage/?hl=en_US
[123] Get started with Google Drive: System requirements,
http://support.google.com/drive/bin/answer.py?hl=en&answer=2374990
[124] "Google Docs, Sheets, and Slides size limits,
http://support.google.com/drive/bin/answer.py?hl=en&hlrm=en&answer=37603
[125] About the Google Drive viewer,
http://support.google.com/drive/bin/answer.py?hl=en&answer=2423485&from=173
8646&rd=1
[126] "Powering Google Search", http://googleblog.blogspot.com/2009/01/powering-
google-search.html
[127] Cloe Albanesius, How Much Electricity Does Google Consume Each Year, 2011,
http://www.pcmag.com/article2/0,2817,2392654,00.asp
[128] "Search Engines Online Trends", http://www.experian.com/hitwise/online-trends-
search-engine.html (last visited 25 March 2013)
[129] Yahoo! Search: http://search.yahoo.com/
[130] Yahoo! Directory: http://en.wikipedia.org/wiki/Yahoo%21_Directory
[131] Yahoo! Mail: http://en.wikipedia.org/wiki/Yahoo%21_Mail
[132] Yahoo! News: http://en.wikipedia.org/wiki/Yahoo%21_News
[133] Mike Andrews, Searching the Internet, IEEE Software, March/April 2012.
[134] Which Crawlers Does Bing Use? http://www.bing.com/webmaster/help/which-
crawlers-does-bing-use-8c184ec0
[135] "Microsoft 's Bing search queries overtake Yahoo! for the first time",
http://techcrunch.com/2012/01/11/microsoft-bing-search-queries-overtake-yahoo-
for-the-first-time-in-december/
[136] The size of the World Wide Web: http://www.worldwidewebsize.com/ (last visit
19
th
March 2013).
[137] Google Green: http://www.google.com/green/bigpicture/
[138] "Facebook Tops Billion-User Mark". The Wall Street Journal. October 4, 2012.
[139] George Koutitas, Prof. L. Tassiulas, Prof. I. Vlahavas Energy Efficiency
Monitoring in Data Centres: Case Study at International Hellenic University, Gent,
February-2012.
[140] Grren Grid: http://www.thegreengrid.org/
[141] Ipoque Internet Studies: www.ipoque.com
[142] G. Halinger, A. Schwahn and F. Hartleb, 2-state (semi-)Markov processes
beyond Gilbert-Elliot: Traffic and channel models based on 2. order statistics, to
appear in Proc. IEEE Infocom, Turin, Italy (2013)
Public

[143] G. Halinger and O. Hohlfeld, The Gilbert-Elliott model for packet loss in real time
services on the Internet, 14th MMB Conference, Dortmund,Germany (2008)
[144] W. Lautenschlaeger and F. Feller, Light-weight traffic parameter estimation for on-
line bandwidth provisioning, Proc. ITC 24, Cracow, Poland(2012)
[145] W. Leland, M. Taqqu, W. Willinger and D. Wilson, On the self-similar nature of
Ethernet traffic, IEEE/ACM Trans. Networking 2 (1994) 1-15
[146] B. Tsybakov and N. Georganas, On self-similar traffic in ATM queues: Definitions,
overflow probability bound and cell delay distribution, IEEE/ACM Trans. on
Networking 5 (1997) 397-409
[147] IBM Cloud Computing: http://www.ibm.com/cloud-computing/us/en/
[148] Google Developers: https://developers.google.com/
[149] Ze Li, Haiying Shen, Hailang Wang, Guoxin Liu, Jin Li: SocialTube: P2P-assisted
video sharing in online social networks, INFOCOM, 2012 Proceedings IEEE,
Orlando, Florida, USA, pp. 2886-2890, 2012
[150] Fangfei Zhou; Liang Zhang; E. Franco; A. Mislove; R. Revis, R. Sundaram:
WebCloud: Recruiting Social Network Users to Assist in Content Distribution, 11th
IEEE International Symposium on Network Computing and Applications,
Cambridge, MA, USA, pp.10-19, 23-25 Aug. 2012
[151] U.S. Environmental Protection Agency: Report to Congress on Server and Data
Centre Energy Efficiency. ENERGY STAR Program, August 2013. Available at:
http://www.energystar.gov/ia/partners/prod_development/downloads/EPA_Data
centre_Report_Congress_Final1.pdf. Last visited on: March 2013.
[152] Jonathan G. Koomey: Growth in Data Centre Electricity Use 2005 to 2010.
Analytics Press, August 2011. Available at:
http://www.twosides.us/Content/rsPDF_218.pdf . Last visited on: March 2013.
[153] Google Website: Efficiency: How we do it. Google Data Centres Website, 2012.
Available at: http://www.google.com/about/datacenters/efficiency/internal/. Last
visited on: March 2013.
[154] Cisco White Paper: The Exabyte Era. Cisco, White Paper, January, 2008.
Available at: http://www.hbtf.org/files/cisco_ExabyteEra.pdf. Last visited on: March
2013.
[155] T. Silverston and F. Olivier, Measuring P2P IPTV Systems, in Proceedings of the
International Workshop on Network and Operating Systems Support for Digital
Audio and Video (NOSSDAV), 2007.
[156] W. Liang, J. Bi, R. Wu, Z. Li, and C. Li, On Characterizing PPStream:
Measurement and Analysis of P2P IPTV under Large-Scale Broadcasting, in
IEEE Global Telecommunications Conference (GLOBECOM), 2009.
[157] J. Jia, C. Li, and C. Chen, Characterizing PPStream across Internet, in 2007 IFIP
International Conference on Network and Parallel Computing Workshops (NPC
2007), 2007.
[158] Microsoft SQL Azure training: https://www.microsoftvirtualacademy.com/training-
courses/introduction-to-microsoft-sql-azure-training
[159] Windows Azure Data Management: http://www.windowsazure.com/en-
us/home/features/data-management/
Public

[160] "What's New in Windows Azure SQL Database", http://msdn.microsoft.com/en-
us/library/windowsazure/ff602419.aspx
[161] Windows Live: http://en.wikipedia.org/wiki/Windows_Live
[162] "Explore Zune Software", http://www.windowsphone.com/pl-pl/how-
to/wp7/start/get-to-know-the-zune-software
[163] H. Kim, kc claffy, M. Fomenkov, D. Barman, M.Faloutsos, KY. Lee, Internet Traffic
Classification Demystified: Myths, Caveats, and the best practices, ACM CoNEXT
2008, December 10-12, 2008, Madrid, Spain.
[164] R. Dey, Z. Jelveh, K. Ross, Facebook Users Have Become Much More Private: A
Large Scale Study, 4th IEEE International Workshop on Security and Social
Networking (SESOC), Lugano, Switzerland, 2012.
[165] Y.A Wang, C. Huang, J. Li, K.W. Ross, Estimating the Performance of
Hypothetical Cloud Service Deployments: A Measurement-Based Approach,
Infocom 2011, Shanghai, 2011.
[166] T. Karagiannis, A. Broido, M. Faloutsos and kc klaffy, Transport Layer
identification of p2p traffic, ACM IMC, October 2004.
[167] T. Karagiannis, K. Papagiannaki and M. Faloutos, Blinc: Multilevel traffic
classification in the dark, In ACM SIGCOMM, August 2005.
[168] T. Choi, C. Kim, S. Yoon, J. Park, B. Lee H. Kim and H. Chung, Content-aware
Internet application traffic measurement and analysis, IEEE/IFIP NOMS, April
2004.
[169] T.T.Nguyen and G. Armitage, A survey of techniques for Internet traffic
classification using machine learning, IEEE Communications Surveys and
Tutorials, 2008.
[170] Amazon EC2: http://aws.amazon.com/ec2/
[171] Amazon EBS: http://aws.amazon.com/ebs/
[172] Amazon S3: http://aws.amazon.com/s3/
[173] Amazon Cloudfront: http://aws.amazon.com/cloudfront/
[174] Sandvine, Spring 2011 Global Internet Phenomena Report, Tech. Rep., 2011
[175] Google Search: http://en.wikipedia.org/wiki/Google_Search
[176] GoogleBot: http://en.wikipedia.org/wiki/Googlebot
[177] PageRank: http://en.wikipedia.org/wiki/PageRank
[178] Bing: http://en.wikipedia.org/wiki/Bing
[179] Wolfram Alpha: http://en.wikipedia.org/wiki/Wolfram_Alpha
[180] Twitter: https://twitter.com/
[181] Google+: https://plus.google.com/
[182] Linked'n: http://www.linkedin.com/
[183] SmoothIT Deliverable D1.1: Requirements and Application Classes and Traffic
Characteristics (Initial Version), March 2008,
http://smoothit.org/index.php?eID=tx_nawsecuredl&u=0&file=fileadmin/public_deli
Public

verables/D1.1-
v4.0.pdf&t=1365403007&hash=9cc3ea9bcc066862ffa0281ac305e3d7e377f7d4
[184] Compuware: http://www.compuware.com/
[185] Wongyai, W.; Charoenwatana, L., "Examining the network traffic of facebook
homepage retrieval: An end user perspective," Computer Science and Software
Engineering (JCSSE), 2012 International Joint Conference on , vol., no., pp.77,81,
May 30 2012-June 1 2012, doi: 10.1109/JCSSE.2012.6261929.
[186] Facebook features: http://en.wikipedia.org/wiki/Facebook_features
[187] "Facebook and Online Privacy: Attitudes, Behaviors, and Unintended
Consequences", http://www.lib.sfu.ca/sites/default/files/10227/facebook10.pdf.
[188] Vijay Kumar Adhikari, Yang Guo, Fang Hao, Matteo Varvello, Volker Hilt, Moritz
Steiner, Zhi-Li Zhang, Unreeling Netflix: Understanding and Improving Multi-CDN
Movie Delivery, Proceedings IEEE INFOCOM, 2012.
[189] Netflix: http://www.Netflix.com/
[190] Vijay Kumar Adhikari, Yang Guo, Fang Hao, Volker Hilt, Zhi-Li Zhang, A Tale of
Three CDNs: An Active Measurement Study of Hulu and Its CDNs, Global Internet
Symposium, 2012.
[191] Hulu: http://www.hulu.com.
[192] Valancius, Vytautas, et al. "Greening the internet with nano data
centers."Proceedings of the 5th international conference on Emerging networking
experiments and technologies. ACM, 2009.
[193] Cloudnomics:
http://www.rackspace.com/knowledge_center/whitepaper/cloudonomics-the-
economics-of-cloud-computing
[194] G. Halinger, G. Nunzi, C. Meirosu, C. Fan and F.-U. Andersen: Traffic
engineering supported by inherent network management: Analysis of resource
efficiency and cost saving potential, International Journal on Network
Management (IJNM), Vol. 21 (2011) 45-64
[195] G. Halinger and O. Hohlfeld, Efficiency of caches for content distribution on the
Internet, Proc. 22. Internat. Teletraffic Congress, Amsterdam, The Netherlands
(2010)
[196] J. Baliga, K. Hinton, and R. S. Tucker. Energy Consumption of the Internet. In
Proceedings of COINACOFT (2007)
[197] J. Baliga, R. Ayre, K. Hinton, W.V. Sorin, and R.S. Tucker. Energy Consumption
in Optical IP Networks. Journal of Lightwave Technology, 27(13):23912403
(2009)
[198] Vijay Janapa Reddi, Benjamin C. Lee, Trishul Chilimbi, and Kushagra Vaid. 2010.
Web search using mobile cores: quantifying and mitigating the price of efficiency.
In Proceedings of the 37th annual international symposium on Computer
architecture (ISCA '10). ACM, New York, NY, USA, 314-325.
DOI=10.1145/1815961.1816002.
[199] Michael Ferdman, Almutaz Adileh, Onur Kocberber, Stavros Volos, Mohammad
Alisafaee, Djordje Jevdjic, Cansu Kaynak, Adrian Daniel Popescu, Anastasia
Ailamaki, and Babak Falsafi. Clearing the clouds: a study of emerging scale-out
Public

workloads on modern hardware.SIGARCH Comput. Archit. News 40, 1 (March
2012), 37-48. DOI=10.1145/2189750.2150982.
[200] http://www.bing.com/blogs/site_blogs/b/bingjobs/archive/2011/05/26/testing-in-the-
bing-cloud.aspx.
[201] Brian F. Cooper, Eric Baldeschwieler, Rodrigo Fonseca, James J. Kistler, P.P.S.
Narayan, Chuck Neerdaels, Toby Negrin, Raghu Ramakrishnan, Adam
Silberstein, Utkarsh Srivastava, and Raymie Stata, Bulletin of the IEEE Computer
Society Technical Committee on Data Engineering.
[202] Google Apps: Energy Efficiency in the Cloud, Google White Paper,
http://static.googleusercontent.com/external_content/untrusted_dlcp/www.google.c
om/en/us/green/pdf/google-apps.pdf.
[203] Wydrych, Piotr (editor). "Deliverable D2.1: Overlay Traffic Management Solutions."
SmartenIT, April 2013.
[204] SmartenIT. "Grant Agreement for STREP: Annex I 'Description of Work (DoW)'."
2012.

Public

9 Abbreviations
3GPP 3rd Generation Partnership Project
AAA Authentication, Authorization, Accounting
API Application Programming Interface
CAGR Compound Annual Growth Rate
CAPEX Capital Expenditure
CDF Cumulative Distribution Function
CDN Content Distribution Network
CPP Critical Peak Pricing
CPU Central Processing Unit
CRM Customer Relationship Management
DC Data centre
DFA Deterministic Finite Automaton
DNS Domain Name System
DoW Description of Work
FoF Friend-of-Friend
GPS Global Positioning System
GSM Global System for Mobile Communication
HSDPA High-Speed Downlink Packet Access
HTML Hyper-Text Markup Language
HTTP Hyper-Text Transfer Protocol
IaaS Infrastructure as a Service
ICT Information and Communications Technology
IP Internet Protocol
IRT Interoute S.A.
ISN Index Serving Node
ISP Internet Service Provider
IT Information Technology
LTE Long-Term Evolution
MNO Mobile Network Operator
MOS Mean Opinion Score
MPLS Multi-Protocol Label Switching
NIST National Institute of Standards and Technology
NSP Network Service Provider
Public

OLA Operation Level Agreement
OPEX Operating Expenditure
OSN Online Social Network
OTT Over-The-Top
P2P Peer-to-Peer
PaaS Platform as a Service
PC Personal Computer
RAM Random Access Memory
RTMP Real Time Messaging Protocol
QoE Quality of Experience
QoS Quality of Service
REST Representational State Transfer
SaaS Software as a Service
SLA Service Level Agreement
SnF Store and Forward
SOAP Simple Object Access Protocol
SSL Secure Sockets Layer
STREP Specific Targeted Research Project
T4T Tit-for-Tat
TCP Transmission Control Protocol
TDG Telekom Deutschland GmbH
ToU Time of Use
TV Television
UDP User Datagram Protocol
VC Value Chain
VDC Virtual Data Centre
VM Virtual Machine
VN Value Network
VNC Value Network Configuration
VoIP Voice over IP
VPN Virtual Private Network
XML X-tensible Markup Language
WAN Wide Area Network
WiFi Wireless Fidelity - 802.11 standards family

Public

10 Acknowledgements
This deliverable was made possible due to the large and open help of the WP1 team of
the SmartenIT team within this STREP. Many thanks to all of them.

Public

Appendix
Public

A. YouTube Detailed Traffic Characteristics
In this appendix, detailed traffic characteristics of the YouTube video streaming application
are provided based on extensive literature overview, specifically works presented in [55] to
[67] are overviewed.
A.1 Steps in YouTube video download
Watching a video on YouTube involves a different set of servers. Initially, the embedding
Web page is delivered through front end YouTube web servers, whereas the video
content is itself delivered by YouTube video cache servers. YouTube currently supports
two containers for video streaming, Flash and HTML5 [56]. At the time of writing, the
adoption of HTML5 for YouTube playback on PCs is still in an early phase, and almost all
browsers use Flash technology as the default to play the YouTube videos [57]. When the
container is Flash, a dedicated Shockwave Flash player must be downloaded to control
the Flash plug-in in the browser.
Simplified Steps in Accessing a YouTube Video
The process of accessing a YouTube video can be summarized as follows:
1. The user requests a video on the YouTube webpage:
http://www.youtube.com/watch?v=videoID and gets to the Web server that delivers
the YouTube HTML page;
2. After downloading the embedding HTML web page, the other contents are
requested in particular the Shockwave Flash Player (embedded in a HTML object
that contains the video parameters);
3. The actual video content is requested from a cache server (lscache server); if this
cache is over-loaded, it sends a redirect (HTTP 302) message to the client
indicating another cache server;
4. The client sends a request the other cache server (tccache server) for the video,
and the FLV file is delivered to the client while being played in the Flash player
(Progressive Download).
The details of the redirections depend on the load of the cache servers and are explained
in the following. We now focus on the video content delivery, and more specifically on the
architecture and the interaction with the cache server infrastructure.
YouTube Cache Server Infrastructure
The users receive the video they request from a cache node. Individual cache nodes are
organized in cache clusters, with all the machines of a cache cluster being co-located. The
number of machines per cache cluster is highly variable and depends, among others, on
the demand for service issued in the region where the cluster is located and also on the
available physical space to host the cache nodes. Each cache node as of 2011 has a 10
Gb/sec network access and 78 TBs of disk storage [58].
The various cache clusters are organized in a three tier hierarchy. The global
infrastructure of the YouTube caches has been revealed by Adhikari et al. [59] in 2011.
They used the distributed infrastructure of the PlanetLab network to request thousands of
videos from different vantage points in the world, which allowed to reverse engineer the
cache infrastructure and the cache selection policies. The work in [55] complements their
Public

findings with more active measurements [60] undertaken in 2011 and 2012 from France.
The analysis in [55] focuses on residential Internet access and reveals the techniques
applied by Google to deliver the videos to residential customers. Since our machines
where connected to the Internet through different ISPs, we were able to observe
differences in treatment of customers coming from different ISPs.
YouTube has a three tier caching infrastructure that comprises of four different logical
namespaces as shown in Figure A-1. The machines allocated to the different cache
clusters are identified via particular naming conventions. We recall here the main findings
of [61] on the YouTube cache clusters. As of 2011 there are:
- 38 primary cache clusters with about 5000 unique IP addresses corresponding to
the lscache namespace;
- 8 secondary cache clusters corresponding to the tccache namespaces with about
650 IP addresses;
- tertiary caches clusters corresponding to the cache and altcache namespaces with
about 300 IP addresses.

Figure A-1: Organization of the YouTube Caches; dashed lines indicate possible
redirections - Figure taken from [55].
All these cache clusters are located in a total of 47 different locations distributed over four
continents; there is no cache cluster in Africa. About ten primary cache clusters are co-
located inside ISPs and the IP addresses of these cache nodes in these clusters are not
part of the address space managed by Google but part of the ISPs address space. Since
we have 38 + 8 + 5 = 51 cache clusters but only 47 different locations, some cache
clusters belonging to different levels of the hierarchy must be at the same physical
location (i.e., some primary and secondary caches are co-located).
For the caches in each cache cluster, a particular logical naming structure is applied.
- Each primary cache cluster has a total of 192 logical caches corresponding to the
lscache namespace, which looks as follows: city_code.v[1-24].lscache[1-
8].c.youtube.com. As city_code the three letter code for the airport closest to that
cache cluster is used.
- There are also 192 logical caches in each secondary cache cluster, corresponding
to the tccache namespace, which are named as follows tc.v[1-24].cache[1-
8].c.youtube.com.
- Each tertiary cache cluster has 64 logical caches corresponding to cache and
altcache namespaces.
Public

Introducing these logical name spaces has the following advantages:
- Each video ID can be deterministically mapped via consistent hashing onto a
unique logical name in the lscache namespace, which makes it easy to decide for
each cache what portion of the videos it is responsible to serve.
- There is a one-to-one mapping between the lscache and tccache namespace.
- The logical naming is the same for each cache cluster and it is completely
independent of the number of real cache nodes in a particular cache cluster.
- It is the responsibility of DNS to map logical cache names onto the IP addresses of
real cache nodes. In [61], each of the logical names from the lscache namespace is
mapped to more than 75 different IP addresses distributed over the 38 primary
cache clusters.
A.2 YouTube redirection process
DNS Level Redirections: YouTube uses HTTP redirections to balance the load among its
caches. As shown in Figure, the redirections usually direct the video request from a cache
layer to the next one. Using traces from a European ISP, Torres et al. [62] observed that
as the total number of requests kept increasing, the percentage of requests handled by
the closest cache cluster located inside that ISP decreased from 80% to about 30%. In
this case, DNS request resolution will direct clients to more distant but less loaded cache
clusters.
Impact of Redirections on Performance: Each redirection involves:
1. DNS query to resolve the hostname of the next cache node,
2. Opening of a new TCP connection,
3. Issuing a new video query.
In case of redirections, the final cache node serving the video will most likely not be the
closest one in terms of RTT, which has been observed in [62] for the most popular videos
of the day. The impact of redirection on the time until the first MB is downloaded (referred
to as video initialization time) has also been studied in [61]. The video initialization time is
on average 33% higher if the video has been fetched through redirections. The fraction of
sessions that have been redirected is evaluated in [57]: between 10% and 30% of all
sessions are redirected at least once. The impact of redirections on the startup delay can
also be important [57]:
- Without redirections, delays are in the order of milliseconds;
- With redirections, delay can increase by orders of magnitude, up to 10 seconds.
A.3 Application-layer traffic management
YouTube videos are requested using HTTP over TCP Both error control and congestion
control of TCP may result in high delay jitter.
The delivery strategy of YouTube videos has been studied in great detail by Rao et al.
[56]. The authors show that the delivery strategy depends on the video container (Flash,
Flash High Definition, or HTLM5), the type of client device (PC or mobile devices such as
Public

smart phones or tablets), and the type of browser (Internet Explorer, Chrome, or Firefox).
The delivery strategy needs to reconcile a number of potentially conflicting goals such as:
- Smooth playout during the entire duration of a viewing session;
- Efficient use of the server resources such as disk I/O and timer management;
- Avoid to transmit too much data in advance of consumption in order to (i) reduce
the amount of buffering at the client, which is particularly relevant in the case of
mobile devices and to (ii) reduce the waste of network and server resources by
sending data that are never used.
Finamore et al. [57] observed that 60% of the videos requested were watched for less
than 20% of their total duration, resulting in an un-necessary transfer of 2539% of the
data. As we shall see in the following section, the impact of playback degradation is a
primary factor in the video transfer interruption. As the video transfer is done via HTTP
over TCP, there is no guarantee that the data can be delivered to the client at the rate at
least as high as the one at which they are consumed. The details of the transfer have
been studied in [63], whose findings we summarize in the following: To increase the
likelihood of a smooth playback, YouTube performs aggressive buffering when a video is
requested. Initially, during a startup phase, the server sends as fast as possible to fill up
the initial client playout buffer. This play-out buffer contains about 40 seconds with Flash,
and 1015 MB with HTML5 with Internet Explorer as a browser, which is typically much
more than 40 seconds worth of video. Once the initial buffer has been filled, two other
strategies are used by the cache server:
- Keeps sending as fast as possible, until entire video is transmitted;
- Limits the rate of the transfer alternating between on-off cycles with a fixed period.
During an on cycle, a fixed size block of video data is transmitted.
The first option is used for HD content, the second one otherwise [63]. We limit our
description to the case of streaming a video to a PC with Flash as container, and refer to
the original paper [56] for more details. Streaming the video to a PC has been the most
extensively studied case [60], [64]. In this case, the server streaming strategy is
independent of the browser: When the startup phase is terminated, the cache server
sends blocks of 64 KBs at a frequency that allows achieving an average transmission rate
of 1.25 times the video encoding rate. As has been first observed by Alcock and Nelson
[60], injecting bursts of 64 KBs means sending 45 maximum size TCP segments back-to-
back into the network. Such large packet bursts will accumulate in the buffer of the
bottleneck link and (i) cause delay spikes that may disrupt other latency sensitive
application and (ii) inflict loss on the bursty YouTube flow itself. In response to these
problems, Google engineers have recently proposed a modification to the server side
sending policy that controls the amount of packets that can be injected back-to-back in
order to limit the size of the packet bursts. The main idea is to adjust the sending window
of the TCP connection on the server dynamically based on the estimated round trip time of
the connection. For details of the new sender algorithm and its impact on packet loss and
burst size see [64].
In this section we have provided the technical basis to understand YouTube content
delivery over the Internet. Next, we investigate what influences the QoE experienced by
the user. In particular, problems in the network may lead to stalling and QoE degradations.
Therefore, we have to identify the key factors that influence YouTube QoE by means of
Public

subjective measurements and build an appropriate model, which can be used for QoE
monitoring later on.
A.4 QoE of YouTube video streaming key influence factors
The following section shows the key results on QoE of YouTube taken from [55]. For a
detailed description of the measurement campaign, we refer the reader to [55].
When it comes to predicting QoE of YouTube, an essential step is determining those key
factors that have the strongest influence on the actual experience. Therefore, we analyse
correlation coefficients as well as support vector machine (SVM) weights [65]. The
Spearman rank-order correlation coefficient between the subjective user rating and the
above mentioned variables is computed. In addition, SVMs are utilized to obtain a model
for classification: Every variable gets a weight from the model indicating the importance of
the variable. However, SVMs are acting on two-class problems only and cannot
differentiate the five different quality categories which were used in these tests on an
absolute category rating (ACR) scale: (1) bad; (2) poor; (3) fair; (4) good; (5) excellent.
Therefore, the categories 1 to 3 of the ACR scale to the bad quality class and the
categories 4 to 5 to the good quality class.
Figure A-2(a) shows the results from the key influence analysis. On the x-axis, the
different influence factors are considered, while the y-axis depicts the correlation
coefficient as well as the SVM weights which are normalized to the largest correlation
coefficient for the sake of readability. We can clearly observe from both measures that the
stalling parameters dominate and are the key influence factors. It is interesting to note that
the user ratings are statistically independent from the video parameters (such as
resolution, video motion, type of content, etc.), the usage pattern of the user, as well as its
access speed. In particular, we used different video contents, but got no differences on
the user ratings. A possible explanation may be that a video user does not care about the
video quality (i.e. which encoding scheme is used, what is the video bit rate, etc.). If the
video stalls, the video experience of the user is disturbed independent of the actual
video characteristics. Thus, for a YouTube QoE model, those video characteristics can be
neglected in contrast to the actual stalling pattern. However from a networking point of
view, higher video bitrates lead to more stalling events if there is a bottleneck in the
network. Hence, the video bit rate may be considered for QoE monitoring to estimate the
number of stalling events.
However, the videos considered in the experiments [65] had a fixed length of 30 s and no
initial delays for buffering the video contents were considered. Therefore, a second row of
experiments was conducted [66] in which the following parameters were varied: number of
stalling events, duration of a single stalling event, video duration, initial delay. Hence in
addition to the first row of subjective studies, the video duration and initial delays were
considered as influence factors. To check again the influence of user demographics on
QoE, the age, sex, and user id were also considered in the analysis.
The results in Figure A-2(b) reveal again that the number of stalling events together with
the stalling length are clearly dominating the user perceived quality, while the user
demographics have no influence. Furthermore, the impact of initial delays is statistically
not significant. We take a closer look at initial delays that may be accepted by the user for
filling up the video buffers to avoid stalling. In case of bad network conditions, providers
need to select a trade-off between these two impairment types, i.e. stalling or initial
delays, which allows QoE management for YouTube video streaming clouds [67].
Public

(a) Stalling parameters, video
characteristics, demographics [65].
(b) Stalling parameters, video duration,
initial delays [66].
Figure A-2: Identification of key influence factors on YouTube QoE.
Therefore, we ask the question whether initial delays are less harmful to QoE than stalling
events for YouTube. However, the results in Figure A-3 show that no statistical differences
are observed for video clips of 30 s. and 60 s. regarding the impact of initial delays on the
QoE. QoE is thereby quantified in terms of Mean Opinion Scores (MOS) which is the
average value of user ratings on the ACR scale for a given test condition, i.e. a certain
initial delay in that case. In summary, the QoE is mainly determined by the number and
the length of stalling events. In contrast, the initial delay until the video playback starts is of
no or only minor importance.

Figure A-3: Initial delays have almost no influence on MOS for videos of duration 60 s and
30 s compared to influence of stalling length [66].
A.5 Consequences for SmartenIT
To assess the intervention and optimization potential of a solution for video streaming
developed in SmartenIT it is important to be aware of the mechanisms that are currently
used for video delivery and to identify the factors that have an influence on the quality of
the provided service. In this appendix we considered the most prominent video streaming
Public

platform YouTube and showed its service infrastructure, traffic management strategies
and key influence factors.
The well-engineered caching infrastructure and locality aware request redirection shows
that in the case of YouTube the potential of optimizing content delivery is rather low, but
also provides an example how traffic management is implemented in an efficient CDN.
The investigations on QoE show, that a SmartenIT solution for video streaming has to
consider stalling as the main influence factor on the user perceived quality.
Public

B. Dropbox Detailed Traffic Characteristics
In this section, detailed traffic characteristics of the Dropbox online storage application are
provided based on work presented in [29].
B.1 Monitoring setup
The authors of [29] conducted experiments to identify what information is exchanged and
what traffic is generated when particular operations take place. These operations include:
- adding or removing files on local folders,
- downloading new files, and
- creating new folders.
During the data collection, Dropbox client version 1.2.52 was used. As Dropbox
communications are encrypted with the TLS protocol and no official description of the
Dropbox is published, a local test-bed was employed: a Dropbox client run in a Linux PC
and used a Squid proxy server under the authors control. On the proxy server, the SSL-
bump module [80] was used to save decrypted traffic flows. Additionally, the original
Dropbox Inc. certificate was replaced at run-time by a self-signed one signing the proxy
server.
Figure B-1 depicts the messages observed while storing a batch of chunks. Initially, the
Dropbox client registers with Dropbox control centre via a clientX.dropbox.com server. As
soon as new files are added in the local folder, a commit batch command on client-
lb.dropbox.com submits meta-information. This meta-information may trigger store
operations directly with the Amazon servers on dl-clientX.dropbox.com; store
operations are marked with grey background in the figure. Note that each operation is
acknowledged by an OK message. Finally, as chunks are successfully submitted, the
client exchanges messages with the Dropbox server on client-lb.dropbox.com to
conclude the transactions.

Figure B-1: An example of the Dropbox communication observed [29].
The knowledge obtained from the aforementioned observations is used to identify with
high probability what operations are performed by the (encrypted) monitored TCP flows.
The Dropbox client was found to exchange data flows mostly with Dropbox Inc. servers; in
particular, three groups of servers were identified:
Public

i. notification servers which provide information upon request to clients on changes
performed elsewhere; these do not included changes in the central storage, as the
latter are advertised as soon as they are performed,
ii. meta-data administration servers which are responsible for authentication and file
meta-data administration; typically, synchronization transaction start with
communication with the meta-data administration servers, and
iii. system-log servers which collect run-time information about the clients, including
exception back-traces (via Amazon, on dl-debug.dropbox.com), and other
event logs possibly useful for system optimization (on d.dropbox.com).
Concerning the main store and retrieve operations, these are all handled by VMs in
Amazon EC2. Clients were observed to communicate with 500 distinct domain names
pointing to Amazon servers in rotation for storage operations in order to distribute the
workload. Additionally, content stored in Dropbox can be accessed through the Dropbox
web interface. For access through the web interface different domain names are used,
e.g., dl-web.dropbox.com for content download from private folders, while
dl.dropbox.com provides public direct links to shared files.
B.2 Methodology and data sets
In [29], the authors extended and used Tstat [81], an open source monitoring tool
developed at Politecnico di Torino to collect data. Tstat monitors each TCP connection
and acquires information about more than 100 different metrics including: client and server
IP addresses, amount of exchanged data, retransmitted segments, etc.
First, in order to extract TLS/SSL certificates offered by the server, a classic DPI approach
was used. The analysis showed that the string *.dropbox.com is used to sign all
communications with the Dropbox server. Second, the Fully Qualified Domain Name
(FQDN) was requested to the DNS server by the clients for server IP addressed in order to
reveal the servers location in the DNS tree hierarchy and to identify each specific Dropbox
service. Third, Tstat was instructed to expose the list of device identifiers and folder
namespaces exchanged with the notification servers.
Table B-1: Datasets for Dropbox evaluation [29].
Name Type IP addresses
Traffic volume
(GB)
Campus 1 Wired 400 5,320
Campus 2 Wired / Wireless 2,528 55,054
Home 1 FTTH / ADSL 18,785 509,909
Home 2 ADSL 13,723 301,448
Monitoring by Tstat took place in 4 points in 2 European countries from March 24th, 2012
to May 5th, 2012. Table B-1 summarizes the dataset for each point, reporting the type of
access technology present, the number of unique IP addresses and the total amount of
data observed in GBs for the entire monitoring period. Note that datasets named Home 1
and Home 2 refer to ADSL and FTTH customers of a nation-wide ISP; these customers
were provided with static IP addresses but the connection could be shared at their home
through their WiFi router to many devices. Campus 1 and Campus 2 refer to (as inferred
Public

by their name) to users in two academic environments, i.e., Campus 1 refers to research
and administrative offices of the Computer Science Department in a European university,
while Campus 2 to the peripheral network of routers of another European university
including campus-wide WiFi hotspots and student dormitories.
B.3 Popularity of different cloud storage providers
Initially, a comparison of the popularity of different cloud-based storage systems was
conducted by the authors in [29]. More specifically, the following services were
considered: Dropbox, Google Drive, Apple iCloud, and Microsoft SkyDrive. Moreover,
other less popular services, e.g., SugarSync, Box.com, UbuntuOne) were considered too,
and reported in [29] as the Others group.
Figure B-2 illustrates the popularity of all considered services in terms of unique IP
addresses belonging to the Home 1 dataset (where static IP addresses are assigned
among end users). that contacted the respective servers at least once within a given day.
Apple iCloud appears to be the most accessed service, i.e., 11.1% of the total number of
IP addresses, followed by Dropbox, i.e., 6.9%; other services present much lower
percentages, e.g., Microsoft SkyDrive has 1.7% popularity, while Google Drive just
appeared just before the end of the monitoring period, thus traces appear only for a
smaller time interval.

Figure B-2: Number of unique clients accessing each cloud-based storage service in
Home 1 [29].

Figure B-3: Data volume per cloud-based storage service (in bytes) in Home 1 [29].
Public

Next, Figure B-3 depicts the total data volume for each service in Home 1 dataset. For this
metric, Dropbox tops all other cloud-based storage services by at least one order of
magnitude, i.e., > 10 GB vs. ~1 GB, with more than 20 GB of traffic exchanged daily.
Although iCloud is accessed by more users daily, the data volumes generated are
significantly lower, due to the fact that users cannot download/upload arbitrary files in their
devices. SkyDrive only presents a sudden increase towards the end of the monitoring
period, though it returns back to its regular daily traffic volume afterwards.
In Figure B-4, the share of total traffic volume for YouTube and Dropbox is illustrated for
Campus 2. As it can be seen, Dropbox is responsible for 100 GB traffic, which is a quite
high share of total traffic and up to 1/3 of YouTube's traffic in a daily pattern. In total, more
than 3.5 TB were exchanged with Dropbox servers during the monitoring period.

Figure B-4: Share of total traffic volume in Campus 2 [29].
Generally, the aforementioned findings reveal according to the authors also the increasing
popularity of cloud-based storage systems; already 6-12% of home users access such
services in a daily basis.
B.4 Performance
The amount of traffic handled by the different sets of servers is studied here. Figure B-5
shows the resulting traffic breakdown in terms of traffic volume and number of flows. It can
be observed that the Dropbox client application is responsible for more than 80% of the
traffic volume in all points, which implies that the application client is highly preferred over
the Web interfaces for exchanging data. Respectively, considering the flows number, then
control servers for client communications are the dominant contributors.
As discussed in Section B.1, Dropbox distributes the load among its servers both by
rotating IP addresses in DNS responses and by providing different lists of DNS names to
each client. Figure B-6 shows the number of contacted storage servers per day in the four
points. Figure B-6 points out that clients in Campus 1 and Home 2 do not reach all storage
servers daily, while, in both Campus 2 and Home 1, more servers are contacted because
of the higher number of devices on those points.
Additionally, during the monitoring, it was identified that all Dropbox requests from clients
were directed towards the same IP addresses regardless of their geographical location.
This practically implies that the same data centres are used by all users around the globe;
thus Dropbox is a centralized service in US.
Public

Additionally, notable is the fact that there is significant difference in the RTT between
control and storage servers. Most probably this is due to the fact that control servers are
located in the U.S. West Coast (e.g., California), while storage servers are in the U.S. East
Coast (e.g., Amazons Northern Virginia data-centres); the latter was also revealed by
taking into account routing information.

Figure B-5: Traffic breakdown between Dropbox servers [29].

Figure B-6: Number of contacted storage servers [29].
In Figure B-7, we observe that up to 40% of flows exchange less than 10 KB, meaning
that they are composed mostly by SSL handshake messages and a small amount of user
data. Moreover, a very high percentage of flows, i.e., varying from 40% to 80%, consist of
less than 100 KB, while most flows were observed to be composed by only a small
number of chunks, i.e., storage flows have no more than 10 chunks in more than 80% of
the cases in all datasets. This behavior is most probably due to:
i. the synchronization protocol sending and receiving file deltas as soon as they are
detected, or
ii. the primary use of Dropbox for synchronization of small files constantly changed,
instead of periodic (large) backups.
Public

Comparing the CDFs for the retrieve and storage operations, we can see that retrieve
flows are normally larger than the store ones, which was expected as many devices were
noticed to be using Dropbox only for downloading.

Figure B-7: Distribution of TCP flow sizes of file storage for the Dropbox client [29].
Finally, the throughput of the Dropbox application is evaluated. In Figure B-8(a) and Figure
B-8(b), the throughput for store and retrieve operations is illustrated. The x-axis represents
the number of bytes transferred in the flow, already subtracting the typical SSL overheads,
while the y-axis shows the throughput calculated as the ratio between transferred bytes
and duration of each flow. The duration was accounted as the time between the first TCP
SYN packet and the last packet with payload in the flow, ignoring connection termination
delays. Flows are represented by different marks according to their number of chunks,
whereas is the maximum attainable throughput (note that is a loose bound for retrieve
operation).
Overall, the throughput is low. Additionally, throughput for store is lower than throughput
for retrieve as expected. In general, the highest observed throughput is only achieved by
flows carrying more than 1 MB. Moreover, flows achieve lower throughput as the number
of chunks increases. This can be seen by the concentration of flows with high number of
chunks in the bottom part of the plots for any given size.

Figure B-8: Throughput of storage flows in Campus 2 [29].
Generally, the throughput is limited by two major factors, i.e., TCP slow start-up times and
Public

application-layer sequential acknowledgments, while flows with a small amount of data and
flows with a large number of chunks are mostly affected. In both cases, high RTT between
clients and data centres amplifies the effects. Those problems are further detailed in [29].
B.5 Service workload
Regarding the ratio of total downloaded data to total upload data, it is observed that users
tend to download more than they upload. Specifically, this ratio is equal to 2.4 for users in
Campus 2 (on average), 1.6 for Campus 1, 1.4 for users in Home 1 and 0.9 only for users
in Home 2; in the latter case only few customers were found to massively upload data and
hence affected the ratio significantly.
Users' behaviour can be distinguished in four categories:
i. occasional users, who abandon their Dropbox clients without synchronizing any
content,
ii. upload-only users, who mainly perform store operations,
iii. download-only users, who mainly perform retrieve operations, and
iv. heavy users, who perform both store and retrieve for large amounts of data.
The occasional group represents about 30% of total number of IP addresses in all four
points. The occasional users exchange negligible amounts of data and stay connected for
shortest time intervals (compared to other groups). Therefore, the service workload
generated by them is marginal.
The upload-only group accounts for 7% of total population of IP addresses monitored, and
it is responsible for up to 21% of total traffic uploaded in Home 1 and 11% in Home 2. The
upload-only users are interested mostly in back-ups and submission of data to third-
parties or geographically dispersed devices. On the other hand, the download only group
accounts for 26% in Home 1 and 28% in Home 2, and generates about 25% and 28% of
total download traffic volume in Home 1 and Home 2, respectively.
The fourth group, namely the heavy users, accounts for up to 37% of IP addresses in
Home 1 and 33% in Home 2. These users are generating the most of the volume
transferred by Dropbox clients, own at least 2 devices, were observed to stay on-line more
than 60% of the total monitoring period and initiated more than 50% of total Dropbox
sessions. The usage of Dropbox for synchronization is a typical scenario for this group.
Therefore, this group is the one having the highest impact on both workload and network
utilization.
Concerning the number of devices, only 1 device was observed to be used in about 60%
of all households using the Dropbox client. The remaining households may have up to 4
devices (up to 40%) and a few of them even more; as expected most of them belong to
the heavy group. Figure B-9 depicts the distribution of the number of devices per
household for Home 1 and Home 2.
Next, in order to measure the extent of Dropbox's usage for content sharing and
collaborative work, the number of namespaces per device was examined. In particular, the
number of users with only 1 namespace (the users root folder) is small: 13% in Campus 1
and 28% in Home 1, while having 5 or more namespaces is equal to 50% in the former,
and 23% in the latter. In general though, users in Campus 1 have more namespaces than
Public

in Home 1. When considering only IP addresses assigned to workstations in Campus 1,
each device had on average 3.86 namespaces.

Figure B-9: Distribution of the number of devices per household [29].
Monitoring the session start-ups by distinct devices and the number of active devices per
time interval, it was discovered that in Campus 1 and Campus 2 both metrics are related
to employee's office hours (i.e., no activity at night), while in Home 1 and Home 2, some
activity was observed early in the morning, and most of the activity took place during
evening hours.
Analyzing the session duration, it was observed that in both home networks, a significant
number of notification flows most of which are originating from a few devices are
terminated in less than 1 minute. These seem to be abruptly terminated due to network
equipment (e.g., NATs or firewalls). As depicted in Figure B-10, most devices in Home 1,
Home 2 and Campus 2 stay connected up to 4 hours in a single session, while in Campus
1, a significantly higher percentage of long-lasting sessions is seen.

Figure B-10: Dropbox session duration [29].
B.6 Consequences for SmartenIT
In this appendix, we overviewed the in depth traffic characterization of Dropbox provided
in [29], which highlights some of Dropbox's inefficiencies, and thus to identify potential for
intervention and optimization. We focused in Dropbox as it is the most prominent example
of online storage applications, which are observed to win an increasing popularity among
Public

Internet users and to contribute significantly to the IP traffic. To assess the intervention
and optimization potential of a solution for online storage developed by SmartenIT in
WP2, it is important to be aware of the mechanisms that are currently used for in Dropbox
and to identify the factors that have an influence on the quality of the provided service.

Public

C. Facebook and YouTube Traffic Statistics
In this section, measurements for Facebook and YouTube traffic profiles are provided, as
well as evaluations of the 2
nd
order statistics proposed in [84] as a basis for
traffic characterization and modeling.
C.1 Packet and Flow Measurement for YouTube and Facebook
In [142], the authors evaluated measurements from a link in the aggregation of the IP
network of Telekom Deutschland GmbH (TDG) by means of a hardware tool for quality
management which captures headers and timestamps of IP packets being exact to the
microsecond. In a case study, they analysed traffic on three parallel 1 Gb/s links
connecting aggregation routers at the boundary of the backbone. The monitored links
connected residential users to the IP network with DSL access bandwidths of up to 16
Mb/s. The results presented in this section are based on a typical measurement trace from
December, 18
th
2011, in a busy hour period 79 p.m.
Within these two hours, around 600 million IP packets have been counted in the downlink.
For the purpose of our statistics, packets are aggregated into flows. A flow is a single TCP
connection or UDP application specified by source/destination IP addresses, TCP ports
and a QoS marking in the packet headers. This quintuple enables the evaluation of
statistics per flow and for aggregates of flows.
Traffic profiles were not only captured for the total traffic, but also for components
contributed by relevant applications. The authors examined different classes of the total
downstream traffic for the popular application platforms YouTube and Facebook. Traffic
has been classified based on IP addresses, ports and traffic pattern rather than using
Deep Packet Inspection (DPI). Due to this deliberately simple approach, only a subset of
each considered application is identified. Statistics on packets (number, size) and flows
(number, size, rate, 2
nd
order statistics) have been collected and summarized along with
parameters selected in Table C-1.
Table C-1: Packet and flow measurement for YouTube and Facebook [142].
Measurement
statistics
Number of
Packets
Number of
Flows
Mean Packet
Size [Byte]
Mean Flow
Volume [MB]
Mean Flow
Rate [Mb/s]
YouTube 87 201 701 34 977 1 462 3.9 2.0
Facebook 11 594 608 14 671 891 0.6 0.2
Complete Traffic 632 705 587 257 010 1 150 2.8 1.0
Table C-1 gives an overview of the mean values for the complete traffic as well as for
portions of traffic which have been identified to belong to YouTube and Facebook
applications. Moreover, Figure C-1 gives more detailed insights into the cumulative
distribution function (CDF) of the packet and flow size. The graphs show that Facebook
traffic has shorter packets and flows than the total traffic. In YouTube traffic, 90% of large
1500 Byte packets are dominant corresponding to a large number of full size packets in
long flows. Facebook IP packet sizes are partly small (25% at ~50 Byte) with only one half
of the packets around 1500 Byte, which indicates smaller flows and/or a mix of different
applications. The flow rate distribution shown in Figure C-2 is also largely different with
50% Facebook flows at a mean rate below 10kb/s comparing to less than 10% YouTube
Public

flows in the 10kb/s range, whereas 50% of the YouTube flows are streaming at more than
1Mb/s while only 5% of Facebook flows achieve 1Mb/s.
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
0 500 1000 1500
CDF of the Packet Size [Byte]
Facebook
YouTube
Complete Traffic

0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
0,1 1 10 100
CDF of the Flow Volume [MByte]
Facebook
YouTube
Complete Traffic

Figure C-1: Statistics on packet and flow sizes [142].
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
0,001 0,01 0,1 1 10 100
CDF of the Flow Rate [Mb/s]
Facebook
YouTube
Complete Traffic

Figure C-2: Statistics on the mean traffic rates per flow [142].
C.2 2
nd
order statistics results for YouTube and Facebook traffic
variability
In this section, the variability of traffic components is considered, which has to be taken
into account in the dimensioning of network links. Link bandwidth must cover the mean
expected traffic rate plus an additional portion depending on traffic variability for sufficient
QoS support, e.g., the mean plus 3-fold standard deviation of the traffic rate in an
appropriate time scale as a dimensioning rule. Moreover, the 2
nd
order statistics of traffic
flows can be used as a criterion that helps to differentiate and identify application types.
C.2.1 Definition of 2
nd
order statistics and autocorrelation function
The 2
nd
order statistics is a basic characteristic of traffic variability [142], [144], [145], [146].
Therefore, a covariance stationary stochastic process
R = {R
t
; t = 0, 1,2, }
with mean = E(R) and variance o
2
= E(R
2
)
2
is considered. For N = 1, 2, , we
denote the average over sequenced blocks of size N in the time series R
t
as
Public

R
(N)
= {R
t
(N)
; t = 0, 1,2, } with R
t
(N)
= (R
tN
+ R
tN + 1
+ + R
tN + N 1
) / N.
The variance o
N
2
of R
(N)
is denoted as the 2
nd
order statistics of the process including o
1
2
=
o

2
. An alternative and equivalent measure for the correlation of a process in different time
scales is the autocorrelation function
o(k) = cov(R
t
, R
t+k
) /o
2
= E((R
t
)(R

t+k
))/o
2
.
The relationship is due to computation rules for the variance of a sum of N random
variables R
tN
+ R
tN + 1
+ + R
tN + N 1
= N R
t
(N)
via covariance coefficients cov(R
tN+i
, R
tN +j
) [142]:

=
+ +
=

=
+ = =
1
1
2
1
0
1
0
2 2
) ( ) ( ) ( 2 ) , cov(
N
k
j tN i tN
N
i
N
j
N
k k N N R R N o o o
This equation determines o
N
2
from o(1), , o(N1) and o

2
. Vice versa, it can also be
transformed in order to obtain o(N) from (o
2
=)o
1
2
, o
2
2
, ,o
N+1
2
.
C.2.2 2
nd
order statistics of self-similar processes
Self-similar models are defined based on properties of their 2
nd
order statistics. A process
R is exactly 2nd order self-similar with Hurst parameter H (0.5 s H < 1) [145], [146], if
. ) ( :
2 2 2 ) ( 2 2
= =
H N
N
N R N o o o
The same property is also valid for each of the derived processes R
(N)
. In this way, the
result is transferrable to time scales of coarser granularity, i.e., longer time frames with
increasing N. The autocorrelation function is again proven to be invariant for all derived
process R
(N)
[145], [146].
C.2.3 2
nd
order statistics for Gilbert-Elliott and 2-state (semi-)Markov models
The 2
nd
order statistics of a Gilbert-Elliott model has been derived in [143]:
|
|
.
|
\
|
(
+

+ =
) (
) 1 ( 1
1
) (
) )( 1 ( 2
1
3
2
2 2
q p N
q p
q p
h h q p pq
p p
N
N
G B
E E N
o
with state specific error rates error h
G
and h
B
, total error rate p
E
= (p h
G
+q h
B
)/(p+q) and
o
2
= p
E
(1 p
E
), where p and q are the transition probabilities from state G to B and vice
versa from B to G.
As preconditions for the derivation, the 2-state Markov chain is assumed to start in steady
state conditions, i.e., Pr{S
0
= G}= p/(p+q) and Pr{S
0
= B}= q/(p+q) for the initial state S
0
.An
extended result of the same type holds for the 2
nd
order statistics of 2-state semi-Markov
traffic models under the same steady state assumptions. As a general 2-state semi-
Markov result derived in [142], it is obtained:
|
|
.
|
\
|
(
+

+ =
) (
) 1 ( 1
1
) (
) )( 1 ( 2
1
3
2
2 2
q p N
q p
q p
q p pq
N
N
B G
N

o o
q p
q p
q p
q p
B G B B G G
+
+
=
+
+ + +
=

o o
o ;
) ( ) (
2
2 2 2 2
2

where p and q are again the transition probabilities and
G
and
B
are the state specific
mean traffic rates. As a main result of traffic modeling by self-similar, full 2-state semi-
Public

Markov, Gilbert-Elliott and 2-state Markov-modulated Poisson processes, the adaptation
flexibility of full 2-state semi-Markov models is shown to be much better than for all
mentioned alternatives, since 2-state semi-Markov models in general provide 2
parameters, which form the shape of their 2
nd
order statistic curve, whereas special 2-
state as well as self-similar processes have only a single adaptation parameter for the 2
nd

order statistics.
C.2.4 Measurement evaluations of the 2
nd
order statistics
The burstiness of traffic measurements was also evaluated in terms of the ratio of the
standard deviation to the mean o / in different time scales. Figure C-3 shows that the
total traffic has a low burstiness of o / = 0.3 already in the smallest considered time
scale of 1ms. Therefore a Byte count of all packets arriving per 1ms intervals was taken
as the traffic volume on this time scale, from which 1ms samples of the variable traffic rate
were computed. When statistics became available on the 1ms time scale, then the
authors easily summed up the mean traffic rate on larger time scales of 0.01s, 0.1s, 1.0s,
, etc., and thus, obtained the traffic variability on those time scales as well. Generally,
the standard deviation of the rate is decreasing for longer time scales.
0
0,5
1
1,5
2
0,001 0,01 0,1 1 10 100 1000
Time scale [s]
S
t
a
n
d
a
r
d

D
e
v
i
a
t
i
o
n

/

M
e
a
n

R
a
t
e
Facebook
YouTube
Total Traffic
o=o
1
o
10
o
100
o
1000
0
0,5
1
1,5
2
0,001 0,01 0,1 1 10 100 1000
Time scale [s]
S
t
a
n
d
a
r
d

D
e
v
i
a
t
i
o
n

/

M
e
a
n

R
a
t
e
Facebook
YouTube
Total Traffic
o=o
1
o
10
o
100
o
1000

Figure C-3: Burstiness over time scales for YouTube, Facebook traffic parts [142].
According to [142], evaluation results showed that the identified partial Facebook traffic
volume has a burstiness starting from 2.2 for 1ms and stays above the maximum of 0.3 for
the total traffic up to the 10s time scale. On the other hand, the burstiness of the YouTube
traffic portion lies between the Facebook and the total traffic curves over the entire time
scales. The burstiness of traffic aggregates is influenced by the statistical multiplexing
effect, i.e., variability is smoothing down in larger traffic aggregates, which is confirmed in
the results of Figure C-3.
Moreover, the burstiness also for single flows has been computed. In average, the
standard deviation of the data rate of a single Facebook flow is 9.5 times higher than its
mean data rate.
Public

C.3 Consequences for SmartenIT
We have evaluated traffic profiles for different fractions of traffic that can be identified to
belong to application platforms. Measurement statistics on packets and flows confirm that
e.g. YouTube traffic has larger flow volumes than the total traffic mix, whereas Facebook
has much more short flows than in the average. The 2
nd
order statistics of traffic fractions
is included as a measure of traffic variability which is relevant for proper link dimensioning
in order to provide enough extra capacity for temporal overload. An update of packet and
flow measurement evaluation is planned for further study within the SmartenIT project with
a more detailed view on the current traffic and application mix.

Sprache

Willkommen bei Scribd!

Deliverable D1.1 Report On Stakeholders Characterization and Traffic Characteristics

Hochgeladen von

Dokumentinformationen

Originaltitel

Copyright

Verfügbare Formate

Dieses Dokument teilen

Dokument teilen oder einbetten

Freigabeoptionen

Stufen Sie dieses Dokument als nützlich ein?

Sind diese Inhalte unangemessen?

Beschreibung:

Copyright:

Verfügbare Formate

Deliverable D1.1 Report On Stakeholders Characterization and Traffic Characteristics

Originaltitel:

Hochgeladen von

Beschreibung:

Copyright:

Verfügbare Formate

Verwandte Titel

Seventh Framework STREP No. 317846 D1.

1 - Report on Stakeholder Relationships

Verwandte Interessen

Dokumente ähnlich wie Deliverable D1.1 Report on Stakeholders Characterization and Traffic Characteristics

Mehr von SmartenIT