Sie sind auf Seite 1von 44

This deliverable reports on the implementation, execution and evaluation of the

EXPERIMEDIA iCaCoT experiment that took place in the Schladming Venue. This
document provides EXPERIMEDIA software components developers with feedback on the
usage of their components, and the venue partner with an overview of experimental results.
D4.11.3
iCaCoT Experiment Results and Evaluation
2014-05-26

Lucia D'Acunto; Omar Niamut; Ray van Brandenburg; Bastiaan Wissingh;
Emmanuel Thomas; Arjen Veenhuizen (TNO)


www.experimedia.eu
EXPERIMEDIA Dissemination level: PU
Copyright TNO and other members of the EXPERIMEDIA consortium 2013-14 2

Project acronym EXPERIMEDIA
Full title Experiments in live social and networked media experiences
Grant agreement number 287966
Funding scheme Large-scale Integrating Project (IP)
Work programme topic Objective ICT-2011.1.6 Future Internet Research and Experimentation
(FIRE)
Project start date 2011-10-01
Project duration 36 months
Activity 4 Experimentation
Workpackage 4.11 EX11: interactive Camera-based Coaching and Training (iCaCoT)
Deliverable lead organisation TNO
Authors Lucia D'Acunto; Omar Niamut; Ray van Brandenburg; Bastiaan
Wissingh; Emmanuel Thomas; Arjen Veenhuizen (TNO)
Reviewers Gert Kienast (JOANNEUM RESEARCH - DIGITAL)
Version 1.0
Status Final
Dissemination level PU: Public
Due date PM32 (2014-05-31)
Delivery date 2014-05-26

EXPERIMEDIA Dissemination level: PU
Copyright TNO and other members of the EXPERIMEDIA consortium 2013-14 3

Table of Contents
1. Executive Summary ............................................................................................................................ 5
2. Introduction ........................................................................................................................................ 6
2.1. Background ................................................................................................................................ 7
2.2. Objectives ................................................................................................................................... 7
2.3. Experiment scenarios ............................................................................................................... 8
3. Experiment Preparation .................................................................................................................... 9
3.1. Participants and Venue ............................................................................................................. 9
3.2. App Functionalities ................................................................................................................. 11
3.3. Support for live streaming ..................................................................................................... 14
3.4. Integration with EXPERIMEDIA Baseline Components ............................................... 16
3.4.1. Integration with ECC ......................................................................................................... 16
3.4.2. Integration with AVCC and SCC ..................................................................................... 16
3.5. Questionnaire approach ......................................................................................................... 18
3.5.1. Purpose of the questionnaires and general strategy ....................................................... 18
3.5.2. Questions format ................................................................................................................ 18
3.5.3. Questionnaire implementation.......................................................................................... 19
3.6. Monitoring platform ............................................................................................................... 19
3.7. Ethics and Privacy ................................................................................................................... 20
4. Experiment Setup ............................................................................................................................. 22
4.1. General setup ........................................................................................................................... 22
4.2. Details from the first experiment run .................................................................................. 23
4.3. Details from the second experiment run ............................................................................. 23
5. Experiment execution ...................................................................................................................... 24
5.1. First experiment run ............................................................................................................... 24
5.2. Second experiment run .......................................................................................................... 26
6. Analysis of experiment results ........................................................................................................ 27
6.1. QoE analysis ............................................................................................................................ 27
6.1.1. Questionnaire results .......................................................................................................... 27
6.1.2. App feature usage ............................................................................................................... 29
6.1.3. User navigation .................................................................................................................... 31
6.2. QoS analysis ............................................................................................................................. 31
6.3. Feedback from venue partner ............................................................................................... 33
EXPERIMEDIA Dissemination level: PU
Copyright TNO and other members of the EXPERIMEDIA consortium 2013-14 4

6.3.1. User friendliness of the tested application ...................................................................... 33
6.3.2. Features and Functions of the tested application .......................................................... 34
6.3.3. Quality of the video stream and requirements for professional training .................... 35
7. Conclusions ....................................................................................................................................... 36
7.1. General ..................................................................................................................................... 36
7.2. Feedback on EXPERIMEDIA Baseline Components ..................................................... 36
7.3. Dissemination and Exploitation ........................................................................................... 37
7.4. Exploitation and follow-up .................................................................................................... 38
Appendix A. Questionnaire ............................................................................................................... 39
A.1. Questionnaire for the first experiment run ......................................................................... 39
A.2. Questionnaires for the second experiment run .................................................................. 40
A.2.1. Adapted questionnaire for the trainers ............................................................................ 40
A.2.2. Questionnaire for the visitors at the fun park ................................................................ 40
Appendix B. Consent form ............................................................................................................... 42


EXPERIMEDIA Dissemination level: PU
Copyright TNO and other members of the EXPERIMEDIA consortium 2013-14 5

1. Executive Summary
This deliverable reports on the implementation, execution and evaluation of the
EXPERIMEDIA experiment for interactive camera-based coaching and training (iCaCoT), which
focuses on interactive video navigation for camera-based coaching and training for visitors and
sports enthusiasts in Schladming. The goal of this experiment is to showcase the unique
properties of tiled adaptive streaming, a technology that was developed in the EU FP7 project
FascinatE and that allows users to navigate freely through high resolution video panoramas,
while the application reduces bandwidth requirements by only sending that part of the video a
user is interested in.
During this experiment, we have developed the iCaCoT application and have run two
experimental periods at the EXPERIMEDIA Schladming venue to test the feasibility of the
concept of interactive camera-based coaching and training. Through the experiment, the existing
EXPERIMEDIA technical components, notably the Experiment Content Component, have
been used and tested. The experiment has had impact by raising the attractiveness of Schladming
as a venue for smart sport tourism, by providing valuable input to MPEG DASH
standardization, by developing technology for large-scale tests by the BBC beyond Schladming
and EXPERIMEDIA, and by enhancing the potential for exploitation of tiled streaming
technology.


EXPERIMEDIA Dissemination level: PU
Copyright TNO and other members of the EXPERIMEDIA consortium 2013-14 6

2. Introduction
The present report documents the preparation, experimental setup and analysis of the
experiments conducted with the interactive Camera-based Coaching and Training (iCaCoT)
concept described in EXPERIMEDIA deliverable D4.11.1, which makes use of the tiled
adaptive streaming technology developed at TNO
1
. This technology, created within the context
of the FascinatE project
2
, allows users to interact with and navigate within a recorded video
using pan-tilt-zoom (PTZ) commands.

Figure 1: Interactive Camera-based Coaching and Training using tiled streaming for video navigation.
To apply the TNO tiled streaming technology within the context of the EXPERIMEDIA
project, i.e. as a tool for training and coaching and as a novel way for Schladming visitors to
record themselves coming down the mountain and sharing their experiences with friends and
family, we have built a number of additional components/features in the following areas:
support for live streaming
additional app functionalities
integration with existing EXPERIMEDIA components;
a monitoring platform that oversees the overall functioning of the iCaCoT components.
In addition, a questionnaire has been developed, with the aim of capturing user feedback in a
direct manner. A detailed description of the (new) architectural components, the integration with
the EXPERIMEDIA baseline components and the questionnaire approach is reported in
Section 3.

1
http://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=6081064
2
http://www.fascinate-project.eu/
EXPERIMEDIA Dissemination level: PU
Copyright TNO and other members of the EXPERIMEDIA consortium 2013-14 7

In the following sections we (i) provide a background of TNO's tiled streaming technology and
its applicability as a tool for coaching and training for skiers, and (ii) summarize the objectives of
the experiments.
2.1. Background

Figure 2: Impression of interaction with video on a tablet device.
Our hypothesis is that the concept of tiled adaptive streaming is particularly well suited to
training and coaching applications, since it offers a trainer using a smartphone or tablet, the
ability to zoom in on his student coming down the mountain, focusing on specific areas, both
temporally as well as spatially. For example, as shown in Figure 2, a football trainer could pause
and zoom in to the video, to highlight a specific situation. This is especially useful in snow
activities, where the exact line followed by the trainee is not known a-priori and can therefore
only be captured using a wide-angle lens located relatively far from the action. By pausing the
video at key moments, trainer and trainee can focus on and discuss details of the performance.
By placing multiple high resolution cameras around strategic positions, it is even possible for a
trainer to view a moment from different angles.
The iCaCoT experiments have been performed in the Schladming venue, a well-known ski
location in Austria, which has hosted the Alpine Skiing World Championship in February 2013.
In addition, Schladming was ideal for the fact that it is also a renowned ski training area, and the
venue partner is involved in supporting and facilitating such ski trainings. Hence, performing
experiments in Schladming has given us the required level of scale for this experiment, i.e. to test
how actual users use the concept of tiled adaptive streaming in a live and real-world
environment.
2.2. Objectives
The experiments have been conducted with the aim of fulfilling the following three main
objectives, as mentioned in D4.11.1:
1) Capture the user experience when interacting with the tiled streaming application:
EXPERIMEDIA Dissemination level: PU
Copyright TNO and other members of the EXPERIMEDIA consortium 2013-14 8

In particular, our focus has been on evaluating the user experience from a Quality of
Service (QoS) and a Quality of Experience (QoE) perspective, to allow for further
improvement of both the application and the underlying streaming technology.
2) Test the feasibility of tiled adaptive streaming as a tool for training and coaching activities:
Here the focus has been on investigating how trainers used the system, and whether the
tiled streaming concept is helpful in their instruction activities.
3) Leveraging information obtained through large-scale user tests to improve tiled adaptive streaming
bandwidth efficiency:
In order to reach this objective, we have collected a large amount of data concerning the
usage of the iCaCoT app and the associated video streams during all the experiments.
2.3. Experiment scenarios
For the experiments, two scenarios have been discussed with the Schladming venue.
One scenario focusses on young ski athletes, training for their participation in large and national
events. The Schladming venue partner, Schladming2030
3
had arranged for coaches / trainers to
participate in the experiment, along with their students, to use the app in their training approach
and to provide direct feedback to students. A second scenario focusses on visitors of a
Schladming fun park, interested in a mix of training and entertainment. Important features here
are the possibilities of social sharing and ease-of-use.
While both scenarios have been pursued during the preparation of the experiments, only the
training/coaching scenario has been fully tested during both experiment runs. This is because
when discussing the fun park / tourist scenario with visitors on the skiing slopes, they referred to
the popularity of GoPro helmet cameras, which provide the ability to film up close, resulting in
better quality footage than a statically mounted camera on the edge of the slope. In addition,
detailed analysis of skiing performance, which is the focus of the iCaCoT app, is of less interest
to recreational skiers.



3
http://www.schladming2030.at/
EXPERIMEDIA Dissemination level: PU
Copyright TNO and other members of the EXPERIMEDIA consortium 2013-14 9

3. Experiment Preparation
This section outlines the preparation necessary to perform the experiments.
3.1. Participants and Venue
The experiment has taken place in the Schladming region. Over a period of 2 months, we have
performed two experiment rounds. The first took place in week 8 (February 17-21) and the
second in week 13 (March 25-29) of 2014. During each round, 2-3 days of experiments took
place, with the remainder of the time spent on system set-up, testing and break-down. In
particular, the Reiteralm area (see a map in the figure below) was chosen as setup for the
experiments, because it is well-suited for (semi-)pro coaching and training purposes.


Figure 3: Reiteralm locations for the iCaCoT experiments.


EXPERIMEDIA Dissemination level: PU
Copyright TNO and other members of the EXPERIMEDIA consortium 2013-14 10

System set-up involved various challenging tasks, such as installing and connecting cameras over
distances of hundreds of metres on the skiing slope, and installing cables via underground
bunkers. Figure 4 and Figure 5 give some impressions of the system set-up effort.

Figure 4: Impressions from system setup on the slope. Installation (left) and debugging (right) in
underground bunkers.


Figure 5: Impressions from system setup on the slope. GoPro (left) and handheld (right) camera setups.

The experiment saw the involvement of the following actors:
End-users of the mobile application
EXPERIMEDIA Dissemination level: PU
Copyright TNO and other members of the EXPERIMEDIA consortium 2013-14 11

Persons who used the iCaCoT application during the two experimental periods are
providing the majority of QoS and QoE data. In particular:
o Trainers (2-5 people): persons that used the application in both the first and the
second experiment run and whose feedback has been used for adaptation of the
experiment and application in the second experiment run.
o Students (1-10 people): persons that participated in the experiments.
Local stakeholders
Local stakeholders in Schladming, primarily representatives of the companies operating
the cable cars who are in charge of the ski areas and local installations. They have
supported the installation and maintenance of the camera setups.
Experimenters
Experimenters have overviewed the execution and analysis of the experiments.
3.2. App Functionalities
To fit the purpose of the experiment, a number of functionalities have been added in the
implementation of the iCaCoT app. These functionalities can be broadly classified into two
categories:
GUI functionalities
o Draw;
o Bookmark a position in time (due to space limitation on the user interface, a
maximum of 6 bookmarks can be stored);
o Stepframe, or step forward, i.e. the ability to step through frames when the video
playback is paused; this function is used by the trainer to show the students his exact
moves, in order to explain what to improve;
o Seek, either via a slider or via dedicated buttons (+3, +15, -3, -15 seconds). Inside the
application, this translates to seeking one or multiple segments. This function serves
the purpose of finding the next student within the video playback, and is used by the
ski trainer after he has finished with the previous student and wants to move to give
feedback to the next student.
Measurement/logging functionalities
o Network monitor, i.e. bandwidth and data usage;
o Usage monitor, where a large set of metrics is captured, as described in
EXPERIMEDIA document D4.11.1 (e.g. navigation commands, bandwidth usage,
etc.);
o Debugging info logging, which the app sends to the TNO monitoring framework.
In addition, for the second run of experiments the following functionalities have been added:
GUI functionalities
EXPERIMEDIA Dissemination level: PU
Copyright TNO and other members of the EXPERIMEDIA consortium 2013-14 12

o Enhanced draw functionality (line-based, arrow-based and dot-based drawing, plus
the ability to choose different colours);
o More control over seeking functionality to make it easier to find students;
o Slow motion playback functionality (plays the video at 1/4 of the original speed).
Figure 8 - Figure 9 show several snapshots of the application interface.

Figure 6: iCaCoT application screenshot. This image shows the complete view from a single camera on
the slope.

EXPERIMEDIA Dissemination level: PU
Copyright TNO and other members of the EXPERIMEDIA consortium 2013-14 13


Figure 7: iCaCoT application screenshot. This image shows zoomed view from a single camera on the
slope.


Figure 8: iCaCoT application screenshot. After a trainer has zoomed into a specific region of interest and
has paused the video, he has used the drawing functionality to highlight critical training aspects, such as
posture and tracks.
EXPERIMEDIA Dissemination level: PU
Copyright TNO and other members of the EXPERIMEDIA consortium 2013-14 14


Figure 9: iCaCoT application screenshot. After a trainer has zoomed into a specific region of interest and
has paused the video, he has used the drawing functionality to highlight critical training aspects, such as
posture and tracks.
3.3. Support for live streaming
In order to process and render a video feed in real time, the following chain of components has
been developed:
Ingestion node, which captures the raw video data and encodes it using Motion JPEG
codec;
Processing node, which takes the input encoded in Motion JPEG received from the
Ingestion node and produces the final watchable video files; this step includes the tiling,
encoding (in H.264/AVC) and multiplexing (MPEG-TS container) of the content;
Segmenter, which produces the temporal segmentation of the content.
The output of the Segmenter is then distributed to the different instances of the iCaCoT app via a
webserver.
Both a high-level and low-level overview of the architecture used in the iCaCoT experiments are
shown in Figure 10 and Figure 11 below:

EXPERIMEDIA Dissemination level: PU
Copyright TNO and other members of the EXPERIMEDIA consortium 2013-14 15


Figure 10: High-level system architecture for the iCaCoT experiments within Experimedia. Green
indicates functional components as implemented during run 01. Orange indicates functional components
additionally implemented for run 02.


Figure 11: Low-level system architecture for the iCaCoT experiments within Experimedia.
EXPERIMEDIA Dissemination level: PU
Copyright TNO and other members of the EXPERIMEDIA consortium 2013-14 16

3.4. Integration with EXPERIMEDIA Baseline Components
The iCaCoT app makes use of existing EXPERIMEDIA Baseline components, namely the
Experiment Content Component (ECC), the Audiovisual Content Component (AVCC) and the
Social Content Component (SCC) components.
3.4.1. Integration with ECC
Within both experiment runs the iCaCoT app had to be integrated with the ECC component, in
order to log the measurement data captured by the app. The ECC is provided with APIs that can
be used by ECC clients in order to communicate about experiments and measurements while the
data is exchanged via a RabbitMQ message bus. These client APIs are available for Java,
Android, C# and C++ clients.
At the time of the iCaCoT experiments, the ECC did not have support for iOS clients, such as
the one built to collect measurement data from the iCaCoT app. Therefore, in order for the
iCaCoT app to use the functionalities offered by the ECC, the intermediary ECC-Proxy
developed by the National Technical University of Athens (NTUA) was used. The ECC Proxy
resides on a separate machine, where (in our architecture) also the ECC component is located
(see Figure 11). The ECC Proxy has been implemented as a Java ECC Client which on one side
communicates with the ECC via the available APIs of the ECC while on the other side provides
an HTTP Servlet which can be used by the iOS clients to communicate their experiment data via
HTTP POST messages. See Figure 12. The ECC Proxy stores the experiment data from the iOS
clients locally in a database and provides the ECC with the data from this database.

Figure 12: Integration of iCaCoT with ECC.
After the experiments were concluded, the measurement data collected by the ECC can be
accessed all at once via two CSV files (one with the metadata and the other containing the actual
measurement data).
3.4.2. Integration with AVCC and SCC
For the second experiment run, the iCaCoT app has been also integrated with the AVCC/SCC
components, to make use of the Facebook sharing functionality. To realize this, the backend of
the iCaCoT domain had to be interfaced with the AVCC component, in order to upload the
video to be shared. In turn, the AVCC component interfaces with the SCC component, which
shares the videos uploaded via the AVCC on an ad-hoc iCaCoT Facebook page (Figure 13).
EXPERIMEDIA Dissemination level: PU
Copyright TNO and other members of the EXPERIMEDIA consortium 2013-14 17


Figure 13: Integration of iCaCoT with AVCC/SCC.
The AVCC component is accessible as a webserver and offers the following interface:
Parameter Description Occurrence
title A string representing the title of the video file to share on Facebook. M
description A string representing the description of the video file to share on
Facebook.
M
category A string representing the category of the video file to share on
Facebook.
O
multiQuality A Boolean representing whether the video file has transcoding profile
'flash'
C
html5Mp4U A Boolean representing whether the video file has transcoding profile
'html5-mp4'
C
html5WebM A Boolean representing whether the video file has transcoding profile
'html5-webm'
C
html5OggM A Boolean representing whether the video file has transcoding profile
'html5-ogg'
C
html5Mp3audio A Boolean representing whether the video file has transcoding profile
'Mp3 audio'
C
userName A string representing the title of the video file to share on Facebook. M
path A string representing the url of the video file to share on Facebook. O

where "M" denotes a mandatory parameter, "O" an optional parameter and "C" a conditional
parameter, and at least one conditional parameter shall be present (i.e. and at least one
transcoding profile shall be selected).
For the iCaCoT experiment, we have used the mandatory parameters, the transcoding profile
'html5Mp4U' and the optional parameter 'path'.
Every time a new video is available for sharing on Facebook, a process running in the iCaCoT
backend sends a POST message to the AVCC webserver endpoint with the above mentioned
EXPERIMEDIA Dissemination level: PU
Copyright TNO and other members of the EXPERIMEDIA consortium 2013-14 18

parameters as part of the POST parameters. Upon receipt of the POST message, the AVCC
webserver responds with a message regarding the status of the request.
3.5. Questionnaire approach
In order to obtain user feedback in a more direct way, TNO has developed a number of
questionnaires meant to be presented to the trainers after their first and second experiment as
well as to the visitors of the fun park. Only the trainers have used the questionnaires to provide
feedback on application and experiment.
3.5.1. Purpose of the questionnaires and general strategy
Using questionnaires to assess the trainers impressions of the iCaCoT application will directly
contribute towards Objective 01 and Objective 02 of the experiment. In addition, combining the
information directly received from the user with the measurement data collected by the iCaCoT
app will enhance the fulfilment of Objective 03 as well.
To be effective, a questionnaire should manage to get the desired information without taking too
much of the participants time. For this reason, the questionnaire has been designed according to
the following principles:
Quick and simple assessment;
Comprehensive impression of the user experience.
3.5.2. Questions format
In order to satisfy principle 1, most of the questions have been formulated in the form of
multiple choices. To satisfy principle 2, the questions have been divided into the following
categories:
User satisfaction: measures users quality of experience by asking him/her direct questions
(e.g. whether his/her experience was good or bad)
GUI usability: measures whether the user can interact fluently with the app. This includes
two aspects:
o Ease of learning (e.g. can the user intuitively learn how to use the app)
o Ease of interaction, which measures whether the apps features have been
implemented in the correct way (e.g. seek forward being implemented as a cursor like
in YouTube)
Functional usability: weather the apps features work as they should (e.g. no major hiccups
within apps usage)
Application value: measures whether the user perceives that the app is useful for him/her

EXPERIMEDIA Dissemination level: PU
Copyright TNO and other members of the EXPERIMEDIA consortium 2013-14 19

3.5.3. Questionnaire implementation
For the purpose of the first experiment, a questionnaire with a total of 26 questions (of which 23
were multiple choice questions) has been developed. For the purpose of the second experiment,
two additional questionnaires where developed:
one directed to the trainers, with 22 questions;
the other directed to the visitors of the fun park, with 19 questions.
The full list of questions presented in each questionnaire is available in 7.4.Appendix A.
The questionnaire was presented to the trainers in the form of a separate app launchable from
the iPads. This app has been created by means of the QuickTapSurvey tool
4
, which allowed us to
deploy the questionnaire on multiple devices as well as collect and aggregate all responses on a
central server accessible via the web. It is important to notice that the questionnaire is completely
anonymous and that it does not use any participant's information. A screenshot of one of the
questions presented to the trainers is showed in Figure 14.

Figure 14: Screenshot from the questionnaire tool.
3.6. Monitoring platform
TNO has developed a light weight monitoring platform which receives real-time status
information from the components within the experiment platform like the iCaCoT app and
AvccConnector. This way, our experimenters were able to monitor in real time the situation
while trainers were using the app.
The different components in the iCaCoT platform can connect to the log server of the TNO
monitoring platform either via a web socket connection or a HTTP post (Figure 15). Finally, the
TNO monitoring platform includes a number of scripts to start up and coordinate all the
processes which are needed to enable the experiment to take place.

4
http://www.quicktapsurvey.com/
EXPERIMEDIA Dissemination level: PU
Copyright TNO and other members of the EXPERIMEDIA consortium 2013-14 20


Figure 15: The TNO monitoring platform.
3.7. Ethics and Privacy
As described in D4.11.1, with the iCaCoT experiment we aimed to adhere to the following
principles with respect to ethics and privacy:
Transparency through notification screen to explain nature and purpose of experiment;
Obtain explicit informed consent through 'I agree' button in the application;
Observational research in situations where those observed would expect to be observed;
Collected data will only be used for purposes of experiment and shared with participants.

During both experiment runs, these principles have been upheld in the following ways:

At the experiment venue, a notification sign was shown to inform all participants and
people on the slopes about the video recordings;
Informed consent by the experiment participants (trainers / coaches) was explicitly
obtained through signed consent forms. These forms also explained the nature and
purpose of the experiment;
Observational research has been conducted through video recordings;
The collected data is only used for the purpose of the experiments. Some specific
recordings have been made for demonstration purposes.
Note that the experiment involved minors, i.e. most of the students. Consent was asked via the
trainers who also monitored the process. The students did not need to download the application
and were not involved in any data measurements.
In relation to the formal requirements as listed in D4.11.1:
EXPERIMEDIA Dissemination level: PU
Copyright TNO and other members of the EXPERIMEDIA consortium 2013-14 21

Tabel 1: Overview of ethical and privacy-related requirements
Requirement in D4.11.1 Fulfilment of requirement
Data controller: TNO, possibly together with
Schladming and in a later stage also IT Innovation.
Each controller will notify its respective DPA.
TNO has notified the Dutch DPA, het College
Bescherming Persoonsgegevens
5
of the experiments.

Notification about the recordings that will be
provided with the ticket, as well as the consent
form for the application, should include all the
relevant information.
As discussed above. The consent form text is
attached in 7.4.Appendix B
Terms and Conditions of the service with the
limitations and purpose to inform the users. The
specification of the purpose is most likely broader
than just scientific research, should cover as well
dissemination and valorisation.
Clear policy on participation of minors: not
allowed unless consent is given by their parent or
guardian.
Areas where experiment will be conducted should
be selected in a way that individuals who do not
want to be recorded could still use the facilities.
Also, areas should be clearly indicated so no
mistakes or incidental entries are made.
The Reiteralm allowed for using a specific slope,
parallel to regular facilities.

Point of contact on the site should be
communicated to the participants in case of
questions / concerns / objections. The points of
contact at the later stage should be indicated in the
information provided and should include a TNO
and Schladming2030 contact person.
Both TNO and Schladming2030 contact persons
were present during both experiment runs. Contact
data for communication after the experiments have
been provided through the consent forms.

Language of the communication will be English
and written German.

The notifications and consent forms were available
in English and German. See Figure 16 for the
German version of the notification sign.


Figure 16: Notification sign in German, as shown on the experiment locations.

5
https://www.cbpweb.nl/
EXPERIMEDIA Dissemination level: PU
Copyright TNO and other members of the EXPERIMEDIA consortium 2013-14 22

4. Experiment Setup
This section documents the setup that was necessary in order to conduct the iCaCoT
experiments.
4.1. General setup
Both experiments took place in the Reiteralm location in Schladming, Austria (Figure 3, in
Section 3). During both experiment runs, a team of TNO experimenters was present in the
venue to setup the infrastructure, monitor the experiments and assist the experiment
participants.
A high level view of the necessary infrastructure is reported in Figure 17.

Figure 17: Infrastructure to support the experiments.
Furthermore, two bunkers, located on the slopes, were used for equipment storage and to host
switches for the cables.
EXPERIMEDIA Dissemination level: PU
Copyright TNO and other members of the EXPERIMEDIA consortium 2013-14 23


Figure 18: Control room where the ingestion node, the processing node, the segmentation server, the web
server, the ECC node and the monitoring framework were installed.
4.2. Details from the first experiment run
The first experiment period took place in week 8 of 2014. Together with the venue partner, the
team spent the first 2 days setting up the necessary infrastructure to support the architecture in
Figure 17. Furthermore, a smoke test was performed to check that the complete ecosystem
worked. Finally, the selection of the participants was carried out.
GoPro Hero 3+ cameras
6
where installed on the Reiteralm slope, with 520m UTP and HD-SDI
cables running along the slopes all the way to the wooden control room (Figure 18). During the
experiment, one GoPro was changed for a JVC Handycam which yielded a better image quality
due to better optics.
4.3. Details from the second experiment run
The second experimental run was executed in week 13 of 2014.Based on the experience from the
first run, the team spent only half a day for experimental setup. The experimental setup was
similar to the first run, except that this time one GoPro was replaced by a Blackmagic Design 4K
camera
7
.

6
http://gopro.com/cameras/hd-hero3-black-edition
7
http://www.blackmagicdesign.com/products/blackmagicproductioncamera4k
EXPERIMEDIA Dissemination level: PU
Copyright TNO and other members of the EXPERIMEDIA consortium 2013-14 24

5. Experiment execution
Each experiment period was accompanied by a "day-to-day script", which aided the
experimenters to ensure an efficient and correct execution of the experiments. Relevant details
of the execution for both experiments are reported in the following paragraphs.
5.1. First experiment run
After having introduced the experiment to the participants in the ski school on the morning of
the third day, the first experiment was performed in the afternoon of the same day. During this
day, two trainers took part in the experiment, with a group of 7-8 students in total. On the next
day, another experiment was performed with two trainers, each having a group of 7-8 students
(Figure 19, 3 examples).
EXPERIMEDIA Dissemination level: PU
Copyright TNO and other members of the EXPERIMEDIA consortium 2013-14 25



EXPERIMEDIA Dissemination level: PU
Copyright TNO and other members of the EXPERIMEDIA consortium 2013-14 26


Figure 19: Ski trainers using the iCaCoT app with their students.
5.2. Second experiment run
The second experiment run was performed within three days. In addition to the ski trainers, this
time also a snowboarder trainer with his student participated in the morning of the first day.
Other four sessions have been performed in the subsequent days, with the ski trainers and
groups of students of up to 10 individuals until Saturday morning.
EXPERIMEDIA Dissemination level: PU
Copyright TNO and other members of the EXPERIMEDIA consortium 2013-14 27

6. Analysis of experiment results
As discussed in EXPERIMEDIA deliverable D4.11.1, our aim for the iCaCoT experiments is to
evaluate the user experience from a Quality of Service (QoS) and a Quality of Experience (QoE)
perspective, to allow for further improvement of both the application and the underlying
streaming technology. To do so, we have captured the following data throughout the
experiments:
QoE
o Direct measurements of user satisfaction through questionnaires;
o Application feature usage;
o User's navigation information within the App.
QoS
o Bandwidth of received streams;
o Interruptions in stream reception;
o Switching delay between resolution layers;
o Networking conditions.
In the section that follows, we report a detailed analysis of the above mentioned data, for both
experiment runs.
6.1. QoE analysis
6.1.1. Questionnaire results
In general, questionnaire results show that the participants had a very good experience with the
iCaCot app, both in the first and second run (75% of the respondents in the first run rate their
experience as good or very good, and in the second run this is 100% of the respondents)
and all of them would recommend the app to others (100% of the respondents in both
experimental runs).

Figure 20: participants' rating of the iCaCoT app in both experimental runs
50%
25%
25%
Experimental Run 1
33%
67%
Experimental Run 2
Very Good
Good
Ok
EXPERIMEDIA Dissemination level: PU
Copyright TNO and other members of the EXPERIMEDIA consortium 2013-14 28

Furthermore, the questionnaire results show that the iCaCoT app performs well against the
expectations of the participants, and this feeling did even increase in the second run. While in the
first experimental run the iCaCoT app exceeded the expectations of 50% of the participants, in
the second run it exceeded the expectations of all participants, as shown in Figure 21.

Figure 21: iCaCoT perceived performance versus participants' expectations.
Moreover, the questionnaire results show the increase in perceived video quality between the 1st
and 2nd experiment run. As shown in Figure 22, all participants perceived the video quality at
least as not good or bad in the 2nd run, whereas the video quality was perceived bad by some
users in the 1st run.

Figure 22: iCaCoT perceived video quality.
0%
10%
20%
30%
40%
50%
60%
70%
80%
Moderately fails
expectations
Meets
expectations
Moderately
exceeds
expectations
Completely
exceeds
expectations
iCaCoT performance vs. expectations
Experimental Run 1
Experimental Run 2
0%
10%
20%
30%
40%
50%
60%
70%
80%
Bad Not good but
not bad
Good
iCaCoT Video Quality
Experimental Run 1
Experimental Run 2
EXPERIMEDIA Dissemination level: PU
Copyright TNO and other members of the EXPERIMEDIA consortium 2013-14 29

In general, the participants were less satisfied with the video quality, especially during the first
run. This can be explained by the type of camera that was used (GoPros). When 4K cameras
where used in the second run, the participants did notice an improvement of the video quality
and considered it overall acceptable.
Furthermore, the participants were relatively satisfied with the set of functionalities offered by
the iCaCoT app (especially in the second run where enhanced drawing functionalities and the
slow motion playback functionality was added). As potential for improvement, they
recommend to focus on more graphical aspects, such as providing the user with the ability to
easily find a student in the video playback and show two students in parallel for comparison.
Finally, all the participants from both runs considered the iCaCoT app very useful as a means for
training and coaching, and consider it beneficial for both students and ski trainers.


Figure 23: perceived usefulness of the iCaCoT app as a means for coaching and training
6.1.2. App feature usage
The following two diagrams report on the usage of the iCaCoT app features.
50% 50%
Perceived Increase in
Students' Learning Curve
Moderately
Very Much
0%
50%
100%
150%
Perceived Usefulness of the
iCaCoT App
Beneficial for
both the
students and
the ski trainers
EXPERIMEDIA Dissemination level: PU
Copyright TNO and other members of the EXPERIMEDIA consortium 2013-14 30



Figure 24: iCaCoT app feature usage during the 1st experiment run.

Figure 25: iCaCoT app feature usage during the 2nd experiment run.
As we can observe from both Figure 24 and Figure 25, the most popular functions in both runs
are "step forwards" and "seek the next segment". As described in section 3.2, the first function is
used by the trainer to show the students his exact moves, in order to explain what to improve.
The second function serves the purpose of finding the next student within the video playback,
and is used by the ski trainer after he has finished with the previous student and wants to move
to give feedback to the next student.
seek_slider
1% paused
6%
seek_next_segment
30%
seek_prev_segment
9%
resumed
5%
bookmark_added
2%
seek_multiple_segme
nts_backward
2%
step_forward
24%
seek_multiple_segme
nts_forward
4%
seek_to_live
1%
slow_motion_enabled
7%
slow_motion_disabled
4%
draw_enabled
2% draw_disabled
2%
camera_change
2%
bookmark_selected
0%
Experimental Run 1
paused
4%
camera_change
1%
draw_enabled
1%
draw_disabled
1%
slow_motion_enable
d
3%
slow_motion_disable
d
2%
step_forward
43%
resumed
3%
seek_prev_segment
6%
bookmark_added
0%
seek_slider
4%
bookmark_selected
0%
seek_to_live
1%
seek_multiple_segme
nts_forward
10%
seek_multiple_segme
nts_backward
4%
seek_next_segment
17%
Experimental Run 2
EXPERIMEDIA Dissemination level: PU
Copyright TNO and other members of the EXPERIMEDIA consortium 2013-14 31

Furthermore, we can observe that the bookmark function was almost never used. This is due to
the fact that the trainers where usually not watching the students from the app in real time, but
rather playing it back every time a new student came down the slope.
Similarly, camera changes did not appear often. This might be due to the fact that the cameras
were well positioned to get a good overview of the slope (and therefore switching between them
was not needed), or that the participants did not know or get used to switch between cameras.
As remarked by one of the trainers, "one of the most useful functions to illustrate the athletes
current position along the course of motion. It gives you a hint of the origin of a potential
mistake in an athletes motion sequence. The strong preference for such trick play functionality
has influenced the additional feature designs for the 2nd experimental run.
6.1.3. User navigation
We have also captured the user navigation within the iCaCoT app, in order to assess areas of
particular interests. Figure 26below shows a heatmap of the most watched areas within a video
for the session with the snowboarder (on the first day of the second run):

Figure 26: Heatmap of the most watched areas
This information can be used in multiple ways. For example, the TNO iCaCoT tiling module can
take this information into account to optimize the tiling for this video. Furthermore, information
about areas of interests such as this can aid the choice for the positioning of the cameras during
the event.
6.2. QoS analysis
In this section, we analyse the results obtained from the measurement data. Specifically, we
looked at the bit rate, data usage and dropped frames. Figure 27 shows the measured data over
one session of about 11 minutes. As we can observe, the iCaCoT app is very efficient in the data
transmitted, and only 8 frames have been skipped in total, which can explain why the iCaCoT
app exceeded the expectations of the participants.
EXPERIMEDIA Dissemination level: PU
Copyright TNO and other members of the EXPERIMEDIA consortium 2013-14 32

Furthermore, we see a correlation between the instant when a frame was dropped and the usage
of the app from the participant: usually a frame drop happens when a video is resumed from
pause. We can also observe that the highest and most frequent peaks in bandwidth usage happen
around frequent switching between pause and resume. This is valuable information that can aid
the further improvement of the iCaCoT app.


Figure 27: Measurement data from first experimental run.

EXPERIMEDIA Dissemination level: PU
Copyright TNO and other members of the EXPERIMEDIA consortium 2013-14 33


Figure 28: Measurement data from second experimental run.

6.3. Feedback from venue partner
In addition to the QoS and QoE analysis results as presented in the previous two sections, the
venue partner has provided post-experiment feedback. In their view, the outcomes of the
experiments have definitely shown that with the iCaCoT trainings application, athletes and
students have a number of several advantages. Generally spoken, a video feedback between the
training runs allows an athlete an immediate spot view of his or her movement and makes the
coaches advices more understandable. Not only professional athletes benefit from such
applications but also semi-professional ski and snowboard athletes as well as talented young
skiers and boarders. The two experiments have already shown that not only the athletes benefit
from the application but also the quality of the training. The following paragraphs summarize the
pros and cons of the application and give some suggestions for the further development of
iCaCoT from the coaches point of view.
6.3.1. User friendliness of the tested application
After a short time of testing the application, the coaches got used to it and were able to focus on
what they wanted to explain their athletes. Most of them mentioned during interacting with
iCacoT that the timeline as well as the application menu should be located at the bottom. The
reason for this is, that the view (of the athlete and trainer) is interfered by the acting hand.
Another problem that has already been discussed and partly solved is the searching function via
EXPERIMEDIA Dissemination level: PU
Copyright TNO and other members of the EXPERIMEDIA consortium 2013-14 34

the time line. First of all, the bookmark function makes sense for a small training-group (3-6
athletes) that stays frequently together during the runs and analysis. Searching an athlete in the
timeline has proven to be time intensive.
6.3.2. Features and Functions of the tested application
Frame to frame function
This is one of the most useful functions in order to illustrate the athletes current position along
the course of motion. Each frame may give you a hint of the origin of a potential mistake in an
athletes motion sequence.
Slow motion function
This function is necessary to explain the results of mistakes in a clear manner. Due to the
decelerated speed of the course of motion, the athlete is able to see and understand the
outcomes of his performance. In the two experiments at the Reiteralm location, a 1:4 slow-
motion was used. Some coaches suggested implementing at least 3 more possibilities of the slow-
motion function namely 1:2, 1:6 and 1:8 standards. Some of the coaches would even favour a
backwards slow-motion function, though difficult to implement.
Fast Forward, Fast Backward (3 seconds, 15 seconds) and live button
This feature was often used as a searching function in order to find a certain athlete. Even
though it took some time to capture the right student, it fulfilled the intended functionality. The
Live button has frequently been used as a kind of searching function and was a good indication
for the coaches time-orientation.
Drawing functions
Generally the offered functions (freedom drawing, arrows, straight lines, dots) satisfied the
expectations and the required visual assistance. However, implementing moving visualisation
tools like moving arrows and automatically estimated angles of two crossing lines would set new
standards of analysis-tools.
Additional desired features and functions
Thinking about a permanent training track which combines the interactive camera based
coaching and training application and electronic timing, an interface should be found. A
combination of visual feedback and integrated electronic time-comparisons (athlete to athlete or
individual time to individual time) would create a product which does not exist yet (high
attractiveness). Another desired function is a face to face possibility, where either two different
athletes or two different runs of one athlete can be compared at one sight.
Storage & Export function
Exporting the video-stream, sharing it through social media or store it in an individual cloud
could be considered as a basic function. The store and export function worked just from the
capture system but unfortunately not directly from the application. The export function should
EXPERIMEDIA Dissemination level: PU
Copyright TNO and other members of the EXPERIMEDIA consortium 2013-14 35

give training groups the possibility to handle the video analysis more flexible. In addition it could
also be helpful to provide access to the video stream if coaches are not able to accompany the
training (personal coaches, individual training without coaches, etc.).
6.3.3. Quality of the video stream and requirements for professional training
During the first experiment, the quality of the streamed video (recorded with GoPros HD) has
been proved as insufficient. This resulted likely from the GoPro lenses with very wide angles.
However, changing the type of camera did not lead to a sufficient solution. In the second
experimental run, a 4K camera was used which provided a useful quality and could be seen as a
minimum standard for a further development of a fixed installed service application. In this case,
coaches would accept some quality resolution-distortion, if the video stream is guaranteed in 4K
quality. Also with a 4K camera, a really high resolution is hardly possible to generate. This
circumstance could be solved, if individual cameras of the users were connectable with the
iCaCoT program. In this case two zooming functions were available, one offered directly by the
individual camera (manual zooming) and one offered by the application.
EXPERIMEDIA Dissemination level: PU
Copyright TNO and other members of the EXPERIMEDIA consortium 2013-14 36

7. Conclusions
7.1. General
In general, we find that the iCaCoT application could reduce the amount of training runs in
order to achieve a certain training objective. Students get additional information and an external
view of their performance and it is easier to determine the origins of faults. A trainer can focus
his or her attention on corrections and can report on direct visible progress of the athletes. While
a trainer might not need video feedback after every single run, one could use it as additional
support for coaching.
A significant advantage of the tiled streaming-based training and coaching application is that is
separates the recording aspect from the interactive training aspect. Previously, trainers would
have focus on recording during training sessions, and would be able to give feedback only after
analysis of the recorded training sessions. With the iCaCoT application, trainers can focus on the
training itself and can give faster feedback.
When considering the initial objectives of the experiment:
1) Capture the user experience when interacting with the tiled streaming application:
Our evaluation of the user experience from a Quality of Service (QoS) and a Quality of
Experience (QoE) perspective has allowed us to further improve both the iCaCot
application and the underlying streaming technology.
2) Test the feasibility of tiled adaptive streaming as a tool for training and coaching activities:
Our investigation of how trainers used the system showed us that the tiled streaming
concept is helpful in their instruction activities.
3) Leveraging information obtained through large-scale user tests to improve tiled adaptive streaming
bandwidth efficiency:
Our collection and analysis of a large amount of data concerning the usage of the
iCaCoT app and the associated video streams during all the experiments has enabled us
to identify aspects to improve and modify.
7.2. Feedback on EXPERIMEDIA Baseline Components
With the iCaCot experiment and application, we have focussed on using the ECC as a metric
logging system. While the ECC has enabled us to capture large datasets for analysis, we
encountered several problems in our usage of the ECC component and proxy. We provide his
feedback to the core component developers so they can further improve these components.
Performance issues with the ECC Proxy
For our experiments different metrics have been defined which relate to both the Quality of
Service as well as the Quality of Experience of our iCaCoT application. One of these metrics was
called CurrentVideoArea which relates to the part of a video frame that is currently displayed
on screen. As this metric is generated for each frame of the video stream, the metric is generated
25 times per second for each iOS client that is using the iCaCoT application resulting in a lot of
EXPERIMEDIA Dissemination level: PU
Copyright TNO and other members of the EXPERIMEDIA consortium 2013-14 37

data that needs to be processed by the ECC Proxy. Due to this high amount of metric data, a lot
of effort was needed in order to optimize the ECC Proxy to handle this amount of data.
Stability of the ECC and ECC Proxy
During our development and experimentation phases we noticed that the ECC and ECC Proxy
where not always as stable as expected. Some times for example, the ECC Proxy was not
detected by the ECC which results in the fact that no measurements are stored within the ECC
database. Even though the ECC was started prior to the ECC Proxy. While this issue has been
mostly resolved thanks to the baseline component developers, the operation during experiments
is still not as stable as one would like.
Post reporting with the ECC
Another problem we encountered during our experiments is the fact that with the current
version of the ECC, the post reporting functionality is rather limited. For example you are only
able to export all metric-data and measurement-data in one CSV file, which should then be
processed manually. With a few million measurements this is not the ideal way to go. Also when
the unfortunate situation arose where the ECC crashed, we were no longer able to access our
metric data via the ECC.
Using the AVCC and SCC through the Social Annotation service
We have not been able to rigorously test the combined AVCC and SCC functionality during our
experiments. From our work on integration of the components, our impressions are that an
operational combination of these components requires in-experiment control and guidance by
the baseline component developers. Therefore, it would be beneficial to create a ready-to-deploy
version of this combination of components.
7.3. Dissemination and Exploitation
Intermediate and final results of the iCaCoT experiment have been disseminated via several
channels:
Several Austrian online news sites, including the largest online tabloid site
o News.at - http://www.news.at/a/video-unterstuetztes-ski-training
o Sciene.apa.at - https://science.apa.at/rubrik/natur_und_technik/Innovative_Ski-
Trainings-App_im_Test/SCI_20140227_SCI39391351417222740
o Meine Woche - http://www.meinbezirk.at/bad-mitterndorf/wirtschaft/innovation-
auf-der-reiteralm-d858501.html
A contribution to the 108th MPEG meeting. This meeting was held in week 14 of 2014
(a week after the second experiment run) in Valencia, Spain. We presented preliminary
results of the iCaCoT experiments within the context of EXPERIMEDIA, in support of
enabling tiled streaming support in the MPEG DASH standard
8
.


8
http://mpeg.chiariglione.org/standards/mpeg-dash
EXPERIMEDIA Dissemination level: PU
Copyright TNO and other members of the EXPERIMEDIA consortium 2013-14 38

A presentation at Immovator Cross Media Caf: Sports and New Media
9
. Immovator is
the Dutch SME cluster for ICT and media.
An abstract for a paper presentation at IBC2014, Europe's largest conference on
broadcast media. The paper will discuss tiled streaming and will highlight some of the
iCaCoT results within EXPERIMEDIA.
7.4. Exploitation and follow-up
The exploitation of tiled streaming technology by TNO aims at licensing the technology to a 3rd
party. In discussions with interested parties, the scalable live streaming feature has been
highlighted as a significant advantage of the technology and system, and progress in the area was
much sought after. With the iCaCoT experiment and the support of EXPERIMEDIA, TNO has
been able to develop such a live system and application, which greatly support ongoing
discussions with potential licensing partners.
In addition to the exploitation of the generic tiled streaming technology, the specific iCaCoT
application, developed for testing and training, has generated interest from parties in both
Austria and The Netherlands. Ongoing joint exploration of the exploitation potential by TNO
and the Schladming venue partners may lead to integration of the system and application at ski
slopes at the Schladming venue. The venue partner has identified two specific product-market
combinations of interest:
Flexible version: Wifi camcorder stream to interact via the iCaCot application. Could
serve as a low-end product for coaching and training. Target groups include sport
associations with the main focus on: Alpine skiing, Freestyle Skiing, Ski jumping and
snowboarding.
Fixed version: it is clear which infrastructure is needed to run the iCaCot application on a
potential location. The high-end version would attract a number of teams interested in a
professional training infrastructure. Target groups include cable-car companies that (may)
offer permanent training tracks for snow-sports accessible through application-stores
Direct follow-up of the iCaCoT experiment is pursued by TNO through collaboration with BBC
R&D towards the Commonwealth Games 2014 in Glasgow. We aim to produce live 4K footage
and deliver this via tiled streaming to end-users in large-scale trial. This allows to determine the
impact of live 4K tiled streaming on regular production environment and to measure the effects
of tiled streaming on bandwidth and latency in a regular content delivery setting. The use of the
EXPERIMEDIA ECC baseline component in this BBC-TNO trial is being explored.

9
http://www.immovator.nl/agenda/cross-media-cafe-sport-en-nieuwe-media
EXPERIMEDIA Dissemination level: PU
Copyright TNO and other members of the EXPERIMEDIA consortium 2013-14 39

Appendix A. Questionnaire
A.1. Questionnaire for the first experiment run
For the first experiment run, a questionnaire directed to the ski trainers was developed. The
questionnaire requests the trainers to give feedback on the aspects reported below. These aspects
are classified according to the categories identified in Section 3.5.
User satisfaction
1 Bad/good
2 Fails expectations/exceeds expectations
3 Annoying/enjoyable
4 Boring/exciting
5 Would not recommend to others/would recommend to others
GUI usability
Ease of learning
6 Difficult to learn / easy to learn
7 Difficult to understand / easy to understand
Ease of Interaction
8 Unfriendly/friendly
9 Difficult to use/easy to use
10 Unpredictable/predictable
Functional usability
11 Limited functionality/comprehensive functionality
12 Could not follow the student on the piste/could follow the student on the piste
13 Bad view over the piste/good view over the piste
14 Unsmooth video navigation /smooth video navigation
15 Bad video quality/good video quality
16 Video start up time too high/ video startup time just right
17 Live video feed latency too long/ live video feed latency just right
18 Video feed interrupted too frequently/video feed never interrupted
Application value
19 Useless/useful
20 Old idea/innovative
21 Weakens trainers teaching ability/enhances trainers teaching ability
22 Decreases students learning curve/increases students learning curve
23 Increase teaching time/decreases teaching time
Open question(s)
24 What feature is most useful to you?
25 What would you change in the iCaCoT app?
26 What would you add to the iCaCoT app?


EXPERIMEDIA Dissemination level: PU
Copyright TNO and other members of the EXPERIMEDIA consortium 2013-14 40

A.2. Questionnaires for the second experiment run
For the second experiment run, two questionnaires were developed: one for the trainers and one
for the fun park visitors.
A.2.1. Adapted questionnaire for the trainers
The questionnaire for the trainers was an adapted version of the first questionnaire. Based on the
feedback from the first run, we could determine that a number of questions were less relevant
for the trainers (such as the ones on the enjoyability or friendliness of the app) or for the second
run (e.g. the ease of learning and interaction, since the trainers were already used to the app).
Therefore these questions were removed in the new questionnaire. Furthermore, a few new
questions have been added in order to assess the impact of changes/improvements done to the
iCaCoT app between the first and second run. These are highlighted in green below.
User satisfaction
1 Bad/good
2 Fails expectations/exceeds expectations
3 Would not recommend to others/would recommend to others
GUI usability
Ease of learning
4 Difficult to learn / easy to learn
Ease of Interaction
5 Difficult to use/easy to use
Functional usability
6 Limited functionality/comprehensive functionality
7 Functionality improved/worsened compared to the previous run
8 Could not follow the student on the piste/could follow the student on the piste
9 Could easily/hardly find a student when playing back the video
10 Unsmooth video navigation /smooth video navigation
11 Bad video quality/good video quality
12 Overall video quality decreased/increased compared to previous run
13 Live video feed latency too long/ live video feed latency just right
14 Video feed interrupted too frequently/video feed never interrupted
Application value
15 Useless/useful
16 Old idea/innovative
17 Beneficial/disadvantageous for the students and for the trainers
18 Beneficial/disadvantageous for the trainers
Open question(s)
19 What feature is most useful to you?
20 What would you add to the iCaCoT app?
21 What would you remove from the iCaCoT app?
22 How could the iCaCoT app make it easier to find a student skiing down the mountain?
A.2.2. Questionnaire for the visitors at the fun park
For the experiment with the visitors at the fun park, it is important to verify whether they find
the application exciting and usable. Hence, we have retained those questions from the original
EXPERIMEDIA Dissemination level: PU
Copyright TNO and other members of the EXPERIMEDIA consortium 2013-14 41

questionnaire posed to the trainers. However, we eliminated all questions related to coaching and
training. The resulting questionnaire is shown below.
User satisfaction
1 Bad/good
2 Fails expectations/exceeds expectations
3 Boring/exciting
4 Would not recommend to others/would recommend to others
GUI usability
Ease of learning
5 Difficult to learn / easy to learn
Ease of Interaction
6 Unfriendly/friendly
7 Difficult to use/easy to use
8 Unpredictable/predictable
Functional usability
9 Limited functionality/comprehensive functionality
10 Could not follow the participant on the piste/could follow the student on the piste
11 Unsmooth video navigation /smooth video navigation
12 Bad video quality/good video quality
13 Live video feed latency too long/ live video feed latency just right
14 Video feed interrupted too frequently/video feed never interrupted
Application value
15 Useless/useful
16 Old idea/innovative
Open question(s)
17 What feature is most useful to you?
18 What would you add to the iCaCoT app?
19 What would you remove from the iCaCoT app?


EXPERIMEDIA Dissemination level: PU
Copyright TNO and other members of the EXPERIMEDIA consortium 2013-14 42

Appendix B. Consent form
Consent Form
Today, we are going to test the "iCaCoT" application, an interactive camera-based coaching and
training application. This test takes place on the Reiteralm slope, in Schladming. The test is
organized by Dutch research institute TNO, in collaboration with Schladming2030.
Purpose for which the data will be processed
Your data will be collected during the iCaCoT Experiment and processed for scientific purposes.
In particular, the iCaCoT Experiment aims at testing of an interactive video streaming
technology, and evaluating its use in the context of training and coaching of ski students.
iCaCoT is part of the European funded (FP7) research project EXPERIMEDIA (Grant
Agreement No 287966; http://www.experimedia.eu).
Organization responsible for the Experiment (controller)
TNO
Brassersplein 2
2612 CT Delft
The Netherlands
Contact Persons
Dr. O.A. (Omar) Niamut
Sr. Research Scientist
Brassersplein 2
2612 CT Delft, NL
T: +31 (0)88 866 72 18
M: +31 (0)65 191 62 42
E: omar.niamut@tno.nl

Lukas Seyfried

Pfarrgasse 1/1
8970 Schladming, AT
T: +3687/22030
M:+31660/34 61 625
E: office@schladming2030.at

Collected Data
TNO guarantees the confidential usage of all personal data in respect to Dutch Data Protection
Law as well as EU Directive 95/46/EC on the protection of persons with regard to the
processing of personal data. TNO is responsible for the organization of the Experiment and will
be collecting the following personal data during the Experiment:
Your Name
Your Gender
Your Age
Your Telephone number
Your Email address
In addition, survey and test data will be collected. Survey and test data refers to the following
categories of personal data:
EXPERIMEDIA Dissemination level: PU
Copyright TNO and other members of the EXPERIMEDIA consortium 2013-14 43

The usage data of the app;
Audio recording of the usage;
Camera recording of the usage.
Personal data (phone number, e-mail address) will not be stored in combination with the survey
and test data. Personal data is only used for administrative purposes (such as contacting you
during the study period) and is not shared outside the company. The personal data of the study
will be stored and then deleted after the completion of the research purpose of this Experiment.
All test data (logging, audio recording, camera, survey data) is anonymized and used solely for
analysis in the study. Results of the analysis and interpretation of data (report of results,
publications) will be anonymized.
Security measures
Appropriate security policies, rules and technical measures are implemented to protect your
personal data from unauthorised access.
Right to access your data
You have, at any moment, the right to request modification, deletion or a complete excerpt of
your data. TNO is considered as data controller in the context of protecting your data and you
can address us by any means at your disposal such that your request can be processed. You can
exercise your rights by contacting Dr. Omar Niamut, see contact persons.
Recipients of the Data
Personal data collected for the iCaCoT Experiment may be forwarded to the coordinator of
EXPERIMEDIA project for further processing. The coordinator is the University of
Southampton IT Innovation Centre (IT Innovation), an organization based in Southampton,
UK. The purpose of such further processing is scientific research necessary for completion of
the EXPERIMEDIA projects objectives, in particular developing policy recommendations for
Future Media Internet as well as identifying the features, risks and opportunities of the Future
Media Internet.
Whenever possible, the data will be anonymized before such forwarding. Your data will not be
used for any other purposes than those listed above. Your data will not be used for direct
marketing purposes. Your data will be deleted once the research purpose has been completed.
IT Innovation constitutes a separate data controller compliant with the UK Data Protection law.
You can access your data and exercise your data subject rights by contacting Mr. Michael
Boniface at IT Innovation at the following address:
IT Innovation Centre
Gamma House
Enterprise Road,
Southampton SO16 7NS, UK
Email: mjb@it-innovation.soton.ac.uk
EXPERIMEDIA Dissemination level: PU
Copyright TNO and other members of the EXPERIMEDIA consortium 2013-14 44

Tel.: +44 23 8059 8866
Free, Specific, and Informed consent
I understand and agree by signing below that the above described categories of personal data will
be processed as outlined in this form. I am informed and take note that I have access to my
personal data and the right of correction. I can also, at all times, request the deletion of my data
by either TNO and/ or IT Innovation Centre at the aforementioned addresses.
Liability Disclaimer
I hereby confirm that I shall refrain from any claims for damage or liability against TNO in
relation to the usage of iCaCoT application and the participation in the experiment. I am aware
that I put myself in an unfamiliar situation using the application and/or participating in the
iCaCoT experiment before, while, and after skiing, and that sport-specific and general personal
injuries cannot be excluded. I declare that I am willing to go skiing on all routes on own risk. In
case of any damage in connection with the experimental equipment, I will not blame TNO, as far
as the damage is not linked to a gross negligence of an employee of TNO. I further declare, that
I dont suffer of any health impairments during participation in the iCaCoT experiment and/or
using the iCaCoT application. use of the Oakley Airwave ski goggles that may impair my ability
to drive, or am under the influence of alcohol, drugs or strong medication that may affect the
sensory skills and perceptiveness.

I have read the above points and I agree.
Schladming, the ...... ........................ 2014
...........................................................................................
Signature test person + Name in block letters

Das könnte Ihnen auch gefallen