PhD Study Plan

Converged Mobile Media: Evaluation of an Interactive

User Experience

Alexandre Fleury

April 2009 - March 2012

 1 Summary
Keywords
Mobile media, Interactivity, Convergence, Evaluation, User experience, Usability

Primary ACM classification

H.5.1 [Information Interfaces and Representation (e.g., HCI)]: Multimedia Informa-
tion Systems–Evaluation/methodology
H.5.2 [Information Interfaces and Representation (e.g., HCI)]: User Interfaces–
Evaluation/methodology, Prototyping, User-centered design

Abstract
This thesis project explores the concepts of convergence and mobility and their impact
on the end user experience with mobile media. Additionally, the thesis’ third focus area
concerns the evaluation of these concepts with end users and the investigation of testing-
related issues. Finally, it is intended to investigate the phenomenon of user-generated content
and its impact on the user experience with mobile media, through collaborative work with
an educational institution located outside Denmark.
First, the study of convergence will cover the use of mobile devices in complement to other
media related devices, such as televisions. Issues related to the transfer of content between
devices of varying type will be investigated conceptually and from a usability perspective.
The study will then focus on the concept of augmented remote control, where the mobile
phone is used as a support to not only traditional TV functions but also to features related
to interactive shows and social networks.
The thesis’ second area of interest is related to the mobile context. The work will espe-
cially focus on contextual data collection and context characterization based on these data.
For what concerns the collection of contextual data, methods and tools will be investigated
and applied to mobile media application users. With regard to context characterization, the
study will focus on mapping contextual data gathered through logging and remote interviews
to meaningful contexts.
Finally, the thesis will study some of the issues related to the evaluation of the two
previous areas with potential end users. The emphasis will be placed on the reliability of test
participants and the interaction between them and the researchers during both supervised
lab-based experiments and large scale unsupervised field trials.

2
 2 Scientific content
Background
In 2009, 40 million users worldwide watched mobile TV, representing about 1% of all mobile
phone users [23]. The mobile TV market is still in its emerging phase yet it demonstrates a
huge growth potential for the near future. If today most mobile TV consumers are located
in Eastern Asia (mostly Japan and South Korea), a study conducted by Rethink Research
Associates shows that Western Europe is expected to lead in revenues from the global mobile
TV market by 2011, with an estimated share of 32% of a foreseen 9 e billion global mobile
broadcasting market.
If it is accepted that fixed TV consumption is part of most people’s everyday life, con-
suming video on a mobile device implies very different factors linked to the surrounding
environment, such as the context of use or the location’s network capability. Therefore it is
vital to understand the users’ habits and needs prior to developing mobile TV services. As
stated in [20, 27] users have very different needs regarding mobile TV content and interaction
when on the move from when watching TV at home.
Broadening the scope from mobile TV to mobile media in general, numerous empirical
studies conducted by industrial and educational institutions illustrate the strong impact
of usability on the degree of service acceptance by end users. For instance, the extensive
bibliography used in [4] provides a valuable source of information regarding methodologies
and setups used to evaluate various aspects of mobile usability. These cover issues from
design guidelines for handheld applications [3] to comparison between laboratory and field
testing of mobile applications [11].
In parallel, some researchers focused their study on issues related to the use of a small
screen. For instance, [13, 14, 15] examine the subject through case studies involving test
users and define requirements for mobile TV interfaces with regards to image resolution,
bandwidth and user interaction.
The Converged, Advanced Mobile Media Platform (CAMMP) project1 provides a unique
environment to investigate such issues in a real environment. Alexandre’s thesis is expected
to be an outcome of CAMMP, therefore the work conducted as part of CAMMP will serve
the thesis work and vice-versa.

State of the art

Mobility and mobile contexts
The usage of mobile applications can vary dramatically depending on the user’s immediate
context. It is therefore primordial to understand and use contextual information properly
when designing or evaluating such application. In order to do so, a definition of what context
is should first be agreed upon. A common definition has been stated in [6], which will be
used as a working basis.
Furthermore, understanding mobile contexts requires capturing contextual data. Exten-
sive research has been carried out trying to define models to describe contextual data [24],
1 http://www.cammp.aau.dk, May 2010

3
sometimes based on an ontology [16], or to infer meaningful information from contextual
data [19, 29]. Researchers have also created tools dedicated to collecting and processing
these data [26].
Finally, the context in which users may consume or produce multimedia content has been
the focus of studies such as [20]. This study reports that users tend to consume mobile videos
primarily to kill boredom when alone or to share fun when together with a relative. It should
be added that further studies have centered their experiments on one specific context (such
as public transportation in [25]) and thoroughly investigated user segmentations and mobile
TV consumption therein.

Interactivity
Media interactivity involves the viewer as an active node of the content delivery chain. This
is the reason why [2] proposes to replace the common hierarchical classification produce-
distribute-view by the edit-share-control taxonomy. For each dimension of this taxonomy,
the authors have identified room for future research. A few examples of topics relevant for
this thesis are:
∙ Methods, tools and techniques for modeling and employing user-generated content
(UGC)

∙ Social practices while watching content, synchronously and asynchronously

∙ Contextual (editorial) information manipulation and usage of complementary support-

ing devices
The use of a mobile device as a complement to the fixed television in the iTV experience
has also been the focus of recent research. For instance [5] evaluated the potential of such a
second screen approach in a representative group of homes in the United Kingdom.
In a later study, [1] identifies four main scenarios where users were engaged in either
controlling, enriching, sharing or transferring digital content. The results show that the
secondary display is mostly seen valuable for previewing and viewing content as well as
accessing enriched information.
It appears therefore crucial to evaluate both the context in which such new interactive
services are used as well as specific usability aspects of the interaction between users and
interactive mobile media related services.

Usability and user experience evaluation

When it comes to evaluating an electronic consumer product, [17] emphasizes three principles
areas to take into consideration: usability, experience and functionality. If the latter is clearly
a technical issue, the definition of usability and experience appears more confusing.Usability
is often defined as a combination of various factors: Learnability, efficiency, memorabiliy,
errors, satisfaction for [18]; Effective, efficient, engaging, error tolerant and easy to learn for
[21].On the other hand user experience is generally accepted as a more global scale, including
pre- and post-ownership interaction with the product.
Finally, the field of mobile usability evaluation is animated by an interesting discussion
on the benefits of field trials over experiments conducted in laboratory. Numerous studies
have compared both approaches [10, 11, 12] and have agreed that if field trials provide more
reliable output due to the realistic environment surrounding the test users, the actual gain
of the field trial method is difficult to quantify, especially when rated against additional
parameters such as costs or practical issues.

4
User involvement in usability studies
Involving users at various stages of the design process of applications is the core idea of user
centered design as described in [18]. This approach has been widely applied, including to
interactive television services. For instance [8] relates how users were interviewed and took
part in focus groups to imagine future applications and equipment. Another approach is
reported in [20], where diary techniques and ethnographic interviews were used to understand
how people use mobile video technologies in their everyday life.
Most of these studies are based on ad-hoc setups, where the participants take part in a
specific study that lasts from few hours (interviews) to few months (diary methods). The
benefits of using a group of core test participants throughout an entire user-centered project
that lasts for several years calls for further research.
Moreover, usability evaluations have long faced the issues of result validity and reliability
[28] and of the individual differences between users [18]. It thus seems that the results from
one experiment could vary quite substantially from one group of participants to another. If
statistical analysis of the results can ensure a confidence interval, additional non-statistical
methods such as multimodal communication analysis [22] or facial expressions analysis [7]
may enhance the test results reliability by informing researchers on the participants trust-
worthiness.

Research questions and methodology

Three research questions have been identified as of primary interest for the thesis. As detailed
in this section, the contributions to the scientific community evolve with the questions from
general to specific.
1. How does mobility impact media consumption and creation in a society
that is always connected?
Answering this question will contribute to the establishment of new Human-Computer
Interaction (HCI) patterns in mobile media consumption. The expected outcome is
a deeper understanding of the effects of the ubiquitous access to digital media on
consumption and creation practices. In order to answer this question, a hierarchy of
situations in which mobile media is consumed will be created. Such situations can
be characterized by a combination of dimensions, including the user’s location and
activity, the surrounding environment, etc. Then it is intended to study the evolution
of media usage through in-depth interviews and remote logging of media consumption
during field trials. Finally, a comparative study of common practices in two different
markets (namely Denmark and Japan) will be performed through in-depth interviews
with senior researchers, as well as active participation in research projects in both
markets.
2. How do end users perceive and use the combination of mobile and fixed
devices in their media experience?
This question aims at informing how consumers interact with multiple pieces of equip-
ment when dealing with media. Emphasis is put on the usability of transferring content
and manipulating services in a multi-devices environment. This question will be tack-
led through the investigation of (1) the handover experience while transferring content
between devices or varying nature, (2) the mobile phone as a second screen for interact-
ing with television programs and services, (3) the mobile phone as a universal remote
enabling the control of various home media appliances. Furthermore, it is intended to
assess the acceptability of seamless integration across services through contextual data
collection from field trials and in-depth interviews.

5
 3. How to improve the researcher/test user communication and measure the
test user’s reliability?
This last research question addresses specific issues related to evaluation methods in-
volving users. It is intended to answer the question of reliability by comparing media
consumption intentions as expressed by users and their actual consumption when using
a fully implemented service in their everyday life. Using multimodal analysis techniques
during usability evaluations will also help assessing the reliability of test participants.
Additionally, the impact of recurrent (educated) test participants on the evaluation
results will be measured. For what concerns the communication between researchers
and test participants, the focus will be put on large scale unsupervised studies. Var-
ious communication schemes will be evaluated and compared. Finally, a cost benefit
analysis will be conducted by comparing the results collected from various test settings
and linking them to the cost of implementing the test setup.

Preliminary results
Four user tests have been conducted so far in the context of CAMMP. The two first studies
have been documented in [9] and were aiming at defining basic knowledge about how people
in Denmark relate to mobile TV and provide indication about which specific areas should be
further investigated in priority. The two later studies built partly on the results from these
and also investigated additional usability issues with various aspects of converged mobile
media.

Mobile TV in a social environment

The purpose of this first test activity was to investigate how users handle the consumption
of rich media in a social context, covering the following questions:
∙ Is it natural for users to consume rich media in a social context?
∙ Do issues such as privacy and personal sphere play a role in such contexts?
∙ Does the social context with its inevitable auditory/visual impact call for specific needs,
such as head-phones to cancel ambient noise?
∙ How does channel switching perform on the mobile device?
The study showed that users would feel comfortable watching TV on their mobile phone
when surrounded by strangers. However the use of earplugs would be necessary to ensure
privacy and an appropriate experience.

Mobile collaboration/competition
The purpose of this second activity was to investigate the concepts of competition and
collaboration involved when using rich media services in a social mobile context. The results
from this evaluation activity provide a basis for future elaboration on interactive rich media
scenarios, associated services and their context of usage. Additionally, the study uncovered
issues to be further investigated in order to ensure mobile participation to a collaborative
service, such as rewards and information quality.

Video content handover

This activity deals with the concept of transferring video content being watched on a mobile
phone to a wall-mounted television set. The overall purpose of the activity is to investigate
if users actually can relate to such functionality, and to determine which set of actions is
preferred to actually perform the transfer. The activity consists of two sub-activities: a
conceptual interview with participants with the purpose of evaluating conceptual/technical

6
issues as well as the need for actually understanding what is going on behind the scenes.
Secondly, a simulation of actual transfer experience is conducted during which un-biased
ideas from the participants are elucidated and pre-selected concepts for handing over video
clips from a phone to a TV screen are evaluated.

Channel switching delays

This activity deals with the investigation of the potential impact of switching delays on
the usability of broadcasting solutions for handheld devices. It is in other words to be
investigated if there exists a comfort threshold (duration wise), above which users will be
annoyed when switching between video feeds. Video clips pre-padded with two types of
transition (icon indicating activity or altered video content and sound) are evaluated by
participants in two contexts (a usability lab and a tent simulating a bus trip). The impact
of those two parameters on the delay acceptability are measured.

Contribution and outcome

The two preliminary studies helped identifying critical aspects of the user experience with
mobile television, such as channel switching delays. The results from the study on seamless
handover across devices are a direct input for the second research question, and the results
from the study on switching delays start addressing the issue of the influence of mobility
on the user experience with mobile media services. All these studies also participate in
answering some of the issues related to the third research question, especially concerning the
trust-worthiness of test participants, through video recording of the test sessions.
The outcome of the entire PhD project is twofold. Answering the two first research
questions will confirm and extend previous work in the field of converged mobile media,
while the issues addressed by the third question will contribute to the HCI community
with original methodological approaches to classic problems encountered by researchers and
practitioners.

Time schedule
Phase 1 (months 1-12) | Thesis planning, PhD courses
The main goal of this first period was to gather grounding knowledge for the thesis. This
has been done by the following means:
∙ Iteratively updating the preliminary study plan submitted at the time of enrollment
in the PhD program in order to plan the overall development of the thesis project
∙ Attending most of the required PhD courses and performing literature review on the
selected areas of interest
∙ Conducting a set of preliminary user tests focusing on general issues, in order to state
hypotheses to the research questions and identify specific areas of focus to be further
investigated

Phase 2 (months 13-30) | Arguments development, user testing

It is expected that the second phase of the project will see the research questions and
hypotheses investigated not only theoretically but also and mostly practically through user
experiments. The results from these studies will be disseminated as they arrive. Work
concerning mobile contexts will be carried out during field trials that are due to start late
2010 or early 2011. Work on interactivity started in early 2010 and will be carried on

7
mainly through iterative lab-based studies and interviews throughout the project. All these
user studies, which started mid-2009, are also the opportunity to investigate the experiment
related issues identified in the third research question. Finally, the second phase should
include the stay abroad, planned as a 4-6 month period starting in September or October
2010.

Phase 3 (months 31-36) | Results validation, thesis writing

The project’s final phase will consist first in summarizing the results gathered in the experi-
ments conducted during the earlier phases in order to answer the research questions. Writing
the PhD thesis will also take most of the time in this last phase, although Alexandre will
keep on being involved in CAMMP related activities.

Milestones
The PhD project will be punctuated by three types of milestones: CAMMP deliverables (see
Appendix A for complete list), dissemination deadlines (see Section 5 for the details) and
a number of user studies to collect the data required for answering each research question.
These milestones will guide the development of the thesis project as the test results and
hence their dissemination will decide whether the research questions can be answered or
not. The user tests will be conducted mostly according to CAMMP’s time schedule, which
plans its test activities in the spring of each year. By the end of June each year, primary
dissemination of the results gathered during the test should be conducted. Depending on
the outcome of these, additional user studies may be conducted outside this time schedule,
in order to address specific thesis-related issues. Although the detail of the next studies is
not known at this time, it can be expected that they follow the plan depicted in Figure 2.1.

Figure 2.1: Tentative project schedule

8
Thesis outline
The thesis is expected to be written in the form of a monograph, reflecting the following
outline. The introduction will extend the state-of-the-art partly completed in this document
and will deduce the main issues of interest for the later development of the thesis. The
body of the thesis will then elaborate on the three main areas framing Alexandre’s work:
(1) Mobility, (2) Convergence and (3) User studies. Each area will be covered from both
theoretical and practical points of view, the later illustrating the arguments elaborated in
the former. Next is a tentative table of content for the thesis.

1. Introduction ii. Mental models

(a) Context of the study (b) Augmented remote control
(b) Theoretical background i. Usability: EPG
(c) Related work ii. Usability: Selected iTV service
2. Part 1: Mobility 4. Part 3: User studies
(a) Data collection (a) Reliability
i. Automatic logging tools i. Questionnaires
ii. Remote interviews ii. Multimodal tools
(b) Automatic context characterization (b) Participant-researcher interaction
i. Data processing i. Intrusiveness
ii. Data-context mapping ii. Remote questionnaires
3. Part 2: Convergence 5. General conclusions
(a) Content transfer (a) Contribution discussions
i. Usability (b) Opening to further research

3 Supervisor-student agreement
Alexandre and professor Lars Bo Larsen will be meeting on a weekly basis as part of the
CAMMP project. The purpose of these meetings is to coordinate the project work and align
it with the other CAMMP partners. Besides these project meetings, specific discussions on
Alexandre’s PhD work will be arranged regularly. In addition to plan the PhD work and
assess its progress, these discussions aim at ensuring the proper allocation of time and tasks
between the CAMMP and the PhD projects.

4 PhD courses
Table 4.1 lists the PhD courses Alexandre has already completed. The remaining credits
shall be obtained through either additional courses related to one of the project’s topics (for
instance on mobile user-generated content), or by attending a relevant conference tutorial.

9
 Course Org. ECTS Type Status
Design and analysis of experiments AAU 4.00 General Completed,
fall 2008
Intellectual property rights AAU 2.00 General Completed,
spring 2009
Professional communication AAU 2.50 General Completed,
fall 2009
Writing and reviewing scientific papers AAU 3.75 General Completed,
fall 2009

Audiovisuality in the era of digital conver- FMKJ* 4.00 Project Completed,

gence spring 2009
Multimodality in human interaction: qualita- FMKJ* 2.00 Project Completed,
tive methods for analyzing audio and video spring 2009
recordings
Social media: analysing identity, sociality and FMKJ* 3.00 Project Completed,
creativity in online networked environments spring 2009
The Political Economy of Information and AAU 5.00 Project Completed,
Communication Technologies - Next genera- summer 2009
tion mobile media
Interaction Design for Ubiquitous Computing AAU 3.00 Project Planned,
spring 2010
Total 29.25

Table 4.1: PhD courses already attended or planned. * Forskerskolen i Medier, Kommunikation
og Journalistik (Roskilde University)

5 Dissemination of knowledge
Dissemination of the work throughout the thesis project will be done by means of written
documents (conference and journal papers, as well as contributions to CAMMP deliverables),
presentations (oral and posters, at conferences and workshops) and prototypes. For instance,
Alexandre was part of the program committee for the workshop on Mobile Living Labs1 at
MobileHCI 2009. Written documents will either discuss a theoretical point, introduce a
methodology or a tool, or present the results from user tests. Presentations will be typically
produced for a specific audience (e.g. for a workshops) in order to present results from
user studies. Prototypes and mock-ups will be produced iteratively in order to investigate
relevant research issues.
Additionally, Alexandre will contribute to a list of CAMMP deliverables (see Appendix A)
that will act as additional milestones for the PhD study plan. Indeed, the research conducted
in the PhD project will be a direct input for those deliverables.
Furthermore, Alexandre has started submitting papers to various venues (conferences,
1 http://livinglab.novay.nl/wiki/MLL09, May 2010

10
workshops and a doctoral consortium) as listed in Table 5.1. It is intended that Alexandre
will keep on publishing his work in that way, targeting in priority the conferences and journal
listed in Appendix B.

Title, (Type) Status

Evaluation Framework for Mobile Rich Media Services (conference) Published, presented
at DHRS 2008
State of The Art Literature Study of Methods for Evaluation Framework Published
(CAMMP deliverable)
Two Perspectives on Mobile Television: Consumption in a Social Context Published, presented
and Collaborative/Competitive Behaviors (conference) at DHRS 2009
Converged Mobile Media: Evaluation of an Interactive User Experience To be presented at
(doctoral consortium) EuroITV 2010
A Pragmatic Approach to Testing Issues in a Mobile Platform that Does To be presented at
Not Yet Exist (workshop) EuroITV 2010
The Challenge of Evaluating Concepts Hidden Behind a Technological Submitted to
Layer (conference) NordiCHI 2010

Table 5.1: Work already or to be presented

It can be foreseen that each part of the thesis will refer to a major paper written by
Alexandre as a main author. According to this scheme, the three following papers can be
expected:

- Inferring mobile media consumption contexts from automated remote data logging, pos-
sibly written in collaboration with Kasper L. Jensen from AAU (department of Me-
dialogy), to be presented at a conference such as Mobile HCI 2011 or submitted to a
journal such as Springer’s Personal and Ubiquitous computing (in which case the paper
would be submitted after one of CAMMP’s field test campaigns in 2011 or 2012)

- Combining a tablet computer and a TV to optimize home media consumption, to be

presented at a conference such as CHI 2011.

- Using multimodal analysis to detect reliability of test participants during usability eval-
uation, to be presented at a conference such as HCI International 2011 or published
in a journal such as Elsevier’s International Journal of Human-Computer Studies (the
manuscript could be then submitted in early 2012)

In addition to these papers, it is expected that Alexandre will co-author at least one
conference paper written in the context of his stay abroad. If the exact scope of the paper
is still to be identified, it will have to do with usability of mobile user-generated content and
can be expected to be submitted in the Spring 2011.
Finally, it is planned to satisfy the required 840 hours of teaching obligations mainly
by supervising student groups working on projects that are relevant either for CAMMP or
the PhD project. Additionally, Alexandre may be involved in some teaching assistance.
Table 5.2 reports the teaching obligations completed so far.

11
 Period Description Hours
Spring 2009 Co-supervision of group VGIS-1025 22.00
Fall 2009 Supervision of group VGIS-920 90.00
Fall 2009 Supervision of group VGIS-921 90.00
Spring 2010 Supervision of group VGIS-1021 45.00
Total 247.00

Table 5.2: Teaching obligations completed so far

6 Rights to patents
No patent applications are planned at the present time, but it is not unlikely that Alexandre -
either directly or as part of the CAMMP project as such - will produce one or more patents.

7 External collaboration
It is intended that Alexandre will conduct part of his research at a foreign research institution.
Japan is envisioned as a destination of first choice for its advance mobile media culture and
the numerous possibilities to gain valuable experience for both the PhD project and CAMMP.
The topic of the research conducted at this institution is still to be defined in details, however
it will concern the field of mobile user-generated content and the applications thereof, for
instance as a social mean of communication.
Professor Shin Mizukoshi from the University of Tokyo has been contacted and prelimi-
nary arrangements have been discussed for a possible stay in his (or one of his colleagues’)
lab. It is intended that the stay abroad would cover a 4-6 month period stating in September
or October 2010.
Funding of the stay abroad will be covered by Alexandre’s salary from CAMMP and from
a funding institution (such as the Otto Mønsteds Fond) that remains to be applied to.

8 Budget
The expenses related to Alexandre’s PhD project are funded under an agreement between the
Department of Electronic Systems and the CAMMP project. These expenses include salary,
CAMMP related travel costs and equipment such as a laptop or test-related equipment.
Alexandre’s salary will also partly cover the expenses related to his stay abroad (travel,
accommodation and other living expenses).

12
 Bibliography
[1] P. Cesar, D. C. A. Bulterman, and A. J. Jansen. Usages of the secondary screen in
an interactive television environment: Control, enrich, share, and transfer television
content. In Changing Television Environments, volume 5066/2008 of Lecture Notes in
Computer Science, pages 168–177. Springer Berlin / Heidelberg, 2008.

[2] P. Cesar and K. Chorianopoulos. The evolution of tv systems, content, and users toward
interactivity. Found. Trends Hum.-Comput. Interact., 2(4):373–95, 2009.

[3] S. S. Chan, X. Fang, J. Brzezinski, Y. Zhou, S. Xu, and S. Xu. Usability for mobile com-
merce across multiple form factors. Journal of Electronic Commerce Research, 3(3):187
– 199, 2002.

[4] C. K. Coursaris and D. J. Kim. A qualitative review of empirical mobile usability

studies. In Proceedings of the Twelfth Americas Conference on Information Systems,
pages 2885–2897, Acapulco, Mexico, August 2006.

[5] L. Cruickshank, E. Tsekleves, R. Whitham, A. Hill, and K. Kondo. Making interactive

tv easier to use interface design for a second screen approach. The Design Journal,
10(3):41–53, November 2007.

[6] A. K. Dey. Understanding and using context. Personal and Ubiquitous Computing,
5(1):4–7, February 2001.

[7] H. B.-L. Duh, G. C. B. Tan, and V. H.-h. Chen. Usability evaluation for mobile device:
a comparison of laboratory and field tests. In MobileHCI ’06: Proceedings of the 8th
conference on Human-computer interaction with mobile devices and services, pages 181–
186, New York, NY, USA, 2006. ACM.

[8] L. Eronen. User centered research for interactive television. In Proceedings of the 2003
European Conference on Interactive Television (EuroITV): From Viewers to Actors,
pages 5–12, Brighton, UK, April 2003.

[9] A. Fleury, J. S. Pedersen, and L. B. Larsen. Two perspectives on mobile television :

Consumption in a social context and collaborative/competitive behaviors. In O. W. Ber-
telsen and A. M. Kanstrup, editors, Proceedings of the Ninth Danish Human-Computer
Interaction Research Symposium, pages 18–21, December 2009.

[10] K. L. Jensen and L. B. Larsen. The challenge of evaluating the mobile and ubiquitous
user experience. In Proceedings of Second International Workshop on Improved Mobile
User Experience, 2008.

[11] A. Kaikkonen, A. Kekäläinen, M. Cankar, T. Kallio, and A. Kankainen. Usability testing

of mobile applications: A comparison between laboratory and field testing. Journal of
Usability Studies, 1:4 – 16, November 2005.

[12] J. Kjeldskov and J. Stage. New techniques for usability evaluation of mobile systems.
International Journal of Human-Computer Studies, 60(5-6):599–620, May 2004.

13
[13] H. Knoche and J. D. McCarthy. Design requirements for mobile tv. In MobileHCI ’05:
Proceedings of the 7th international conference on Human computer interaction with
mobile devices & services, pages 69–76, New York, NY, USA, 2005. ACM.

[14] H. Knoche, J. D. McCarthy, and M. A. Sasse. Can small be beautiful?: assessing image
resolution requirements for mobile tv. In MULTIMEDIA ’05: Proceedings of the 13th
annual ACM international conference on Multimedia, pages 829–838, New York, NY,
USA, 2005. ACM.

[15] H. Knoche, M. Papaleo, M. A. Sasse, and A. Vanelli-Coralli. The kindest cut: enhancing
the user experience of mobile tv through adequate zooming. In MULTIMEDIA ’07:
Proceedings of the 15th international conference on Multimedia, pages 87–96, New York,
NY, USA, 2007. ACM.

[16] P. Korpipää and J. Mäntyjärvi. An ontology for mobile device sensor-based context
awareness. In 4th International and Interdisciplinary Conference on Modeling and Using
Context, Lecture Notes in Artificial Intelligence, pages 451 – 459, Stanford, California,
2003. Springer-Verlag.

[17] N. McNamara and J. Kirakowski. Defining usability: quality of use or quality of ex-
perience? Professional Communication Conference, 2005. IPCC 2005. Proceedings.
International, 0:200–204, July 2005.

[18] J. Nielsen. Usability Engineering. Morgan Kaufmann, September 1994.

[19] P. Nurmi and S. Bhattacharya. Identifying meaningful places: The non-parametric

way. In Pervasive ’08: Proceedings of the 6th International Conference on Pervasive
Computing, pages 111–127, Berlin, Heidelberg, 2008. Springer-Verlag.

[20] K. O’Hara, A. S. Mitchell, and A. Vorbau. Consuming video on mobile devices. In M. B.

Rosson and D. J. Gilmore, editors, CHI ’07: Proceedings of the SIGCHI conference on
Human factors in computing systems, pages 857–866, New York, NY, USA, 2007. ACM.

[21] W. Quesenbery. The five dimensions of usability. In M. J. Albers, B. Mazur, and M. B.

Mazur, editors, Content and Complexity (Information Design in Technical Communi-
cation), pages 81–102. Lawrence Erlbaum Associates, Mahwah, NJ, January 2003.

[22] T. Stivers and J. Sidnell. Introduction multimodal interaction. Semiotica, 156(1):1–20,

2005.

[23] K. Taga, C. Niegel, and L. Riegel. Mobile tv: Tuning in or switching off? Technical
report, Arthur D Little, 2009.

[24] S. Tamminen, A. Oulasvirta, K. Toiskallio, and A. Kankainen. Understanding mobile

contexts. Personal Ubiquitous Comput., 8(2):135–143, 2004.

[25] C.-C. Tao. Market segmentation for mobile tv content on public transportation by
integrating innovation adoption model and lifestyle theory. Journal of Service Science
and Management, 1:244–250, December 2008.

[26] H. ter Hofte, K. L. Jensen, P. Nurmi, and J. Froehlich. Mobile living labs 09: Methods
and tools for evaluation in the wild: http://mll09.novay.nl. In MobileHCI ’09: Proceed-
ings of the 11th International Conference on Human-Computer Interaction with Mobile
Devices and Services, pages 1–2, New York, NY, USA, 2009. ACM.

14
[27] J. Trefzger. Mobile tv-launch in Germany - challenges and implications. Technical
Report 209, Cologne University, November 2005. Working Papers of the Institute for
Broadcasting Economics, Cologne University.

[28] M. Wenger and J. Spyridakis. The relevance of reliability and validity to usability
testing. Professional Communication, IEEE Transactions on, 32(4):265 –271, dec 1989.
[29] C. Zhou, D. Frankowski, P. Ludford, S. Shekhar, and L. Terveen. Discovering personally
meaningful places: An interactive clustering approach. ACM Trans. Inf. Syst., 25(3):12,
2007.

15
 A CAMMP deliverables

Type Deliverable title Due

IR Evaluation results of 2009 lab test May 2009
IR Evaluation results of 2009 user test May 2009
D State-of-the-art literature study of methods for evaluation framework June 2009
IR State of the art picture contribution - 2009 August 2009
IR The second set of key concepts, scenarios and use cases and the first September 2009
results from the scenario validation
IR Specification of applications and UIs in 2010 trials October 2009
IR Methods for 2010 iteration of evaluation framework November 2009
IR Evaluation results of 2010 lab test May 2010
IR Evaluation results of 2010 user test June 2010
IR State of the art picture contribution - 2010 August 2010
IR The third and final set of key concepts, scenarios and use cases and September 2010
the first results from the scenario validation
D Specification of applications and UIs for 2011 user trials October 2010
IR Methods for 2011 iteration of evaluation framework November 2010
IR Evaluation results of 2011 lab test May 2011
IR Evaluation results of 2011 user test June 2011
IR State of the art picture contribution - 2011 July 2011
D The forth key concept, scenarios and use cases development that will August 2011
be implemented in the project
IR Specification of applications and UIs in 2012 trials September 2011
IR Methods for 2012 iteration evaluation framework October 2011
D Communication-performance evaluation models and techniques with April 2012
respect to user scenarios
D Evaluation results from field trials May 2012

Table A.1: CAMMP deliverables and internal reports that Alexandre will contribute to
(Type: IR = Internal report, D = Deliverable)

16
 B Venues for dissemination

Conference name Next edition

Date Location
EuroITV (European Conference on Interactive TV and Video) 9-11 June Tampere,
2010 Finland
DIS (Designing Interactive Systems) 16-20 Au- Århus,
gust 2010 Denmark
UbiComp (ACM International Conference on Ubiquitous Computing) 26-29 Copenhagen,
Septembre Denmark
2010
NordiCHI (Nordic Conference on Human-Computer Interaction) 16-20 Octo- Reykjavik,
ber 2010 Iceland
CHI (ACM Conference on Human Factors in Computing Systems) 7-12 May Vancouver,
2011 Canada
HCI International (International Conference on Human-Computer 9-14 July Orlando,
Interaction) 2011 USA
Mobile HCI (The International Conference Series on Human Com- September Stockholm,
puter Interaction with Mobile Devices and Services) 2011 Sweden

Table B.1: Relevant conferences

Journal name Publisher

Transactions on Computer Human Interaction ACM
Behavior and Information Technology Taylor & Francis
Pervasive and Mobile Computing Elsevier
Convergence, The International Journal of Research into New Media SAGE
Technologies
Personal and Ubiquitous Computing Springer
International Journal of Human-Computer Interaction Taylor & Francis
International Journal of Human-Computer Studies Elsevier

Table B.2: Relevant journals

17