You are on page 1of 17


Chapter 16
Smart Displays in Ambient
Intelligence Environments
Fernando Reinaldo Ribeiro
Algoritmi Research Centre, University of Minho, Portugal

Rui Jos
Algoritmi Research Centre, University of Minho, Portugal

A public display that is able to present the right information at the right time is a very compelling concept.
However, realising or even approaching this ability to autonomously select appropriate content based
on some interpretation of the surrounding social context represents a major challenge. This chapter
provides an overview of the key challenges involved and an exploration of some of the main alternatives
available. It also describes a novel content adaptation framework that defines the key building blocks for
supporting autonomous selection of the Web sources for presentation on public displays. This framework
is based on a place model that combines content suggestions expressed by multiple place visitors with
those expressed by the place owner. Evaluation results have shown that a place tag cloud can provide
a valuable approach to this issue and that people recognize and understand the sensitivity of the system
to their demands.

1. INTRODUCTION multimedia environments. However, most public

displays today are mere distribution points for
In Ambient Intelligence scenarios, the environ- pre-defined and centrally created content. They
ment can perceive and react to people, sense on- assume passive users and they are completely
going Human activities and proactively respond unaware of the Human activities taking place at
to them. Public digital displays have always been that same location. Consequently, they are not very
part of this vision (the boards (Weiser, 1993)), valued by their potential users and their content is
and their increasingly ubiquitous presence in our often perceived as too institutional or dull (Huang,
socio-digital landscape has been opening new Koster, & Borchers, 2009).
opportunities for their use as important building Ambient Intelligence, social media and the
blocks for many types of Ambient Intelligence ubiquitous presence of interaction devices rep-
DOI: 10.4018/978-1-4666-7284-0.ch016

Copyright 2015, IGI Global. Copying or distributing in print or electronic forms without written permission of IGI Global is prohibited.

Smart Displays in Ambient Intelligence Environments

resent huge opportunities to move towards new display system that autonomously selects from
display concepts that are able to act as situated web sources the content deemed more relevant
artefacts. These displays would no longer be iso- according to a dynamic place model that is sensi-
lated from the people and the data around them. tive to the people around the display. Regardless
Instead, they would be smart displays capable of of specific technical options, e.g. the use of tag
dynamically aligning their behaviour with their clouds or the use of Bluetooth names to express
usage circumstances. This smart behaviour may tags, the overall system provides a general frame-
include the ability to dynamically integrate content work for the key building blocks that an adaptive
from the web and select sources according to their display should be able to support, and will thus
relevance to the social context around the display. be used as a reference system to describe how to
The number and diversity of content sources on approach this particular problem.
the Internet offers the potential to guarantee a
continuously updated stream of relevant content
for the displays. The potential is so vast that we 2. APPROACHES FOR ADAPTIVE
can safely say that content would no longer be a CONTENT ON PUBLIC DISPLAYS
scarce resource and that proper selection would
indeed become the key problem. Mobile technol- The idea of a public display that is able to present
ogy, on the other hand, may offer the potential to the right information to the right users, at the right
implement multiple forms of automated person- time and in the right way is obviously very compel-
alization, or adaptation, of the displays. ling. However, realising it, or even approaching
This chapter explores the concept of smart it in some way, is extremely challenging because
displays in Ambient Intelligence environments. of the complex issues involved, such as obtaining
This generic concept has been explored from information about preferences in a non-obtrusive
many different perspectives and through the and privacy-preserving way, the need to combine
development of many types of reactive displays the various preference expressions of the people
that sense and adapt in some way to their sur- in that place, and the complexities involved with
rounding circumstances. These various systems making meaningful inferences about relevance
entail a very broad range of design approaches and based on the information available. The first major
assumptions, highlighting the diversity of ways challenge however is to properly characterize the
in which the notion of smart display can be ap- problem and its key assumptions, as a very broad
proached. We will start by presenting an overview range of approaches have been followed to explore
of the design space, framing our work within this different views of this problem domain. To clarify
broader concept of smart displays. We will then the overall design space for adaptive content on
take a closer look at the more specific concept public displays, we will start with an overview of
of an adaptive display as a display that is able o the overall scheduling principles that determine
select content that is tailored to the preferences how content is selected and we will analyse how
or goals of the individuals or groups visiting the previous work has explored the different possibili-
place where the display is set. This would make ties offered by that design space.
each display system unique and closely related with
the specific place where it is installed, providing 2.1. Content Scheduling
the ground for highly situated displays that reflect for Public Displays
the expectations, interests and practices associated
with the people in a particular place. As part of our The process of determining which content to show
work on this topic, we have developed an adaptive on public displays is normally termed content


Smart Displays in Ambient Intelligence Environments

scheduling. The overall scheduling process may be 1 represents this range of possibilities and how
seen as spawning two major phases, specification the different entities can have some role in each
and execution, and possibly involving three types of these two phases.
of entities: Display Owners, System, and Viewers. The specification of the schedule by the display
Specification is the phase where the schedule owner is where the most fundamental decisions
is being designed to correspond to an envisioned about the intended display concept are normally
display concept. A display owner is expected to made. In traditional Digital Signage systems, this
have a major role in this phase as the designer of is actually where all the decision are made, as
some intended user experience. The existence of the schedule, both content and the way it will be
this intention is what separates the public display presented, is fully specified in-advance by display
from a simple desktop system where the user could owners, leaving no space for smart content selec-
simply access the entire range of services without tion or user requests. However, smart displays may
any system-defined boundaries. This process oc- explore a dynamic scheduling model in which the
curs outside the real display context and therefore specification defines policies that determine the
the specification must either ignore that context high-level behaviour of the system, but not exactly
by making a full specification in advance or leave what it will do. In the end, the effective behaviour
some open decision points that will be resolved exhibited by the system will result from the com-
in execution time. Execution is the phase where bination between those policies and the stimuli
immediate decisions must be made to select what received from the environment. When considering
to display next. Scheduling may enable multiple how to approach this goal, we have explored two
types of dynamic decisions to occur at this phase major alternatives: modelling this problem as a
that may take into account the execution context context-awareness problem and modelling it as a
or explicit indication from owners or users. Figure recommendation problem.

Figure 1. Design space for scheduling decisions


Smart Displays in Ambient Intelligence Environments

2.2. Adaptive Scheduling as inferred, and then a dissemination stage, in which

Context-Aware Content Selection those rules would become domain knowledge
ready to be embedded into scheduling processes.
To approach adaptive scheduling as a context- It is unclear, however, how much of the generated
awareness problem, the scheduler should be able knowledge would be generic enough to be applied
to access data about the display context and make to multiple displays. Each display will have its
scheduling decisions that reflect rational choices own context and content items and, moreover,
about what is most appropriate for each particular these will be evolving with time.
context. This requires some type of knowledge In a process of unsupervised learning, the
that associates context states with specific content scheduler would be able to learn with the experi-
A first possibility is to embed into the sched- ence gained from the operation at a particular place
uling process a number of behaviour rules that and seamlessly adapt to the specific characteristics
represent the empirical knowledge of how the of that environment. However, this would require a
system should select content according to the generic feedback mechanism whereby users could
continuously changing context around the dis- express their opinion on the scheduling decisions,
play. For example, we may want to say that a or more specifically about the relevance of what
particular piece of content should be scheduled is being shown. This learning process could lead
for presentation when the local temperature is to two complimentary types of result: information
above a certain threshold, or that another item about the global popularity of individual items;
should not be shown twice in the presence of any and information about the effect of context in
particular Bluetooth device, or that yet another item content popularity. To explore the potential of
should only be shown when there is no one being this approach, we conducted a user study to assess
detected in the immediate physical proximity of what type of context variables could possibly affect
the display. However, specifying meaningful rules peoples perception about content relevance (F. R.
that can apply to generic usage situations can be Ribeiro & Jos, 2007). The results have shown that
extremely difficult. Even though it seems intuitive particular content items were systematically rated
to think that certain situations can affect what is more relevant, regardless of any context changes.
the most appropriate content to be presented in We have not found any statistical significance
a particular context, it is not trivial to explicitly whatsoever in any of the associations between
translate, based only on empirical knowledge, the relevance of a particular item and any of the
subtle interpretations of context into formal and context variables. People seemed to associate the
generic specifications that will rule the display relevance of content much more with the place in
behaviour in multiple real-world contexts. Smart- which it was being presented rather than with any
ness seems to be a rather vague concept that for of the forms of context being sensed.
most cases does not map directly into just one These first experiments have revealed impor-
predefined type of reaction. tant limitations in the notion of context-aware
Considering these challenges, an alternative is adaptation in public displays. The difficulties in
to enable the display system to learn a particular expressing or inferring meaningful associations
notion of smart behaviour, by training the system between context variables and content items
to generate new domain knowledge. mean that the scope of situation in which those
In a supervised learning process, there would approaches can be effective will not be as broad
be a training stage, in which a set of training cases as is often suggested. While not claiming to
of the expected behaviour would be generated, have extensively explored all the alternatives, it
an inference stage, in which new rules would be seems clear that context-awareness would only


Smart Displays in Ambient Intelligence Environments

be efficient in cases where we have a very direct technical limitations resulting from the lack of
association between a particular context state and a mouse and keyboard, but essentially because
a specific system reaction, normally something the display is public and not under the control
in the form of a trigger. These conclusions are of any single user. Most public displays are thus
aligned with previous findings that suggest that conceived with the single goal of being seen by the
context-aware systems should promote a tight people in their vicinity, without considering any
integration between sensing and action instead of other forms of interaction. Even in the few cases
decomposing them as separate parts of the same in which there is some interaction support, the
process (Leahu, Sengers, & Mateas, 2008) or that richness of those interactions, and consequently,
the relevance of the sensor readings corresponding the richness of the digital footprints they generate,
to context variables cannot be established a priori is much more limited than in the more traditional
outside the specific situation of use in which it is scenarios of recommender systems, such as the
being generated (Dourish, 2004). web. Public displays can thus be used without
generating any relevant information about how
2.3. Adaptive Scheduling as they are used, much less about the users interests
a Recommender System or preferences. Without such information, it is not
feasible to implement recommendation techniques
Approaching adaptive scheduling as a recom- that depend very heavily on implicit or explicit
mendation problem may be formulated as follows: expressions of interest.
Given a set of preferences implicitly or explicitly
expressed by the people around a display, how Profiles for Place-Based Adaptation
can they be combined with the characteristics
of the place to enable the system to select from While most recommendation systems target indi-
web sources the most appropriate content to be vidual recommendations, public displays must be
displayed in that context. By framing the issue of designed for shared and communal use in public
adaptive scheduling as a recommendation prob- and semi-public settings. Therefore, instead of a
lem, we can potentially benefit from the wealth user profile, recommendations for public display
of well-known solutions that already exist for should be based on some sort of place profile that
that problem. However, the specific scenario of combines the preferences of the person managing
recommending content from web sources for pre- the display, who we call the place owner, with the
sentation on a public display raises new challenges preferences of the multiple people that may be in
that break some of the assumptions we often find the vicinity of the display.
in recommendation algorithms and that may limit A place owner who installs a public display
their applicability. This section summarises the will have specific expectations regarding the way
main differences between both problem domains. in which the display is going to contribute to the
creation of a particular concept of place. An ef-
Limited Set of Preference Indications fective profile model must acknowledge this role
of public displays and provide the place owner
Unlike other media, attention and engagement with some control over the nature and scope of
are far from being a given with public displays. the recommendations. If the selected content is
In most cases, the system is autonomously select- not aligned with the place values, practices or
ing what to presented next and people are very commercial strategies, its public presentation may
limited in their ability to influence the display become a source of embarrassment.
decisions. This happens, not just because of


Smart Displays in Ambient Intelligence Environments

Place visitors, on the other hand, are the main gested and being presented, the selections of the
consumers of the content presented on the display system will be shown without necessarily being
and for that reason they must also play an active evaluated by a Human, and possibly without any
role in guiding the display behaviour. Since public immediate mechanisms for skipping it. Further-
displays will typically have multiple simultaneous more, given that people will not normally have
users, the adaptation process will need to consider the possibility to control presentation, content will
the best strategy for dealing with the potentially normally be shown in its full extent. This raises
very varied interests expressed by those people. considerably higher risks of presenting irrelevant
This generates a trade-off between the selection or even inappropriate content and substantially
based on a profile combing the multiple interests raises the responsibility of the system in being
of the multiple persons present and the selection able to only recommend appropriate items.
based the use of each individual profile, one at
the time (Alt et al., 2009). The first is a balanced Dynamic and Open Content Sources
approach, but faces the risk of not really matching
anyones specific interests. The second approach A common assumption in content selection for
can be targeted for each individual, but it raises public displays is the existence of some predefined
additional privacy issues and may conflict with list of content alternatives from which content
the idea of public displays as a place-making tool. should be selected. This allows a strict control
over the type of content that can be presented
Select, Not Just Recommend on the display, but it would fail to address our
initial motivation of benefiting from the wealth
Most recommender systems assume some type of of content and information sources on the web.
collaborative process or mixed-initiative model. To really take advantage of the potential of web
The system may suggest multiple data items from information, the display system should allow
various sources and present them in the form of display users to freely express their interests and
short summaries with links for further details, but be able to dynamically select anywhere from the
the user is then expected to assess the multiple Internet the most adequate sources.
relevance cues provided and select which content The focus on dynamic content from web sourc-
may be of interest. For example, when browsing es also means that the relevance of the respective
a video sharing web site, new video suggestions content is likely to face considerable oscillations.
are normally presented after a video is finished. The same source may at a given moment have rich
This gives users the possibility to explore the video and recently updated information and at some
collection in a more serendipitous way, but they are other moment strongly deprecated or even non
also free to ignore the suggestion and follow their existing information. The notion of relevant source
own path. In Internet radio systems, the system may thus change very quickly, not just because of
autonomously selects the next music, but the user changes in the display context, but also because of
can easily override selections by skipping to a next changes in the content itself. As a consequence,
song. The implicit assumption is that the system the adaptation processes at the various selection
responsibility is mainly to facilitate a selection levels must all be very dynamic and frequently
process that is ultimately controlled by the user. re-evaluate their selection decisions. Moreover,
In a public display, none of these possibilities previous feedback on a particular source may itself
is naturally available. Since there is normally no become deprecated very quickly, given that the
intermediate stage between content being sug- respective content may have changed considerably.


Smart Displays in Ambient Intelligence Environments

Presentation Cycles of people in the vicinity of the display. Context-

aware informative display (Zhu, Zhang, Zhang, &
In a recommender system, content is presented Lim, 2007) and context sensitive public display
once and after that the system assumes the user for adaptive multi-user information visualization
is no longer interested in the same content. In a (Morales-Aranda & Mayora-Ibarra, 2007) also
public display, the same content may be presented aim to provide people with relevant content in an
multiple times because it will most likely be seen opportune and personalized way. Dynamo (Izadi,
by different people at different times. A recom- Brignull, Rodden, Rogers, & Underwood, 2003)
mendation system for public displays needs to or BlueBoard (Russell, 2003) are examples of
take into account the existence of presentation systems that give users direct control of the display
cycles in which it becomes acceptable to show and thus content selection is directly handled by
the same content again, if it remains relevant. The users rather than by the system. The BlueScreen
calculation of relevance may thus have to balance (Payne, David, Jennings, & Sharifi, 2006) selects
the inherent relevance of a content item with the and displays adverts in response to users detected
effect on that relevance of previous presentations in the audience. It utilizes Bluetooth-enable
of that same content item. devices as proxies for identifying users and uti-
lizes history information of past users exposure
to certain sets of adverts. Advertisements are
3. RELATED WORK preferentially shown to those users that have not
seen them yet. Muller et al. (Mller, Kruger, &
In this section, we analyse the main approaches Kuflik, 2007) describes a mechanism to adapt
for adaptive content on public displays that have advertisements on digital signage to the interests
been explored in previous work. We also look into of the audience. He proposes a system that uses a
recommender systems and the use of tag clouds nave Bayes classifier to estimate the probability
for interaction and profile representation. that a user is interested in a certain advertisement.
It uses adverts keywords, users history, time,
3.1. Scheduling in Situated Displays location and voucher collection information as
feedback to determine the best advert to display.
For most system, the scheduling process is a based Tacita is a system to allow mobile users to express
on a fixed schedule that cycles through pre-defined personalisation preferences to nearby public dis-
content, but several adaptive scheduling alterna- plays (Kubitza, Clinch, Davies, & Langheinrich,
tives have been explored that introduce sensibility 2012). A mobile client discovers nearby displays,
of the display to some type of external variable. determines the set of applications available, and
Proactive displays (McCarthy, McDonald, So- triggers personalisation. Instant Places (Jos,
roczak, Nguyen, & Rashid, 2004; McDonald, Pinto, Silva, & Melro, 2013) enables people to
McCarthy, Soroczak, Nguyen, & Rashid, 2008) express their content preferences in the form of
select content that is scheduled on-the-fly ac- pins that are recognised when the user checks-in
cording to the interests of users within the direct to a display using a mobile client. The displays
vicinity of the display. It recognizes specific pres- will then preferably select the content sources
ences and display information from associated associated with those pins.
profiles. Similarly, Groupcast (McCarthy, Costa, Even though some of these systems are able
& Liongosari, 2001) uses identification and user to support unassisted and adaptive scheduling,
profiles to present information about the interests they employ customized scheduling algorithms,


Smart Displays in Ambient Intelligence Environments

mostly based on individual profiles. In our work, systems are clear examples of adaptive displays,
we assume that we have no a priori knowledge their focus is attracting and maintaining user
about users profiles, and use a place specification attention. They do not address the autonomous
as the basis for autonomous content selection. selection of content based on the current context
around the display.
3.2. Adaptive Engagement
3.3. Recommender Systems
A number of audience behaviour frameworks
have been proposed to model the various phases Approaches to recommender systems deal mainly
of engagement with public displays as a series with two types of entities: users and items. The
of sequential steps that users need to go through user entity is normally a user profile that is based
to move from being simple passers-by to being on either manual user input of preferences or
engaged users. Adaptive engagement defines dif- automatic user modelling i.e. deriving user pref-
ferent audience situations that can be sensed by the erences and providing recommendations on the
display and specific ways in which the behaviour basis of users history of content consumption. The
of the display should adapt to the audience, more item entity is usually characterized with a set of
specifically their distance or their attention to the metadata that is supplied by the source, but this
display. information can also be extended with additional
The Hello.Wall (Streitz et al. 2003b) is an information that is inserted by users. These entities
ambient display is designed around an audience are the basis of all recommendation techniques
behaviour model that considers three levels of that normally depend on extensive data about both.
interactivity corresponding to three zones: Am- However, the specificities of how they are used
bient, Notification and Cell interaction zones. differ between the three main recommendation
The display aims to maximise engagement by techniques in use today: content-based recom-
exhibiting an appropriate behaviour for each en- mendations, collaborative recommendations and
gagement phase. The interactive public ambient hybrid approaches.
display (Vogel and Balakrishnan 2004) detects In content-based recommendation, the system
the audiences body posture and adjust behaviour suggests to the user the items that best fit the user
according to a four-phase interaction model based profile. A set of attributes that characterize each
on proximity and attention that may be seen as a item is used to determine appropriateness of the
refinement of the three phases of the Hello.Wall item for recommendation purposes. The user will
model. The Peddler Interaction Framework (Wang, be recommended items that are similar to the ones
Boring, & Greenberg, 2012) extends the Audience preferred in the past. A key limitation of these
Funnel Framework (Michelis & Mller, 2011) to techniques is that the system cannot recommend
incorporate continuous proxemic measures such as items that are different from anything the user has
distance and orientation, attention states, such as seen before. This is particularly limiting in our
digression and loss of interest, and the passerbys case because the ability to find new content that
interaction history. The goal is to adjust the display is totally unrelated with what has already been
behaviour is a way that maximises the ability to presented is an important requirement.
attract Attention, maintain Interest, create Desire, Collaborative techniques, in their common
and lead customers to Action. Even though these form, are also difficult to apply. They make recom-


Smart Displays in Ambient Intelligence Environments

mendations based on information of people with 4. A FRAMEWORK FOR ADAPTIVE

similar tastes and preferences, trying to predict CONTENT ON PUBLIC DISPLAYS
the utility of items for a particular user based on
the items previously rated by other users. Their As part of our work on this topic, we have de-
main limitation is not being very good in dealing veloped an adaptive content scheduler for public
with new content items and also with frequently displays that is able to select the information feeds
updated sources (Das, Datar, Garg, & Rajaram, deemed more relevant for the current social set-
2007), making them unsuitable for our scenarios of ting around the display (F. Ribeiro & Jos, 2013).
selecting content feeds in a timely way. Moreover, Even though this scheduling system has been
the focus on a single display, the reduced number instantiated around specific technologies, such
of users in our system and also the potentially very as content feeds and tag clouds, its fundamental
low rating density raise additional challenges to principles should also apply to other instantiations
the application of collaborative recommendation of the problem domain and may thus constitute a
techniques. more general framework to the issue of content
Gonzlez et. al (Gonzlez et al., 2006) pres- recommendation for public displays. There are
ent a system that extends traditional approaches three main sub-systems in this architecture: a
to recommender systems. They analysed cross- place sub-system representing a dynamic and
disciplinary trends from the users affective fac- evolving view of place that combines the place
tors perspective in the next generation of ambient owner specifications with the contributions made
recommender systems. They combine a model of by place visitors; a selector sub-system that takes
the users emotional information with intelligent keywords from the tag cloud in the place sub-
agents and machine learning to provide relevant system and retrieves relevant content for each of
recommendations in everyday life. This work those tags using dynamic web sources as content
also builds on previous work in recommendation providers; and a scheduler sub-system that consid-
systems and retrieval models for feed search (Ar- ers the available content as well as the weight and
guello, Elsas, Callan, & Carbonell, 2008; Bihun presence level of the represented tags to select
et al., 2007; Seo & Croft, 2007). which content is going to be shown next. This
The use of tag clouds for content recommen- layered design has evolved throughout multiple
dation has been described by Pessemier iterations and its final architecture is represented
(Pessemier, Deryckere, & Martens, 2009). Tag in Figure 2.
clouds are generated from user ratings to create
a form of personal profile. These tag clouds are 4.1. Place Sub-System
then used to recommend movies to that person.
In our work, we also suggest the use of tag clouds A place model combines the place-making role
for recommendation purposes, but in our case of a place owner with the ability to allow the
this corresponds to a place or situation profile, multiple visitors to also exert some influence on
and not to the profile of a single individual. This the characteristics of place. In our model, a tag
related work demonstrates how tag cloud can cloud was used as a shared and evolving view
indeed support many roles outside their original of the expressions of interest made by the place
context. However, the use of tag clouds as a situ- owner and the people around the display.
ated representation of place that drives the content The place tag cloud is first created by the place
selection on a public displays remains to the best owner. Place-making parameters allow a place
of our knowledge a novel approach. owner to provide additional characterization and


Smart Displays in Ambient Intelligence Environments

Figure 2. System architecture

specify adaptation boundaries for a tag cloud. cloud. Excluding these words is already a common
Even though the tag cloud is expected to emerge procedure in traditional tag clouds, but in this case
from interaction, these parameters provide a way we may also want to use this to prevent abusive
for aligning the display behaviour with the general or offensive keywords from making it into the
expectations of appropriateness of the display tag cloud. We also have a list of seed keywords
owner and its place-making objectives. The main that serve to initialise the tag cloud and maintain
part of these specifications is a set of keyword a number of place keywords when there are not
lists that enable some control of the tags in the enough keywords being generated. Seed tags are
tag cloud. The first is a blocked words list. This defined with a minimum popularity value that
list may include words that are very common in a determines how visible they remain when other
given language, but not very meaningful in a tag tags begin to emerge. A SeedsOnly parameter


Smart Displays in Ambient Intelligence Environments

can be used to determine that a particular tag should be able to consider the generic relevance
cloud will only accept seed words. This works of a source, in the sense of popularity, and also
as a white list that restrains the accepted words its timeliness, in the sense of how up to date is
to those on the list and may be useful to promote the respective information.
aggregation around thematic tag clouds, e.g. a Using keywords from the place tag cloud we
tag cloud with sport teams, emoticon symbols, recur to a feeds aggregator for searching sources
or music styles. Finally, there is also a list of according to the needs of the place model. The
contextual keywords that can be used to provide result is a large set of sources without any relevance
additional context to the words in the tag cloud. criterion. We then apply a relevance algorithm to
For example, Sports and Football could be added promote sources that rank higher in popularity.
to a tag cloud representing football teams. This This popularity is independent of the usage con-
is particularly important if the tag cloud is to be text where the content is consumed and is based
used for selecting web content. on generic measurements such as the percentage
The tag cloud will then evolve with the continu- of all Internet users who visit a given site or the
ous stream of words being generated by various traffic to the site. We use Alexa search engine to
types of implicit and explicit interactions, such obtain the traffic rank, a measure obtained from
as Bluetooth names, Obex exchanges or SMS/ Alexa Toolbar users. Another important measure
MMS messages. In our prototype, place visitors of feed relevance is the number of users subscribed
could publish their own tags into the tag cloud, by to the feed. Higher values of subscriptions denote
including tag commands in their Bluetooth device higher feed interest. Through the Newsgator API,
name, as described in (Jos, Otero, Izadi, & Harper, we get the number of users that subscribe each of
2008). Tags that have just been generated have a the sources. Because both traffic rank and feed
special meaning because they correspond to the subscriptions have distinct numeric domains they
presence or interaction events created by people need to be normalized. We do that by defining ac-
who are now in the vicinity of the display. The ceptance thresholds for both measures. Sigmoid
presence of tags represents an additional dimen- functions are used for characterizing acceptance
sion that is not normally included in traditional tag intervals and provide a smooth interpolation be-
clouds, but may be key to interpret the immediate tween the limits of those intervals. Both traffic rank
relevance of those tags in content selection, possi- and number of subscribers are thus combined in a
bly favouring tags that are currently present, albeit single function that determines the most relevant
less popular, instead of popular tags that no one sources for each of the tags associated with a place.
is currently generating. The tag cloud resulting
from combining the initial specification by the 4.3. Scheduler Sub-System
place-owner with the continuous stream of words
originating from presence and interaction provides The scheduler sub-system decides which content to
the evolving and dynamic representation of place present next on the display. This decision involves
that will be used as the basis for content selection. two steps. The first is to select which of the tags in
the place tag cloud will be used next for content
4.2. Selector Sub-System presentation. For each tag, there is information
about the respective popularity and whether or
The selector sub-system is responsible for au- not it is currently present, i.e. someone who is
tonomously finding and selecting relevant content now around the display is announcing that tag.
sources from the Internet. To fully benefit from Additionally, there is also a list of recently used
the wealth of sources on the Internet, the selector tags. When selected for presentation, a tag is then


Smart Displays in Ambient Intelligence Environments

placed on this list for a number of iterations to public deployment to assess its overall opera-
prevent it from being successively selected. The tion in a real-world scenario. This diverse set of
algorithm selects the most popular tag among those evaluation approaches enabled us to progressively
who are present and not in the waiting queue, or improve architectural designs and gain a broad
if none is present, the most popular tag not in the understanding of the key issues involved.
waiting queue. The first study addressed relevance from the
The second step is the selection of which of perspective of timeliness (F. R. Ribeiro & Jos,
the content feeds associated with the selected 2009b). The goal was to understand the key crite-
tag will be displayed next. This decision is based ria for evaluating the timeliness of content across
on a multi-criteria utility function that considers several types of dynamic sources and results have
three parameters: timeliness, content structure shown that the users perception of timeliness was
and scheduling history. Timeliness represents the properly represented by the concept of timeliness
measure of the temporal pertinence of content, i.e. as supported by the framework. The results sug-
how recent are the last updates. Content structure gest a reasonable match between our model and
is an empirical measure of the appropriateness of the users perspectives on timeliness and also
the content in the feeds for presentation on a public show that the model is able to make comparative
display. This is not related with the semantics or calculations of timeliness for different types of
the quality of content, but only with the fact that, dynamic source. The appropriateness and accu-
in certain cases, the structure of content can make racy of the method for obtaining relevant sources
it inadequate for presentation in a public display, based on simple keywords from the place model
either because it is link-intensive, because the was evaluated in a second study (F. R. Ribeiro
text is too long, or for some other similar reason. & Jos, 2009a, 2009b). The results have shown
For example, we analyse the length of the text, that keywords can be very effective in driving
the content language, the number of links found user-generated content, but they often need to
in the text or the number of image links in the be complemented with contextual information
feeds. Scheduling history considers the previous that disambiguates their semantics. The ability
presentation of similar content. The scheduler of the framework to autonomously select from
should be aware of the recently presented content web sources the content deemed more relevant
and avoid presenting content that is very similar, according to a dynamic place model was also
even if supplied from distinct sources. This is evaluated (F. Ribeiro & Jos, 2013). The place
very common with news sources, which may be model was based on a tag cloud that combined
showing very similar headlines. A combination content suggestions expressed by multiple place
of these parameters, jointly with the scheduler visitors with those expressed by the place owner.
behaviour configuration, supports the scheduler Results of this experiment indicate that people
decisions of what to present at each moment. perceive the content autonomously selected from
the tags as being relevant to the context where
4.4. Evaluation the display is situated. They also recognized and
understand the sensitivity of the system to their
The evaluation of this adaptive content system demands and how their interactions influenced
comprised a set of smaller user studies and a final the display behaviour.


Smart Displays in Ambient Intelligence Environments

5. DISCUSSION AND CONCLUSION to dynamically integrate the preferences of both

place owner and place visitors into a single rep-
This chapter has discussed the overall issue of how resentation, and thus generate a representation
to support adaptive content selection for public of the social environment as a whole instead of
displays. We have explored different adaptation each individual interest at a time. Still, the use of
strategies, particularly those based on context- tag clouds as place models is obviously a limited
awareness and content recommendation. approach from the perspective of descriptive rich-
Our assessment of the context-aware approach ness and we are not suggesting that they are an
suggests that context per se was not enough to appropriate model for other place representation
support adaptive content selection, in part due scenarios. However, their simplicity and their
to the limited context information available, but visual nature may have been an important element
also largely because of the challenges involved in in the positive results obtained with our system.
specifying or inferring generic context adaptation Through the use of the tag clouds, we have made a
rules for matching context situations with the key part of the system model explicit and visible,
utility of particular content items. More context allowing people to reason about it and actuate
information, and possibly more work on rule for- according to their goals. It also enabled people
mulation, could probably lead to better results, but to perceive interactions being generated by other
our subsequent work with the tag clouds would nearby users and observe the respective effect on
also highlight how a more simple approach, albeit the system. This ability to build an understanding
one people can reason about, may also have impor- of what was happening and interpret the system
tant advantages. We have also identified obvious behaviour has been suggested in the interviews as
similarities between content adaptation in public something that may have contributed to increase
displays and recommendation systems. However, user acceptance, even when the content shown
there are also clearly different assumptions in both was not perceived as the most appropriate option.
problems that severely limit the direct application We consider this to be an important finding, but it
of existing recommendations techniques. remains an interesting research topic to quantify
Based on these observations, we have devel- this effect and be able to understand how does
oped and evaluated a novel content adaptation this ability to reason about what is happening in
system for public displays that uses a shared the subjective perception that people formulate
and public place profile in the form of a place about the system behaviour compares against
tag cloud to combine the multiple interests of other approaches potentially more effective and
the place owner and place visitors. Overall, the sophisticated, but less open to user inspection.
positive results obtained during the evaluation
suggest that this is a viable approach to the prob-
lem of content adaptation for public displays. The REFERENCES
results showed that place visitors recognize the
sensitivity of the system to their demands and that Alt, F., Balz, M., Kristes, S., Shirazi, A. S., Men-
a place tag cloud was able to provide an adequate nenoh, J., Schmidt, A., et al. (2009). Adaptive
representation of place. User Profiles in Pervasive Advertising Environ-
Despite their simplicity, tag clouds have re- ments. Paper presented at the European Confer-
vealed to be a promising solution to our specific ence on Ambient Intelligence. Salzburg, Austria.
scenario. Their main advantage is their ability doi:10.1007/978-3-642-05408-2_32


Smart Displays in Ambient Intelligence Environments

Arguello, J., Elsas, J. L., Callan, J., & Carbonell, Jos, R., Otero, N., Izadi, S., & Harper, R. (2008,
J. G. (2008, March 30-April 2). Document Rep- October-December). Instant Places: Using
resentation and Query Expansion Models for Bluetooth for Situated Interaction in Public Dis-
Blog Recommendation. Paper presented at the plays. IEEE Pervasive Computing, 7(4), 5257.
International Conference on Weblogs and Social doi:10.1109/MPRV.2008.74
Media. Seattle, WA.
Jos, R., Pinto, H., Silva, B., & Melro, A. (2013).
Bihun, A., Goldman, J., Khesin, A., Marur, V., Pins and posters: Paradigms for content publication
Morales, E., & Reynar, J. (2007). US Patent No. on situated displays. IEEE Computer Graphics
20070061297 A1. US Patent & Trademark Office. and Applications, 33(2), 6472. doi:10.1109/
MCG.2013.16 PMID:24807941
Das, A. S., Datar, M., Garg, A., & Rajaram, S.
(2007). Google news personalization: scalable on- Kubitza, T., Clinch, S., Davies, N., & Lang-
line collaborative filtering. Paper presented at the heinrich, M. (2012). Using mobile devices to
16th international conference on World Wide Web. personalize pervasive displays. Mobile Comput-
Banff, Canada doi:10.1145/1242572.1242610 ing and Communications Review, 16(4), 2627.
Dourish, P. (2004). What we talk about when
we talk about context. Personal and Ubiquitous Leahu, L., Sengers, P., & Mateas, M. (2008, Sep-
Computing, 8(1), 1930. doi:10.1007/s00779- tember 21-24). Interactionist AI and the promise
003-0253-8 of ubicomp, or, how to put your box in the world
without putting the world in your box. Paper
Gonzlez, G., Rosa, J. L. d. l., Dugdale, J., Pavard,
presented at the 10th International Conference
B., Jed, M. E., Pallamin, N., . . . Klann, M. (2006).
on Ubiquitous Computing. Seoul, South Korea.
Towards Ambient Recommender Systems: Results
of New Cross-disciplinary Trends. Paper presented
at the Workshop on Recommender Systems in McCarthy, J. F., Costa, T. J., & Liongosari, E. S.
European Conference on Artificial Intelligence (2001). UniCast, OutCast & GroupCast: Three
on Recommender Systems. Riva del Garda, Italy. Steps toward Ubiquitous Peripheral Displays.
Paper presented at the International Confer-
Huang, E. M., Koster, A., & Borchers, J. (2009).
ence on Ubiquitous Computing. Atlanta, GA.
Overcoming Assumptions and Uncovering Prac-
tices: When Does the Public Really Look at Public
Displays? Paper presented at the 6th International McCarthy, J. F., McDonald, D. W., Soroczak, S.,
Conference on Pervasive Computing. Sydney, Nguyen, D. H., & Rashid, A. M. (2004). Augment-
Australia. ing the Social Space of an Academic Conference.
Paper presented at the CSCW. Chicago, IL.
Izadi, S., Brignull, H., Rodden, T., Rogers, Y.,
& Underwood, M. (2003, November). Dynamo: McDonald, D. W., McCarthy, J. F., Soroczak,
A public interactive surface supporting the co- S., Nguyen, D. H., & Rashid, A. M. (2008).
operative sharing and exchange of media. Paper Proactive Displays: Supporting Awareness in
presented at the Symposium on User Interface Fluid Social Environments. ACM Transactions
Software and Technology. Vancouver, Canada. on Computer-Human Interaction, 14(4), 131.
doi:10.1145/964696.964714 doi:10.1145/1314683.1314684


Smart Displays in Ambient Intelligence Environments

Michelis, D., & Mller, J. (2011). The Audience Ribeiro, F. R., & Jos, R. (2009a). Place-aware
Funnel: Observations of Gesture Based Interaction content selection from dynamic web sources for
With Multiple Large Displays in a City Center. public displays. In Proceedings of 5th Interna-
International Journal of Human-Computer In- tional Conference on Signal-Image Technology &
teraction, 27(6), 562579. doi:10.1080/104473 Internet-Based Systems (pp. 302-309). Marrakech:
18.2011.555299 IEEE Computer Society Press. doi:10.1109/
Morales-Aranda, A. H., & Mayora-Ibarra, O.
(2007, June 19 -25). A Context Sensitive Public Ribeiro, F. R., & Jos, R. (2009b). Timeliness for
Display for Adaptive Multi-User Information dynamic source selection in situated public dis-
Visualization Paper presented at the Third In- plays Paper presented at the 5th Int. Conference
ternational Conference on Autonomic and Au- on Web Information Systems and Technologies.
tonomous Systems. Athens, Greece. doi:10.1109/ Lisbon, Portugal.
Russell, D. M., & Sue, A. (2003). Large Inter-
Mller, J., Kruger, A., & Kuflik, T. (2007). active Public Displays: Use Patterns, Support
Maximizing the Utility of Situated Public Displays. Patterns, Community Patterns. In K. OHara, M.
Paper presented at the Adjunct Proceedings of Perry, E. Churchill, & D. Russell (Eds.), Public
User Modeling. Corfu, Greece. doi:10.1007/978- and Situated Displays (Vol. 2, pp. 317). Dor-
3-540-73078-1_52 drecht, The Netherlands: Springer Netherlands.
Payne, T., David, E., Jennings, N. R., & Sharifi,
M. (2006). Auction Mechanisms for Efficient Ad- Seo, J., & Croft, W. B. (2007). UMass at TREC
vertisement Selection on Public Displays. Paper 2007 Blog Distillation Task. Paper presented at
presented at the European Conference on Artificial the Text Retrieval Conference. Gaithersburg, MD.
Intelligence. Riva del Garda, Italy.
Wang, M., Boring, S., & Greenberg, S. (2012).
Pessemier, T. D., Deryckere, T., & Martens, L. Proxemic peddler: a public advertising display
(2009). Context aware recommendations for that captures and preserves the attention of a
user-generated content on a social network site. passerby. Paper presented at the 2012 Interna-
Paper presented at the 7th European Interactive tional Symposium on Pervasive Displays. Porto,
Television Conference. Leuven, Belgium. Portugal. doi:10.1145/2307798.2307801
Ribeiro, F., & Jos, R. (2013). Smart content Weiser, M. (1993). Some Computer Science Issues
selection for public displays in ambient intel- in Ubiquitous Computing. Communications of the
ligence environments. International Journal of ACM, 36(7), 7584. doi:10.1145/159544.159617
Ambient Computing and Intelligence, 5(2), 3555.
Zhu, M., Zhang, D., Zhang, J., & Lim, B.Y.
(2007). Context-Aware Informative Display. Pa-
Ribeiro, F. R., & Jos, R. (2007). Proactive Sched- per presented at the International Conference on
uling for Situated Displays. Paper presented at the Multimedia and Expo. Beijing, China.
Workshop on Ambient Intelligence Technologies
and Applications. Guimares.


Smart Displays in Ambient Intelligence Environments

ADDITIONAL READING Linden, T., Heikkinen, T., Ojala, T., Kukka, H.,
& Jurmu, M. (2010). Web-based framework for
Alt, F., Shirazi, A. S., Kubitza, T., & Schmidt, A. spatiotemporal screen real estate management of
(2013). Interaction techniques for creating and interactive public displays. Paper presented at the
exchanging content with public displays. Paper Proceedings of the 19th international conference
presented at the Proceedings of the SIGCHI Con- on World wide web, Raleigh, North Carolina,
ference on Human Factors in Computing Systems, USA. doi:10.1145/1772690.1772901
Paris, France. doi:10.1145/2470654.2466226
Mller, J., Alt, F., Schmidt, A., & Michelis, D.
Cardoso, J., & Jos, R. (2009). A Framework for (2010, October 2529). Requirements and Design
Context-Aware Adaptation in Public Displays. In Space for Interactive Public Displays. Paper pre-
R. Meersman, P. Herrero, & T. Dillon (Eds.), On sented at the International Conference on Multime-
the Move to Meaningful Internet Systems: OTM dia Firenze, Italy. doi:10.1145/1873951.1874203
2009 Workshops (Vol. 5872, pp. 118127). Berlin:
Springer Berlin Heidelberg. doi:10.1007/978-3- Mller, J., Exeler, J., Buzeck, M., & Krger, A.
642-05290-3_21 (2009). ReflectiveSigns: Digital Signs That Adapt
to Audience Attention. In H. Tokuda, M. Beigl, A.
Clinch, S., Davies, N., Kubitza, T., & Schmidt, Friday, A. J. B. Brush, & Y. Tobe (Eds.), Perva-
A. (2012). Designing application stores for sive Computing (Vol. 5538, pp. 1724). London:
public display networks. Paper presented at the Springer Berlin Heidelberg. doi:10.1007/978-3-
Proceedings of the 2012 International Sympo- 642-01516-8_3
sium on Pervasive Displays, Porto, Portugal.
doi:10.1145/2307798.2307808 OHara, K., Lipson, M., Jansen, M., Unger, A., Jef-
fries, H., & Macer, P. (2004). Jukola: democratic
Davies, N., Langheinrich, M., Jos, R., & Schmidt, music choice in a public space. Paper presented
A. (2012). Open Display Networks: A Communi- at the Proceedings of the 5th conference on De-
cations Medium for the 21st Century. Computer, signing interactive systems: processes, practices,
45(5), 5864. doi:10.1109/MC.2012.114 methods, and techniques, Cambridge, MA, USA.
Elhart, I., Langheinrich, M., Davies, N., & Jos, doi:10.1145/1013115.1013136
R. (2013, 18-22 March). Key Challenges in Ap- Storz, O., Friday, A., & Davies, N. (2006). Sup-
plication and Content Scheduling for Open Per- porting content scheduling on situated public
vasive Display Networks. Paper presented at the displays. Computers & Graphics, 30(5), 681691.
2013 IEEE International Conference on Pervasive doi:10.1016/j.cag.2006.07.002
Computing and Communications Workshops
(PERCOM Workshops), San Diego. doi:10.1109/
Konomi, Shinichi, Inoue, Sozo, Kobayashi,
Takashi, Tsuchida, Masashi, & Kitsuregawa, Ma- Ambient Intelligence: Digital environments
saru. (2006). Supporting Colocated Interactions that proactively supports people in their everyday
Using RFID and Social Network Displays. IEEE lives.
Pervasive Computing, 5(3), 48-56. doi: 10.1109/ Context Awareness: Systems that sense their
mprv.2006.60 environment and adapt their behavior accordingly.


Smart Displays in Ambient Intelligence Environments

Interactive Systems: Systems that accept input Ubiquitous Computing: A paradigm where
from users and react to these inputs. the environment becomes intelligent by integrating
Recommender Systems: A system that is able information processing capabilities into everyday
to recommend new content that may be of inter- life objects and places.
est to an entity, based on information about the User Generated Content: Content that is
preferences, past actions or properties of the entity. created by consumers or end-users.
Smart Displays: A display situated in a public
place that is able to present the right information
to the right users, at the right time and in the
right way.