Sie sind auf Seite 1von 522


and Innovation
Proceedings 2018
ENoLL Office
Pleinlaan 9
B-1050 Brussels

T: +32 2 614 85 47
F: +32 2 629 17 00
This report is a compilation of the papers presented
between the 22nd and 23rd of August 2018, in Geneva,
Switzerland, as part of the OpenLivingLab Days 2018
conference. The publications here contained are a
result of the double-blind review and evaluation
procedure launched on February of 2018 as part of the
“Call for Papers” responding to the theme of the
OpenLivingLab Days 2018 conference: “Living Labs
and the Sustainable Development Goals: From
Theories to Practice”

This year’s “Call for Papers” encourage contributions

from three different paper categories to encourage a
diverse participation of actors: Research Papers –
scientific research; Innovation Papers – practitioner
case studies; and Research in-progress – scientific
research not yet completed.

ISBN (e-book): 9789082102789
© 2018 ENoLL - European Network of Living Labs
All rights reserved

Review Panel
Anna Ståhlbröst – Botnia Living Lab

Steering Committee
Pieter Ballon – ENoLL Secretary

Evaluation Committee
Abdorasoul Habibipour – Botnia Living Lab
Alen Keirnan – RMIT
Ana Salas – Consorcio Fernando de los Rios
Anna Ståhlbröst – Botnia Living Lab
Annabel Georges - iMEC
Anotonis Billis - Thessaloniki Active & Healthy Ageing Living Lab
Carolyn Hassan – Knowle West Media Centre
David Guimont – Living Lab en Innovation Ouverte
Diana Chronéer - iMEC
Dimitri Schuurman - iMEC
Francesco Molinari – Lunigiana Amica
Gareth Priday – Australian Network of Living Labs
Gavin McCrory – Chalmer University of Technology
Hanna-Greta Puurtinen – TAMK Living Lab
Idoia Muñoz – GAIA & Bird Living Lab
Johan Wenngren – Botnia Living Lab
Lynn Coorevits - iMEC
Mikko Julin – Laurea University of Applied Sciences
Natasja Van Buggenhout - VUB
Ömer Onur - Başakşehir Municipality / Başakşehir Living Lab
Parvaneh Westerlund – Botnia Living Lab
Shaofu Huang – University of Bristol
Sonja Pedell – Future Self and Design Living Lab
Tanguy Coenen – iMEC

ENoLL Office Contributors

Clara Mafé
Table of Contents
Top 7 Papers selected by the Evaluation Committee
Constraints on upscaling and social inclusion in smart city living lab
experiments and ways to anticipate them: lessons from four “smarter” labs by
Francesca Cellina, Roberta Castri, Mario Diethart, Thomas Höflehner, Nicola Da
Schio, Marc Dijk........................................................................................................ 11
Innovation Management in Living Lab projects: the Innovatrix Framework by
Dimitri Schuurman, Aron-Levi Herregodts, Annabel Georges, Olivier Rits ............... 26
International Innovation Sprint Bridging the Sustainability Gap between
Metropolitan Core and Peripheries by Tuija Hirvikoski, Kaisla Saastamoinen
and Mikael Uitto ........................................................................................................ 43
Learning with Community: Developing Citizen-Led Housing by Shaofu Huang,
Ruth Deakin Crick, Melissa Mean, Carolyn Hassan, Colin Taylor ............................ 57
Participatory software development via Living Labs: A case of Deajeon
by Hee-Sook. Yoo, Su-jin. Jung, Seong-gyu. Park, Miran. Cho................................ 77
Transnational piloting for smooth internationalization of health-tech start-ups
by Päivi Haho, Metropolia and Virpi Kaartti .............................................................. 89
User needs and expectations as a challenging factor for successful living lab
research initiatives involving older adults: The DDRI Experience by Tiziana C.
Callari, Louise Moody, Nikki Holliday, Ed Russell, Janet Saunders, Gill Ward, Julie
Woodley ................................................................................................................. 104

Actors motivations, needs and expectations

Living labs as instrument for societal change: the role of intermediary actors
and activities by Jos van den Broek, Timo Maas, Isabelle van Elzakker, Jasper
Deuten .................................................................................................................... 121
Models for Living Lab’s sustainability. Evidences from Italy and the Netherlands
Edoardo Gualandi and Luca Leonardi .................................................................... 134
Motivational Goals for using Electronic Health Record Applications by Rachel
Burrows, Sonja Pedell, Leon Sterling, Tim Miller, Antonette Mendoza ................... 151
Smart Campus: a route using 4G and 5G by Esmat Mirzamany and Joe Barrett
............................................................................................................................ 161
Spectrum Analysis Digital Arts-and-Culture Assets and the Future of Co
Creating Decentralised Technologies by Olivier Zephir, Soenke Zehle, Olivier
Buchheit ................................................................................................................. 173
The Austrian Real-World Laboratories - How to manage multi-stakeholder
engagement in the wide field of mobility research by Lina Mosshammer, Doris
Wiederwald ............................................................................................................. 187
The Developing Process of the Digital Game Which Supports Well-being of
Small Children by Päivi Marjanen, Janne Pirttikoski, Vesa Valkonen and Viktoria
Tiainen.................................................................................................................... 200

Governance and process-related areas

“Daegu Living Labs” as City Innovation Platform by Hee Dae Kim ................. 214

Listening to locals? Question-oriented approach in Field Lab Amsterdam East

by Sandra Bos …………………………………………………………………………..230
Living Labs – an Approach for achieving Sustainable Change in City Logistics
by Nina Nesterova, Hans Quak, Tariq van Rooijen ............................................... 238
Living-Lab-as-a-Service: Exploring the market and sustainability offers 
Labs in Germany by Justus von Geibler, Julius Piwowar, Annika Greven .......... 252
Schools4energy: a living laboratory for energy awareness in schools by
Carmelina Cosmi, Filomena Pietrapertosa, Giuliano Sarricchio, Michele Giordano,
Monica Proto, Marco Tancredi, Monica Salvia ...................................................... 272
The Circle of Mediators: Towards a governance model for tackling
sustainability challenges in a city by Anne Äyväri, Annukka Jyrämä, Tuija Hirvikoski
............................................................................................................................. 287
The management of information and knowledge flows in Urban Living Labs: a
constructivist approach by Carolina Campalans ................................................ 306

Theoretical and Methodological Challenges

Applying the Living Lab Approach for the Design of Public Spaces– A Living
Lab Case Study by Sonja Pedell, Gareth Priday, Alen Keirnan, Flavia Marcello, and
Andrew Murphy ..................................................................................................... 320

Comparison of Health and Wellbeing Living Lab Business Models – Preliminary

result based on Business Model Canvas Evaluation by Teemu Santonen, Mikko
Julin ........................................................................................................................ 339
Developing a mutualized R&D for network organizations based on Living Lab
methods: the transformation of Sion military airport into a commercial airport
by Emmanuel Fragnière, Benjamin Nanchen, Marine Héritier, Randolf Ramseyer,
Magali Dubosson and Joelle Mastelic .................................................................... 358

Evaluating the Performance of the Regional Living Lab Concept on Integrated

Sustainable Energy Planning by Giannouli Ioanna, Tourkolias Christos, Georgiou
Paraskevas, Cantero Celada Sergio, Fernández Maroto Miguel ............................ 375
Evolving Community Living Labs through Dual Safety Knowledge Circulation
Systems to Prevent Injuries in Children by Mikiko Oono, Yoshifumi Nishida, Koji
Kitamura, Kimiko Deguchi ...................................................................................... 396
Initiating Participation: Methodological and Practical Challenges of Living Lab
Projects for Early Stages of Research and Development by Andreas Bischof,
Albrecht Kurze, Sören Totzauer, Michael Storz, Kevin Lefeuvre, Arne Berger....... 407

Living Lab 65+ – Participatory testing of technical assistance systems in the
natural home environment of senior citizens by Misoch, S., Lehmann, S., Pauli, C.,
Hämmerle, V., Guggenbühl, U. & Konstantas, D. ................................................... 422
Living Lab Research: A State-of-the-Art Review and Steps towards a Research
Agenda by Abdolrasoul Habibipour ....................................................................... 432
Making open innovation work for Sustainable Development Goals:
Sustainability-orientation and assessment based on the SDG-Check by Justus
von Geibler, Julius Piwowar, Annika Greven .......................................................... 448
Solar Living Lab: implementing sustainable experimentation process,
responsible innovation and community engagement in a solar energy research
facility by Silvana Di Bono, Filippo Paredes and Fabio Maria Montagnino. ........... 462
The benefits and challenges of co-creation with seniors: an interdisciplinary
social innovation project designed to improve quality of life by D. Campisi, N.
Nyffeler, D. Roulet Schwab, V. Le Fort, L. Bergeron .............................................. 481
The Island Approach (TIA): a suited projective technique for evaluating a bevy
of attributes by Natasja Van Buggenhout, Wendy Van den Broeck, Iris Jennes .. 496

This publication is a collaborative effort of several
individuals representing the European Network of
Living Labs and its network members.

Constraints on upscaling and social inclusion in
smart city living lab experiments and ways to
anticipate them: lessons from four “smarter” labs
Francesca Cellina1*, Roberta Castri1, Mario Diethart2, Thomas
Höflehner2, Nicola Da Schio3, Marc Dijk4

* Corresponding author
1 Universityof Applied Sciences and Arts od Southern Switzerland (SUPSI), ISAAC
2 University of Graz, Regional Centre of Expertise Graz-Styria

3 VUB, Cosmopolis, Department of Geography, Vrije Universiteit Brussel,

4 Maastricht University, International Centre for Integrated Assessment &

Sustainable Development (ICIS)

Category: Research in-progress

The ‘Smart City Living Lab’ is an emerging approach in European cities, referred to
projects devised to design, test and learn from innovative socio-technical practices
(i.e. ‘new ways of doing something’) in real-time and in urban contexts, with a diversity
of stakeholders. However, successful implementation of new practices in the reality
of LLs does not guarantee the large-scale adoption required to reach their full effect
in resource efficiency. Also, there is a risk of exclusion of social groups not matching
the required ‘smart citizen’ profile. Acknowledging such drawbacks, we focused on
how to foster upscaling and avoid social exclusion, developing a novel approach that
anticipates such problems, and testing it through ‘smarter’ LL experiments
addressing mobility-related topics in four European cities. In this paper we summarize
the key constraints we have identified and comment on the strategies we
implemented in our Living Lab activities in order to cope with them.

Keywords: living lab; inclusion; upscaling; smart city; mobility.


The ‘Smart City Living Lab’ is an emerging approach in European cities, referred to
projects devised to design, test and learn from innovative socio-technical practices
(i.e. ‘new ways of doing something’) in urban contexts, with a diversity of
stakeholders. A Living Lab (LL) was defined as an institutional environment for open
innovation that supports experimentation with real users in real contexts (Folstad
2008; Hillgren 2013). It may be organized in a variety of ways (long-term or short-
term, independent from or embedded in the municipal organization (Kemp and Scholl
2016), provider-driven or user-driven (Leminen 2013)), but commonly characterized
by situated experimentation, diversity and participation, learning, and evaluation.

The interaction of the social and technical dimensions often makes urban
infrastructure quite resistant to change and requires specific attention when new
practices are to be introduced (Hommels 2005, 2010). The current approach to LLs
focuses on small-scale performance tests and technology-user interactions, mostly
neglecting the larger social-institutional context (Karvonen & van Heur, 2014;
Karvonen et al., 2013). Therefore, successful implementation of new practices in the
reality of LLs is not a warrant for the large-scale adoption required to reach their full
effect in resource efficiency. Another limitation of the current LL approach to smart
urban technologies is its focus on ‘smart citizens’ as users and partners, namely
citizens with both the cognitive and material resources to consume and co-produce
the smart services of the smart city. Citizens lacking these resources will normally not
be included as users and co-creators in LLs, nor are they likely to be able to use the
smart services once these are implemented on a larger-scale (Dutilleul et al., 2010).
The consequence is not only limited adoption and use of these smart technologies
but also social inequality and exclusion (Evans & Karvonen, 2014).

In the ‘SmarterLabs’ project we focused our attention on how to foster upscaling and
avoid social exclusion, developed a novel approach that anticipates such problems
and tested it through ‘smarter’ LL experiments addressing mobility-related topics in
four cities: Bellinzona (CH), Brussels (BE), Graz (AT), and Maastricht (NL). Action
research activity in such LLs is still going on and we are now performing final
evaluations of effectiveness. However, the elements collected so far allow us to
identify a set of recurrent constraints on inclusion and upscaling and to comment on
the strategies we implemented in order to cope with them. After a brief introduction
to our ‘smarter’ pilots (Section 2), in this paper we present the constraints we
identified and the lessons we gathered (Sections 3 – 14), concluding with
considerations on future work (Section 15).

Main constraints on social inclusion and upscaling

In the framework of the SmarterLabs project we performed a literature research,

developed a retrospective analysis on mobility and land planning processes and
directly engaged in action research activity by designing, managing and evaluating
four ‘smarter’ LL processes. In Bellinzona citizens were involved in co-designing a
smartphone app aimed at promoting individual behaviour change and rewarding those who
reduce car use. In Brussels citizens were involved in participatory measurements of air
quality, with the aims of increasing awareness on the impact of urban traffic flows on
local air pollution and co-designing more sustainable mobility scenarios. In Graz
citizens and local stakeholders were engaged in the ‘smart’ redesign of Griesplatz, a
large square in the centre of the City, especially important as a traffic hub. Similarly,
in Maastricht a series of focus group meetings exploiting a web-based design tool
were held to engage stakeholders in co-designing the renovation of the central station
Table 1 summarizes the constraints we identified in our LL pilots. We refer to
upscaling as the emergence and expansion of an innovative practice (i.e. a new way
of doing something) in the particular urban area and to social exclusion as a
multidimensional, multi-layered and dynamic deprivation that people may suffer
because of new urban practices. Note that we understand social exclusion as a key
constraint affecting upscaling itself: addressing constraints on social inclusion is a
pre-condition to effective upscaling.

Table 1 Constraints precluding social inclusion and upscaling.

Constraints on social inclusion Constraints on upscaling

Exclusion from the Exclusion in the Related to LL Related to
LL LL design context
Citizen’s lack of Reproducing Limited learning Low stakeholder
financial, intellectual existing power receptiveness
and human resources structures inside
the Lab
Mismatching goals Poor timing Low institutional
between the citizens receptiveness
and the Lab
Overlooking people Wait-and-see High institutional
outside lab context attitude fragmentation
Sticky urban
Neglecting effects
outside project

Citizen’s lack of financial, intellectual and human resources

Living Labs can be complex and long lasting. To participate meaningfully, citizens
need to have time and energy, a certain level of understanding of the discussion and
sometimes also specific economic and intellectual resources. People with no, low or
very discontinuous revenues might be excluded from the Lab, since earning their
living can leave little space to other activities. Also, people with precarious
employment or residential conditions might lack the possibility to plan for long term
and therefore commit to participate in a Lab. People responsible for taking care of
elderly or children, as well as people working during non-office shifts might lack the
material time to join the Lab. Foreigners and new-comers can be excluded because
of their limited proficiency in the language. In addition, people lacking a minimum
understanding of the issue at stake or acquaintance with the technology used in the
Lab (e.g. because of low education level or age) are also at risk of exclusion. Socially
marginalized groups may tend not to participate in community initiatives due to a lack
of self-determination, of financial or educational resources, or both.

While it is virtually impossible for LLS to be inclusive of all relevant groups, it is

desirable to minimise exclusion. It is important to reflect on desired outcomes and
apply stakeholder and requirement analysis tools to identify potential types of
exclusion and adequate coping strategies. This exercise is essential in the design
phase, though it requires ongoing reflections at different stages of the Lab. Also,
involving all Lab participants (not only the ‘institutional’ initiators) in explicit reflections
concerning causes and outcomes of exclusion and seeking for solutions would be

For example, efforts to minimise exclusion were at the core of the Brussels Lab since
the beginning. Different adjustments were also made in progress, to cope with
unexpected circumstances. At very early stages, the organizers (one of the local
universities, and a network of neighbourhood committees) identified potential barriers
to inclusion and opted for establishing different sub-groups, precisely to include the
broadest variety of population. Throughout the process, regular outreach efforts were
made towards groups at potential risk of exclusion, also relying on a ‘focal person’
identified in each group. For instance, venues and schedules for each group were
strategically selected: for EU officers, meetings were convened in EU premises at
lunch time, for groups of parents and shopkeepers, small meetings were organised
in the early morning (just after leaving the children in school/before opening the shop),
for young professionals, meetings were organised at early evening in a central
neighbourhood. Several smartphones were purchased to ensure everybody could
still take part in the Lab, as well as tablets, used for demonstrative purposes. Less
acquainted people with smart technologies where dedicated more training time. In

some cases, however, these efforts were not enough to bridge the gap, resulting in
participants not using the technology.

Similarly, the City of Graz aimed to act in a district with challenging circumstances:
high proportion of migrants, various cultures and ethnics, education levels and
incomes below average. The strategy to reach out to marginalized groups such as
migrants, elderly people and children was to offer different formats of LL activities:
workshops, social safaris, online questionnaires, mental maps, etc. Lab organizers did
not wait for people to show up, but actively approached them on the street, literally
bringing the Lab to the people. By repeatedly offering possibilities for stakeholders to
participate and actively approaching them, over a long period of time also
marginalized groups were included.

Mismatching goals between citizens and the Lab

Due to their experimental character, LLs often struggle to define concrete goals,
especially in initial stages. Trial-and-error and learning-by-doing approaches can be
a challenge when it comes to communication. On the one hand, it is important to be
transparent and inform stakeholders and potential participants about LL objectives in
advance. On the other hand, participatory processes are more beneficial when a
common vision for the Lab is created together with participants. What happens in
practice is that stakeholders often feel information is lacking (due to little or wrong
communication) or – even worse – disagree with Lab objectives (due to not being
sufficiently involved in their creation). The latter strongly depends on the facilitators.
Aiming to involve a variety of people, special attention needs to be paid to their
individual demands and desires. Communication, including awareness raising
campaigns to inform people and establish a connection between them and the LL
facilitators, is the key to success throughout a Lab’s lifetime. Through a constant
exchange, expectations can be kept realistic and results are better accepted.

For example, the LL in Graz was initiated by the city government which aimed to
improve the quality of life in the traffic-dominated area of Griesplatz. Although
changes in traffic infrastructure were not supposed to be part of the participation
process, the residents around Griesplatz attributed the LL ambitious desires in that
aspect too. The divergent goals between citizens and the LL could be partly attributed
to legal circumstances that prevented rapid changes in infrastructure (i.e. concession
for bus operators). However, also communication turned out to be misleading, as
external policy-makers interfered with the information provided by the facilitators of
the LL. Whether they were justified or not, it was the task of the organizers to address
high expectations among the residents and clarify certain issues: they clearly pointed
out that the Lab itself was an approach to a solution of existing problems and not

another initiative to fight against – the Lab would facilitate dialogue between city
decision makers and stakeholders. To this purpose, various communication channels
were used, including newspapers, Facebook, public events and direct interaction with
people via the Lab’s district office.

Overlooking people outside Lab context

Living Labs are experiments situated in a specific geographic context, ranging from
a building block to a neighbourhood, a municipality or a whole urban area. While there
is a certain flexibility in choosing the scale within which to operate, any choice implies
the definition of boundaries excluding people living beyond them. Often, people living
outside or faraway the project context self-exclude themselves, relinquishing to join
the Lab either because of the effort to reach Lab meetings or because they do not
feel immediately concerned - though they might be impacted by the project.
When setting a LL, therefore, thorough reflections on the multiple scales relevant to
the LL, and on the actors that at any scales might be included/excluded, are needed.
Adequate logistic arrangements can help minimise exclusion. This includes
communicating LL purposes, adapting them and adjusting the frame. Overall,
constantly discussing with (potential) participants about the objectives and the frame
of the Labs can be particularly helpful in defining a shared vision, thereby increasing
motivations and obtaining a broader audience.

For example, in Brussels the place of residence was one of the most solid barriers to
broad inclusion. As indicated above, to minimise exclusion based on participants’
residence, different LL ateliers were held in different locations, depending on the
participants’ home and daily schedules. However, despite the outreaching efforts, the
Lab was eventually not successful in including participants from all neighbourhoods,
nor participants living outside the regional borders, many of whom are commuters
towards Brussels – thus largely contributing to, and being impacted by, air pollution
in the city. Main reasons for failing in engaging them were the lack of time and
resources to identify suitable locations at the urban periphery and their relatively
lower concern for the issue at stake (i.e. widespread perception that suburban living
is less impacted by air pollution). To complement for this shortcoming, constant
efforts of networking and coordination with other organisations were made, to share
good practices and lessons from the Lab: by experience-sharing with organisations
in nearby cities, the conditions were created for replication in other contexts.

Reproducing existing power structures inside the Lab

One fundamental aim of LLs is to establish a democratic structure that guarantees

that every voice is heard and considered. However, in practice, instead of achieving

real participation, various circumstances can lead to mere reproductions of the power
structures already existing in real life. This could be the result of deliberate
management in the LL, if run as an alibi activity. Or, LL organizers might not be aware
of the heterogeneity of stakeholders and precautions needed to provide any group
with equal opportunities.

To avoid reproducing existing power structures, first these need to be assessed by

carrying out a stakeholder analysis. Then LL organizers have to design a
communication strategy to address all identified target groups, applying tailor-made
methods for each of them. Flexibility in the use of methods is a key requirement (e.g.
not only conversation or only ICT tools). Also inviting people at various levels and
occasions and building trust and social cohesion plays an important role. Organizers
should facilitate development of activities along different tracks and allow each group
to adapt to their speed. Next to the methodology, also the locations should be neutral
and unbiased, providing access to everybody. If that is not possible, meetings should
change locations over time.

For example, the Lab in Graz involved residents, shop owners, bus operators, city
entities and politicians. All of them filled out certain roles that contained different
levels of power. Moreover, a couple of persons repeatedly ‘sabotaged’ events by
excessively raising their voices and acting as opinion leaders. Lab organizers aimed to blur
the borders between them, enabling each person to participate equally. This was
achieved by offering different formats of LL activities (social safaris, questionnaires,
mental maps, etc.) and carefully selecting locations: the city district office next to
Griesplatz remained a neutral place for diverse activities throughout the whole
project, complemented by outdoor activities in the district. These measures created
awareness and social cohesion among the people involved.

Limited learning

Living Lab processes are frequently run by actors already engaged with other
compelling duties, being them civil servants, for city-owned LLs, or voluntary citizens,
for civil society-owned LLs. Therefore, they often lack time or resources to perform
explicit monitoring of the lessons learnt throughout the process. Even when single
actors draw their assessments and conclusions, they often lack a comprehensive
view of the process, and therefore a comprehensive knowledge. If no single actor has
an overview of all options, mechanisms and impacts emerged during Lab activities,
limited transfer of learning is possible to future users, precluding upscaling.
Explicit learning strategies are needed, capable of capturing and monitoring
knowledge creation and transferring it to all actors. To this purpose, first goals and
ambitions of the actors need to be understood. Then, knowledge exchange can be

favoured through people-to-people real-life interactions (i.e. physical meetings),
which make learning more rewarding and comprehensive to all and also ensure tacit
knowledge to emerge.

For example, the Lab in Bellinzona was a pilot project, run on a voluntary, politically
non-binding base. On one hand, this favoured acceptance of the LL approach by the
City, but on the other hand it made also responsibilities and commitment by the City
to contribute to the participatory knowledge-sharing process less pressing. This made
the process of capitalizing on the ‘lessons-learnt’ from the Lab and integrating them
into the City’s policies more difficult.

Therefore, a learning strategy was explicitly designed, with the aim of monitoring
knowledge creation. The strategy included analysing the project’s impacts, assessing
the level of engagement and satisfaction by Lab participants, and reporting and
communication of results, both internally to all actors involved, as well as externally,
through local media. In parallel, it was avoided external experts driving a one-way learning
process, by defining ‘their problem’, providing ‘their knowledge and technology’, and
preparing ‘their solutions’. This would have meant limiting lab participants as testers of the
app functionalities, not assimilating the knowledge developed during the LL, nor transferring
it to future users. To avoid this, a user-centered approach was adopted and the app was
directly co-designed within the Lab meetings. This helped increasing intrinsic
motivation, enduring participation and learning and knowledge-sharing between

Wait-and-see attitude

It is not infrequent that Living Labs are managed as routine projects, with no special
attention to diffusion of knowledge and learning, during and after the pilot: either
upscaling effects are expected to occur by themselves or strategies are put into place
after the pilot ended.

At the beginning of Lab activities, what can reasonably be up-scaled should be

identified, and an upscaling strategy should be designed, together with the relevant
communication and dissemination measures – keeping it flexible and open to the
evolution of activities in the Lab, which, for their open and participatory nature, might
impose adjustments respect to initial plans. Such a strategy might be developed with
the actors engaged in the Lab, who might get engaged also for its practical
implementation. Also, the identified dissemination and communication activities need
to be tailored to the specific context where Lab results are up-scaled, by choosing
the right channels, time and language.

The Lab in Bellinzona is emblematic: initially, the process was approached as a
sequence of closed and separate steps: first, the app is developed, then a plan is
made to promote it to the population, finally it is assessed whether additional citizens
need to be engaged. Namely, no specific upscaling strategy would have been
devised by the City. Promoting app use to the population was initially expected to
follow a rather traditional communication plan: press conference and distribution of
information leaflets. No particular efforts would have thereafter been planned to
actively advertise the initiative and promote app use.

However, the very choice of engaging citizens in app co-design triggered their intrinsic
motivation and commitment, thus innately generating communication and dissemination
possibilities versus the outside. Therefore, the ‘multiplier effect’ triggered during LL
experiences thanks to committed participants was explicitly exploited: Lab
participants were actively engaged in promoting app use among their circle of family
and friends. Also, specific functionalities were explicitly included in app-design
(‘collective challenges’), with the aim of periodically launching them and continuously
attracting new citizens to join app use. The product of the Lab itself (the app), was
therefore endowed with an inbuilt mechanism to favour its diffusion and counteract
the ‘wait-and-see’ dominant approach.

Poor timing

‘Poor timing’ in the implementation of the LL precisely refers to a situation where

broader dynamics, namely the particular social, economic, cultural and political
conjuncture, are disregarded and the experiment is designed as if it takes place in a
vacuum. No immediate replication of LL best practices is likely to be successful
without adequate customisation and adaptation to local conjuncture. This includes
accounting for broader socio-economic, cultural and political considerations, ensuring
links with the existing public debate, with what communities consider their priorities
and with what stakeholders consider feasible.

In practice, avoiding ‘poor timing’ involves maintaining flexibility throughout the Lab,
to ensure that both its objectives and frame can be adjusted and continuously re-
defined by all actors. An important precondition is to place citizens at the core of the
process, as they are likely to have the most detailed understanding of the local
context. Also, it also requires active coordination with other societal developments
and initiatives regarding related contents. This can be done at different levels, from
simple information sharing to building bridges and identifying possibilities of
cooperation. Ensuring that Labs are well linked to the broader societal debate is also
a way to ensure that participants feel recognised, thereby strengthening internal

In Brussels, an effort was made to link the Living Lab with the broader public debate.
To begin with, the original frame around 'smart mobility' was immediately adapted by
LL initiators, so that air quality and people health were at the core. Soon after, the
organizers also engaged in open dialogue with all stakeholders active on the topic,
contributing to establishing a platform for discussion for all civic movements striving
for better air and a network of researchers working on air quality and citizen science.
Both efforts contributed to reaching a broad audience and ensuring the LL was part
of a broader discussion.

Low stakeholder receptiveness

Even though results produced within the Lab are aligned with the original plans and
expectations, the policy climate might no longer support adoption of the innovation
pursued in the Lab. Alternatively, outcomes of the Lab might not find consensus
beyond Lab participants. In both cases, Lab outcomes would lack support or
agreement by the population, as well of the political majority needed to activate the
envisioned upscaling measures.

To avoid this, Labs should open to participation as much and as early as possible, by
activating participatory processes already from the development of visions, selection
of methodologies and identification of actions to be performed. In doing so, ‘the
common’ should always be emphasized and already existing networks and coalitions
between groups of stakeholders should be exploited. Also, building relationships with
successful initiatives already developed by other actors would be beneficial.

For example, when the City of Maastricht started re-designing the station area, a LL
visioning workshop was organised around the vision for mobility in Maastricht in 2040.
Around thirty people, stakeholders from different backgrounds and interests
(residents, entrepreneurs, commuters, urban planners, mobility operators), came
together to picture a vision of the transportation system in 2040, with attention for
accessibility, affordability and quality of the living environment. Participants could
experience their own design in a 3D visualization model (SketchUp), learn about the
designs of other stakeholder groups, obtain feedback and then adjust their own
vision. Involving actors since the visioning level favoured later consensus.

Low institutional receptiveness

Sometimes barriers might be due to the lack of open-mindedness and receptiveness

by institutions and policy-makers. Decision-makers might in fact be unfamiliar with,

or not open to co-design approaches, believing that interaction with stakeholders
adds unneeded complexity to policy development. Favouring expert-driven ways of
thinking and agreement with powerful lobbies, policy-makers might not show (or
indeed not have) real commitment: in the best-case Lab outcomes would not get full
support from City government; in the worst case, the Lab itself would be an alibi

To cope with such constraints, early inclusion of policy-makers should be sought for.
Provided that activities in the Lab are adequately designed, namely that Lab
organizers show genuine commitment and give voice, role and responsibility to
diverse groups of citizens, civil society organizations and experts, policy-makers and
institutions might start appreciating the approach and its benefits. Then, it would be
a matter of repetition: once multiple successful pilot processes are carried out,
institutions and policy-makers will embrace approaches and processes, supporting
their outcome.

For example, the City of Bellinzona was formally owning the LL process; however,
they were unaware of the potential of participatory LL projects in supporting policy
development. Therefore, they lacked leadership and predominantly relied on advice
and superintendence by the local university. They mainly perceived the Lab as a
technology innovation testing ground: a single, small-scale, closed and controlled
process, aimed at developing and evaluating the mobile app prior to its roll-out at city-
level. In particular, City decision-makers tended to cling to authoritative governance
styles, rather than opening up to more consultative, cooperative or even facilitative
approaches (Ker Rault, 2008), mainly due to the fear of losing formal power and
responsibility on the decision. Their main concern was to avoid possible financial and
personal drawbacks and, inadvertently or not, the tendency was to keep the living
Lab in the policy periphery.

However, leadership can only be learnt through experience: providing first-hand

opportunities of experiencing public participation processes is a first start. Thus,
researchers involved in Lab organization tried to promote a new political culture by
ensuring the presence and active participation of representatives of the Municipality
(civil servants, politicians) in LL meetings. This helped getting local authorities and
decision-makers gradually acquainted with the concept that LLs may represent a
valuable learning-by-doing tool and a constructive and enriching means for reflection
on practices or policy.

Also, to favour Lab acceptance by decision-makers, the strategy was to focus at first
on an app development: practical and technologically oriented, this was perceived as
a low-conflict topic and therefore easily supported. Later on, capitalizing on the actor-
and context-dependent knowledge created while Lab participants were testing the

app and concretely experiencing new mobility behaviors, discussion in the Lab was
upscaled to policy-related topics regarding future mobility scenarios (‘What would we
need to make mobility more sustainable in Bellinzona?’). This way, also potentially
scaring, far-reaching and conflicting discussions were spontaneously introduced in
the Lab with the support of the institutions.

High institutional fragmentation

Even when policy-makers embrace the LL participatory approach, its outcomes might
suffer from limited diffusion due to fragmented institutional arrangements, which
hinder clear distribution of responsibility and effective cooperation between involved
city departments. Fragmentation into units and departments (‘silo compartments’)
within and between public administration institutions, in fact, makes both horizontal
and vertical dissemination of results rather difficult.

Transparency and collaboration between administrative units and organizations must

therefore be actively fostered. In Bellinzona, for example, administrative organization
at the City level was the main obstacle preventing diffusion of the LL approach to
other fields than mobility and institutionalization of new governance practices. The
strategy to overcome ‘silo compartments’ barrier was to actively engage councillors
and civil servants, instead of waiting for them to spontaneously express interest in
process or results. Thus, it was planned to invite them to attend LL meetings, in order
to personally experience how they work and the effort needed, and guess their
potential in addressing complex or conflictual topics. In the end, the envisioned
strategy was not put into practice, mainly due to ‘low institutional receptiveness’
mentioned above. However, this gap will at least partially be closed, by inviting
councillors and civil servants to a workshop aimed at presenting the approach and
discussing its opportunities and limitations, as emerged from final assessment of the
whole LL process.

Sticky urban assemblage

Changes in urban contexts are sometimes tricky to achieve, due to technical,

infrastructural, legal or financial aspects. There is an obduracy to urban assemblages
that can result from persisting infrastructure, long-term contracts or legal ‘lock-ins’. In
such cases decisions need to be taken by multiple stakeholders or entities on a
political level and cannot be attached to the outcome of a participatory process only.
For example, the Living Lab in Graz aimed to improve the quality of life in the traffic-
dominated area of Griesplatz. However, due to its purpose as traffic hub, not all

infrastructural elements could be replaced according to citizens’ desires: long-term
contracts with bus operators forced to wait or find alternative locations for bus stops
which occupy a big part of the square.

If circumstances don’t allow big changes, a LL should focus on what is actually

possible. Communication strategy and methodology have to be designed
accordingly, in order to avoid wrong expectations among LL participants. A focus on
behavioural measures can be fruitful in order to trigger structural change over time.
Also collecting ideas and concepts to apply in future, under different circumstances,
can be a strategy.

In the case of Griesplatz, people started to complain that elaborated discussions

ended up in little outcome. Thus, the organizers remained flexible and changed their
strategy by focusing on small steps: in order to deliver visible outcomes of the
participatory process, they provided small and quick improvements for the Griesplatz
area, such as a bike lane, enlargement of a green area, benches, or painting
temporary zebra crossings as awareness-raising measures. They released press articles
ensuring that ‘no idea was lost’ and would be put into place at a later stage within a
public architectural competition.

Neglecting effects outside project locality

Replicating pilot projects in the broader urban area can be prevented either because
generated knowledge is very much related to the specific context of the Living Lab or
because the whole Lab process only focused on the pilot project, neglecting or
forgetting the effects on its boundaries. To avoid this, it is important to always
consider projects’ indirect and cross-scale effects, also outside the boundary of
analysis, by actively engaging stakeholders of the broader urban context.

This aspect was not considered in Maastricht, when in 1971 the first parking garage
was situated below the main square that until then had served as an open-air public
parking. The preparation plan for the garage was developed in consultation with
business representatives from the city centre, and intended to secure the economic
attractiveness of the city as well as the spatial attractiveness of an open square. At
first, parties involved were pleased with the results: parking capacity had increased
and the square was cleared of cars. Along time, however, when parking tariffs
strongly increased, urging people to park shortly to allow high circulation, negative
effects started to become manifest. Not at the square, this remained clean, but the
growing flows of in- and outbound traffic increased congestion, noise and air pollution
at the inner-ring with its neighbouring residents.

To avoid replication of similar errors, when the City activated the redevelopment
process for the station area and the related Living Lab visioning workshop, the project
of the station area was explicitly put in the broader urban context of Maastricht, thus
preventing that an improved train station area would go at the expense of other urban


In this paper we have presented lessons learnt from action research in four ‘Smart
City Living Labs’ in the field of mobility, held in Bellinzona (CH), Brussels (BE), Graz
(AT), and Maastricht (NL). We were in particular interested in understanding which
factors typically hinder effective social inclusion and upscaling possibilities, thus
conditioning LL’s overall impacts. Once we identified such ‘constraints’, we
developed ‘smarter’ methodologies aimed at anticipating them, in both design and
management of Living Lab activities, and practically tested them. At the time of
writing, final assessments are still being performed. However, insights obtained so
far already allow us to identify simple coping strategies that, if implemented, could
produce tangible improvements in the effort towards wider social inclusion and


Part of this research project is financially supported by the Swiss Innovation Agency
Innosuisse, within the Swiss Competence Center for Energy Research SCCER


Dutilleul, B., F. A. J. Birrer, W. Mensink (2010). Unpacking European Living Labs:

Analysing Innovation's Social Dimensions. Central European Journal of
Public Policy, 4(1): 60-85.
Evans, J., & Karvonen, A. (2014). ‘Give Me a Laboratory and I Will Lower Your
Carbon Footprint!’—Urban Laboratories and the Governance of Low‐Carbon
Futures. International Journal of Urban and Regional Research, 38(2), 413-
Følstad, A. (2008). Living labs for innovation and development of information and
communication technology: a literature review. eJOV: The Electronic Journal
for Virtual Organizations and Networks. Volume 10, “Special Issue on Living
Labs”, 99-131.
Hillgren, P. A. (2013). Participatory design for social and public innovation: Living
Labs as spaces for agonistic experiments and friendly hacking. Public and
collaborative: Exploring the intersection of design, social innovation and
public policy, 75-88.
Hommels, A. (2005). Studying obduracy in the city: toward a productive fusion
between technology studies and urban Studies. Science, Technology and
Human Values, 30, 323-351.
Hommels, A. (2010). Changing obdurate urban objects: the attempts to reconstruct
the highway through Maastricht. In I. Farias & T. Bender (Eds.), Urban
Assemblages: How Actor-Network Theory Changes Urban Studies.
Karvonen A, Evans J & Van Heur B (2013). The Politics of Urban Experiments. In:
Hodson M, Marvin S (eds) After Sustainable cities? Pp 104-115. Routledge,
Karvonen, A., & Heur, B. (2014). Urban laboratories: Experiments in reworking
cities. International Journal of Urban and Regional Research, 38(2), 379-392.
Kemp, R., & Scholl, C. (2016). City labs as vehicles for innovation in urban planning
processes. Urban Planning, 1(4), 89-102.
Ker Rault, P.A. (2008). Public Participation in Integrated Water Management – a
Wicked Concept for a Complex Societal Problem. Dissertation, Cranfield
University, UK.
Leminen, S. (2013). Coordination and participation in living lab
networks. Technology Innovation Management Review, 3(11).

Innovation Management in Living Lab projects: the
Innovatrix Framework
Dimitri Schuurman1, Aron-Levi Herregodts*1,
Annabel Georges1, Olivier Rits1

*Corresponding author
1 imec.livinglabs

Category: Research Paper

Despite being described as ‘orchestrators’ and innovation intermediaries, the Living
Labs literature on concrete guidelines and tools for innovation project-related
innovation management is scant. Within this paper, we propose the Innovatrix, an
innovation management framework built upon existing business model and
innovation management tools and frameworks and iterated based on practical
experience in Living Lab projects. We illustrate its added value within three practical
case studies that lead to three propositions regarding innovation management in
Living Lab projects. First, Innovatrix helps to scope the user involvement activities,
which leads to a more efficient use of resources. Second, Innovatrix forces the project
owner to focus on a limited number of customer segments, which increases the
efficient spending of the scarce entrepreneurial resources. Third, Innovatrix allows to
capture the iterations and pivots that were made during an innovation project, which
helps to link outcomes with certain Living Lab activities.

Keywords: Living Labs; Innovation Management; Business Modeling; User

research; Assumption; Validation; Testing.


Although we seem to be living in an era where founding a start-up company has never
been easier, studies point out to high mortality rates of these organizations. A Harvard
Business School study put forwards a failure rate of 75% (Gage, 2012), whereas a
CB Insight study even proposes 90% for digital innovations (CB Insights, 2018). This
pleads for more research and insights into innovation management and innovation
intermediaries that can help these failure rates go down. One specific type of
innovation intermediary are the so-called Living Labs.

Living labs are complex partnerships, as they facilitate university-industry

relationships but also relationships between large companies, SMEs and startups.
Living Labs are often referred to as public- private-people partnerships (4P’s)
(Westerlund and Leminen, 2011). Based on a meta-review of the Living Labs
literature, Schuurman (2015) defines Living Labs as an organized approach (as
opposed to an ad hoc approach) to innovation consisting of real-life experimentation
and active user involvement by means of different methods involving multiple
stakeholders, as is implied in the Public-Private-People character of Living Labs.
Moreover, he also concludes that Living Labs are emanations of both Open
Innovation and User Innovation practices, as external inputs, including end-user
contributions, are used to iteratively design and cocreate the innovation in
development. This opening of the innovation process and the involvement of external
actors in a structural process have the potential to increase the value and
sustainability of the business model of the innovation (Baccarne et al., 2013).
However, there is only a limited amount of literature available that combine Living
Labs with business models. Rits et al. (2015) note that the majority of the papers in
this field deal with the business model of Living Labs themselves, but that explicit
integration of business model research in Living Labs is very rare. Moreover, in his
literature review of the most influential Living Lab papers, Schuurman (2015)
discovered that the majority dealt with the Living Lab organizations. This feels contra-
intuitive as Living Labs are regarded as innovation instruments and innovation
intermediaries that are capable of closing the gap between research and market
introduction (Almirall & Wareham, 2011). Therefore, we would expect much more
attention for the Living Lab project activities, and practical guidelines how to approach
innovation projects in a Living Lab setting. In this paper, we focus on this project level
and look for innovation management guidelines in Living Labs. After a review of Living
Labs literature, we propose the Innovatrix framework, based on existing innovation
management and business model tools and frameworks. We then investigate the
practical implementation of Innovatrix by means of four case studies, selected from
a sample of 40 Living Lab projects that used the Innovatrix framework.

Innovation management in Living Labs

Living Labs are regarded as complex phenomena where three analytical levels can
be distinguished: the organizational level, the project level and the individual user
interactions level (Schuurman, 2015). The defining elements of Living Labs, real-life
context, multi-stakeholder, multi-method, active user co- creation and medium- to
long term (Schuurman et al., 2013), are situated among these three separate, but
interlinked layers. The multi-stakeholder characteristic especially applies to the
organizational level. In this domain, Leminen (2015) provides a very diverse overview
of actor roles and management implications for Living Lab networks. Managing value
capture and value creation processes within Living Lab organizations is crucial for
their sustainability, but also cumbersome (Schaffers et al., 2007). This also resonates
with the medium- to long-term element. On the user interactions level, end- user co-
creation is regarded as the way to involve users. The literature describes various
ways and strategies to facilitate the process of co-creation (e.g. Kristensson et al.,
2008) and provides an overview of different user characteristics and user roles
(Leminen et al., 2014; Schuurman & De Marez, 2012) of Living Lab participants.

In terms of the project level, the real-life aspect and the multi-method approach are
characteristics. There is some literature on the real-life aspect and on context (e.g.
Bergvall-Kåreborn et al., 2009). The other element is the multi-method nature of Living
Lab projects. These Living Lab projects are described as a structured approach to
open and user innovation (Almirall & Wareham, 2008; Leminen et al., 2012;
Schuurman et al., 2016). This way, we look at Living Lab projects from an innovation
management perspective. However, Living Lab papers on methodology tend to
describe a very specific methodology, which is specific for a certain Living Lab, or an
innovation process with rather fixed elements and building blocks (e.g. Bergvall-
Kåreborn et al. , 2010). The most concrete are the works of Pierson and Lievens (2005)
and Schuurman et al. (2016) who put forward a quasi- experimental design, with a pre-
test, an intervention and a post-test. Next to this, there is little to no literature that looks
at innovation management in Living Lab projects, with the exception of some studies
on ‘Living-Labs-as-a-service’ (see the next section). This is surprising, as already in
2006, at the start of the Living Labs movement, Niitamo et al. state that “[i]n Living Labs
there is a need to combine highly self-organized and self-managed processes with
multi-disciplinary R&D and innovation management processes”. Ståhlbröst (2013) also
defines a Living Lab as “an orchestrator of open innovation processes focusing on co-
creation of innovations in real-world contexts by involving multiple stakeholders with
the objective to generate sustainable value for all stakeholders focusing in particular
on the end users”. Nonetheless, this has not lead to an abundance of papers and
studies that unravel or describe this process of orchestration. Coupled to this lack of
innovation management guidelines for Living Lab projects, there are only few studies

that present concrete results of the outcomes of Living Lab projects, and even fewer
that compare Living Lab projects with other innovation projects. Ballon et al. (2018)
relate this to the complexity of the Living Labs phenomenon, but also urge for more
studies in this domain. Therefore, within this paper, we will propose a tool that can be
used as innovation management approach and that also enables to elicitate impact
and outcomes of Living Lab interventions.

Business model components as innovation management


A notable exception of the search for innovation management anchor points for
Living Labs can be found in the scant literature on Living-Labs-As-A-Service. These
Living Labs, focused on delivering specific services to external customers, play the
role of innovation intermediary between entrepreneurs and end-users (Ståhlbröst,
2013). Coorevits and Schuurman (2014) argue that the Validation board
(, from the Lean Start-up methodology, can be used as a
tool to structure Living Lab projects as it is focused on planning and executing user
research. Rits et al. (2015) argue for the integration of business model research with
user research in Living Labs. In this context, they refer to established tools linked to
business modeling and technology entrepreneurship, such as the Business Model
Canvas (Osterwalder & Pigneur, 2013), the Lean Canvas (Maurya, 2012), and the
Value Proposition Canvas (Osterwalder, Pigneur, Bernarda & Smith, 2015).
D’Hauwers et al. (2015) proposed the iLLAB, a hypothesis driven Living Lab
framework incorporating both user and business model learning, based on elements
from the above business model tools. They see the iLLAB tool as an aggregation of
principles from Ries (2011), the Osterwalder Value Proposition Design (2015), the
business model matrix of Ballon (2007), the business model canvas of Osterwalder
(2010) and Porter’s five forces model (1985) and translated into a set of strategic
components. They developed their own framework to gather assumptions for user
research, as the input from the other frameworks remained too high-level to define
and execute user research. The validation board (Ries, 2011) functioned as main
framework as it puts the customers at the core and focuses on customer hypothesis,
problem hypothesis and solution hypothesis. This is also in line with the work of
Wildevuur et al. (2013) who designed the People Value Canvas (PVC) tool to help
build value propositions during user-centric service development processes. The
PVC consists of nine building blocks, describing the input that has to be provided to
establish the value proposition. The PVC is an iteration on the business model
canvas. This facilitates a process-oriented approach, more specifically for highly
iterative (and lean) innovation processes allowing for structured learning and

However, based on practical experience with the validation board and the iLLAB-
tool, we felt that more structure could be added to the existing elements. This is
acknowledged by Herregodts et al. (2017) who develop a framework on knowledge
uncertainties. Within these uncertainties, a major distinction can be made between
knowledge related to the current environment versus knowledge related to the
innovation under development. While the first is closely related to problem and
opportunity identification, the second is related to the formulation and evaluation of
solutions. This framework is based on the metaphoric use of ‘states’. States relate
to reference points, either from the perspective of the organization or the individual
(Gourville, 2005). Where the existing, ‘current state of being’, the ‘as-is’ or ‘status
quo’ is opposing ‘possible future states’ (Alasoini, 2011). These insights led to the
Innovatrix- framework (figure 1), which used elements from the previously discussed
frameworks and tools, but that also considers this dichotomy between current and
future state-knowledge. This resonates with a more process-oriented approach and
should facilitate innovation management in an optimal way. We now briefly introduce
and discuss the elements that compose the Innovatrix: customer segment, current
practices, needs, value proposition, solution, barriers, value capture & key partners.
The eight components will be discussed in more detail.

Figure 1. Innovatrix assumption & validation matrix

Customer segment – Current state

As used in the Validation Board (Ries, 2011) and the Business Model Canvas,
Innovatrix starts from customer segments. However, there is room for multiple
customer segments and the other elements are all linked to customer segments, and
do not necessarily apply for all segments. This enables more fine-grained assumption
development. In the Innovatrix framework there’s room for three customer segments
(the grey areas in the framework) to cater to the need of clear focus through limited
scope. The first column is used as an overarching column to map similarities between
defined segments. Following Innovatrix checks can be used to gauge for relevant
input to the Customer Segment criteria: What customer segments to focus on? What
are key characteristics? What is the use-context?

Needs – Current state

Osterwalder (2015) includes customer jobs, pains, and gains in the Value Proposition
Design canvas, which is the basis for the needs identification in the Innovatrix
framework. Furthermore, Ries (2011) links customer segments - customer problems
and the fit with the potential solution and/or value proposition. Following Innovatrix
checks can be used to gauge for relevant input to the Needs criteria: What are the
needs of the customer segment? How do we prioritie these needs?

Current practices – Current state

One missing pillar in Ries (2011), Osterwalder (2010) and in Ballon (2007), is the
competition and the differentiation of an SME/start-up/innovator. Competition refers
to the Five Market Forces of Porter (1985), which draws from five forces model. The
five forces make up the attractiveness of a market. The five forces can be defined as
(1) rivalry within the industry, the (2) threat of new entrants, (3) the threat of
substitutes, (4) bargaining power of suppliers and, (5) bargaining power of buyers.
Assessing the rivalry within the industry can help identify the difficulties of entering
the market. If for example, the market consists of multiple strong players (i.e.,
Oligopoly market), the need to diversify can lead to high barriers to entry. On the other
hand, if several new entrants enter the market (Monopolistic competition), it could
indicate that it’s an attractive market with lower barriers of entry. For some products
and/or services one can find possible substitutes, which can serve as an alternative
to the specific service and/or product. Following Innovatrix checks can be used to
gauge for relevant input to the Current Practices criteria: Who or what are
competitors, alternatives, customer behaviour? What are the pains and gains of these
current practices?

Value proposition – Current & future state

The value proposition is covered by the Lean Matrix of Ries (2011), the Value
Proposition and the Business Model Canvas of Osterwalder (2010,2015) and, by the
Business Model Matrix of Ballon (2007). The value proposition is the match between
the needs of customer segments, and how this can be solved with the solution
provided by the innovator. Following Innovatrix check can be used to gauge for
relevant input to the Value Proposition criteria: What (measurable) impact will you
create for this customer segment?

Solution – Future state

The solution refers to ‘the functional architecture’ of Ballon in the Business Model
Matrix (2007). The functional architecture comprises of the technical systems
composed of at least one building block (or module), governed by specific rules (or
intelligence) that interwork (or not) with other technical systems through
predetermined interfaces. The composition of the solution in the key modules and
technical systems enables the researcher and the innovator to identify the unique
selling point of the innovation compared to the competition. This division is less
explicitly included in Osterwalder (2010), even though the difference can be
significant in certain innovations. Following Innovatrix checks can be used to gauge
for relevant input to the Solution criteria: What are the components of your (digital)
solution? How do these components differ for the different customer segments?

Barriers – Future state

According to Steinkühler, Mahlendorf & Brettel (2014) self-justification is the most

empirically supported explanation for escalation of commitment, the “...tendency to
become locked-in to a course of action, throwing good money after bad or committing
new resources to a losing course of action” (Staw, 1981). Therefore, Steinkühler and
his colleagues (2014) argue that self-justification cannot be totally avoided but for de-
escalation of the commitments, the search for disconfirming evidence can help.
Therefore, it was decided to explicitly include ‘barriers’ as an element to look for this
disconfirming evidence. Following Innovatrix checks can be used to gauge for
relevant input to the Barriers criteria: What are the barriers for adoption, usage and
market entry?

Value Capture – Future state

Ballon (2007) included the financial model in the Business Model Matrix, which
described the revenue model and the revenue sharing model. Osterwalder (2010)
also considers the revenues model, where the pricing level and the pricing model are
mentioned. The researchers opted to utilize the definition of ‘Value capture’, which

comprises of the pricing model and the pricing level, and in cases where revenue
sharing is applicable, this section can be utilized. The application of the Innovatrix
matrix in different projects shows that partners can face difficulties identifying their
pricing model and pricing level, and thus this needs to be included in the Innovatrix
matrix. It is important to note that Value capturing has an important link with how
pressing the customer need is, and to the associated value the partner promises to
deliver. Following Innovatrix checks can be used to gauge for relevant input to the
Value Capture criteria: What value (monetary and non-monetary) do I receive in
return? What price should I set (and how)?

Key Partners – Future state

The value network definition is an alternative to the broad market-based approach of

the Business Model Matrix of Ballon (2007). In the value network analysis, though the
applicability is more adapted to innovations in the form of partnerships required to
deliver the innovation to the customers and with whom do innovators need to
collaborate. Following Innovatrix checks can be used to gauge for relevant input to
the Key Partners criteria: Who are your key partners? How to interact with

Innovatrix put to practice: from workshop to innovation

management process

In practice, the Innovatrix is used twofold: 1) as an innovation framework in a hands-

on workshop session and 2) as an innovation management process.

First, as an innovation framework, the Innovatrix is used at the start of an innovation

project, during a kick-off workshop. The most important roles in such a workshop are
the trained Innovatrix facilitator and the innovator (or the innovator’s team). The
workshop starts with an innovation pitch provided by the innovator. After this pitch,
the eight distinctive Innovatrix criteria – as described above – are ‘filled’ with relevant
input from the innovator. Here, the facilitator plays an important role in the gathering
of all relevant input through the means of very specific probing questions in the form
of ‘Innovatrix checks’. The gathered input is then awarded one of initially two
possible statuses, based on the nature and the strength of the input: either
assumption (the input has not yet been validated and is thus hypothesized by the
innovator) or validated assumption (the input has already been validated in previous
activities). Depending on the assumption status, the input is mapped on different-
coloured post-its: yellow (assumptions) or green (validated assumptions). The
outcome of an Innovatrix workshop is the mapping of assumptions and validated

assumptions, followed by marking the most important assumptions as ‘key
uncertainties’. The focus of following research activities should then put on these
key uncertainties.

Second, the Innovatrix is used as support of the innovation management process.

Here, the the Innovatrix framework is used as the starting point of a living lab
innovation project. The outcome of the Innovatrix workshop, key uncertainties, is
then translated into testable assumptions and matched with adequate research and
innovation activities. Research and innovation activities are then carried out. In a
next Innovatrix workshop, the Innovatrix update, the focus is placed on these key
uncertainties that were object of research and innovation activities. Here, the
assumption status is under debate based on the outputs of the research and
innovation activities. In an Innovatrix update, the status of an assumption can be
changed in dialogue with the entrepreneur with following possible statuses:
assumption (the research has not been validated), validated assumption (the input
was validated), new insight (new information arose from the research and innovation
activities) and invalidated (the assumption proof to be invalidated). The Innovatrix is
thus used in support of the innovation management process through the mapping
and changing of assumption statuses in a structured dialogue with the innovator
throughout the entire living lab innovation project. This process is repeated until the
end of the living lab project.


Within this paper, we adopt a mixed design with quantitative and qualitative data.
For the quantitative part, we look at all innovation projects carried out by the user
research team of imec.livinglabs from 2011 up till 2018, which makes for a sample
of 86 projects. For this sample we coded whether Innovatrix was used or not, based
on the project deliverables. In practice, the older projects did not use Innovatrix,
whereas the more recent projects all made use of Innovatrix. We also coded the
status of the project in terms of project outcome: ‘on the market’ if the innovation is
available for adoption by end-users, ‘abort’ if the innovation project is stopped and
the team members disband, ‘reboot’ if the innovation project is stopped, but the team
members continue with a new innovation project, based on the insights, and ‘in
development’ is used to indicate that the innovation is not launched yet. This can be
regarded as an ‘in between’ category, as over time these projects will either become
available on the market, be aborted or be rebooted. The data for the initial coding of
the projects is taken from a post assessment interview at the end of the project.
However, every year this database is updated based on an online search and a
personal follow-up with the project owners to assess changes. The last update of
the status dates from May 2018.

All of these projects were innovations with a digital component. The majority (58)
had a focus on B2C, whereas the remaining 28 projects could be labelled as B2B.
For an idea of some of the projects, see Schuurman (2015) and Schuurman et al.

For the quantitative analysis, we simply compared the numbers of the projects with
Innovatrix and the (older) ones without Innovatrix (see Table 1). Because of the
relatively small sample size, as compared to the outcome categories, no chi-square
tests could be performed as the expected cell numbers were less than 5 for more
than 20% of the cells. Therefore, we simply report the percentages. For the
qualitative study, we selected cases where the Innovatrix had explicit value. These
cases were chosen after a review of all projects by the co-authors, so the cases can
be considered as illustrative case studies (Yin, 2017).


The dataset consisted of in total 86 entrepreneurial innovation projects. Almost half

made use of Innovatrix, whereas 46 projects did not use the framework. When we
look at the outcomes over all, we already see a few striking results. Roughly 1 out of
4 of the projects is stopped after the project and almost 1 out of 10 is rebooted based
on the project insights, whereas 1 out of 10 is still ‘in development’ or implementing
the lessons learned. However, it is remarkable that almost 60% ends up in a market
launch. Referring back to the CB Insights study, where only 10% made in to the
market, the type of projects is rather different. In that study, entrepreneurs from an
incubator were studied, whereas in our study, the entrepreneurs should be
considered as more mature as they contacted an innovation intermediary (our Living
Lab) for support, defined a project proposal and paid for the project. However, this
big difference is still remarkable.

Table 1

Living Lab Living Lab
Status projects without projects with Overall
Innovatrix Innovatrix
Abort N: 19 – 41% 2 – 5% N: 21 – 24%

Reboot N: 5 – 11% 2 – 5% N: 7 – 8%

In development N: 0 – 9% 9 – 23% N: 9 – 10%

On the market N: 22 – 48% 27 – 67% N/ 49 – 57%

Total N: 46 – 53% N: 40 – 47% N: 86

As we look to the split between projects with and without Innovatrix, there are quite
some interesting differences as well. Only two projects were aborted, whereas in
the non-Innovatrix sample this is over 40%. Also, the percentage of reboots is also
half as high in the Innovatrix-category. All of the projects that are still ‘in
development’ belong to the Innovatrix category, which illustrates the point that this
is a ‘transit’ category because over time all projects end up in one of the other
categories. Finally, in this data set, two out of three innovation project that used
innovatrix during their Living Lab project resulted in a market launch. As a lot of
these projects were finished quite recently, these numbers will undoubtfully
change, but still this contrasts heavily with the 9/10 failure ratio.

We now turn over to three case studies where Innovatrix was used and provided
specific value to the innovation project.

Motosmarty: This project was about a mobile application that detected the driving
behaviour of young people in order to give feedback and assess their risk profile. In
term of end-user focus, there were no issues as the target population were young
people and students. Co-creation sessions, surveys and user tests were performed
to iterate the application. However, in terms of customer segment for the generated
data of the application, there was no focus at all. Here, the Innovatrix was used to
explicate all knowledge and assumptions regarding these customer segments (17
in total!). This led to discussions inside the team and made them realize that focus
was needed, otherwise they would burn all their resources without finding a paying
(B2B) customer. They are now on the market as Road Skippers and as Smart
Drivers and focus on insurance companies.

Spott: This Living Lab project was about a new way to recognize, like, share and
buy products with your smartphone based on what you see during a TV show or TV
commercial. The Innovatrix workshop at the start of the project indicated three types
of end-user segments and the television stations. In terms of value capture, the
assumption was that an affiliate marketing fee was the main source of income for
Spott. However, based on the co-creation sessions and field trials with the

application, it appeared that the ‘buying’ of items recognized during the TV-show
was not that common, but that more adept TV viewers felt more connected to the
TV content as they received more information on the objects that were used or worn
by their favorite characters. This newly discovered evidence made Spott delete one
of the end-user segments, that was focused on buying, and turned more towards
the ‘heavy’ TV-viewers, with as main income TV station paying to use the application
for their TV shows. The higher attraction of the content could attract advertisers,
which made the application interesting for them and increased their willingness-to-
pay. In this project, Innovatrix brought scope, identified unexpected outcomes and
enabled to focus on a limited number of segments. At the moment, Spott has already
been launched in multiple countries worldwide and is growing rapidly.

Lab Box: Lab Box is the organization behind Pikaway, a multi-modal transport
application that is able to plan and book trips, and is not restricted to one or a few
means of transport. At the starting workshop, it appeared that the three customer
segments were still rather high level and in need of specification. To tackle this, we
did a segmentation survey and subsequently conducted a field trial with
representatives of the user segments. This enabled to create persona, which
provided focus for the developers to prioritize their development backlog. Moreover,
one of the key assumptions captured during the first workshop, the need for a one-
stop shop application, could be iteratively validated by capturing the frustrations with
the current practices in the segmentation survey and co-creation session, but this
was also expressed in the field trial so this was also reflected in the future state
(solutions element). At the end of the project, the people from Lab Box asked for the
main take-aways and next steps for them based on the project. By going through
the Innovatrix and the modifications step by step, we could easily extract the main
learnings and key elements to work on before the market launch. At this moment,
Pikaway is available in the app store and a launch in the Play Store is planned for
the near future.

Discussion & Conclusion

Although Living Labs are regarded as orchestrators, which hints at an innovation

management approach, there is a lack of literature and studies that further explicate
this role of the process. Rather, Living Labs are described and studied in terms of
defining characteristics (such as real-life experimentation, active user co-creation
and public-private-people partnership), but it is left untouched how these elements
should be managed and utilized according to the needs and characteristics of a
specific innovation project. For Living Labs to take the next step in becoming mature
and established innovation organizations, we feel that this innovation management
role should be further elaborated and that this is even crucial in the complex entities
that Living Labs are. The three-layered model of Schuurman (2015) provides a

useful framework to anchor these elaborations. In our literature review, we noticed
that the largest gap in terms of the orchestration role in Living Labs is situated on
the project level. Therefore, within this paper we focused on the question how
innovation management in Living Lab projects can be facilitated and supported by
tools or frameworks.

As a result, we presented the Innovatrix framework, consisting of eight elements,

derived from existing business model tools and frameworks, with as specific
characteristic the fact that all elements of the framework should be specified for
each customer segment that is identified. Moreover, the Innovatrix also has a clear
distinction between the current state elements (the top three) and the future state
elements (the bottom four), which gives it a more dynamic, process-like feeling.

Based on an analysis of 86 Living Lab projects, where just over half did not use
Innovatrix and just under half did use Innovatrix, we discovered that two out of three
projects using Innovatrix resulted in a market launch. More data is needed to
validate these findings, but this gives a first indication of the effectiveness of the
framework. Moreover, based on three illustrative case studies of projects from the
sample that used innovatrix, we can derive three propositions regarding the use
and implications of the Innovatrix framework.

First, Innovatrix helps to scope the user involvement activities, as it clearly

explicates assumptions, related to the different customer segments, and it also
enables indicating which assumptions are key for taking the next steps in the project.
This leads to a more efficient use of resources and facilitates the selection of
representative users for the given customer segments and also guides the choice of
method to validate the assumption.

Second, Innovatrix forces the project owner to focus on a limited number of

customer segments, as there is only room for three to four segments maximum. If
there are more segments, the elements of the Innovatrix help to choose between
different segments in terms of focus. This increases the efficient spending of the
scarce entrepreneurial resources and helps decision making for the innovation
teams. Third, Innovatrix allows to capture the iterations and pivots that were made
during an innovation project. This is esecially useful to link outcomes with certain
Living Lab activities, as this has appeared rather problematic to this date (see Ballon
et al., 2018). Innovatrix serves as a visual summary of key elements and
assumptions regarding an innovation project from the viewpoint of the end-user.
By capturing snapshots of its state before and after a research activity, the
modifications and alterations become apparent. To this end, at the moment a digital
version of Innovatrix is being built that enables to fill out Innovatrix digitally and keep
track of all changes during a project. Moreover, Innovatrix is also an interesting tool
to facilitate the discussion between Living Lab researchers and the project owners.

To further explore and validate these propositions, more research is needed and
more cases are needed to assess the value of Innovatrix. Also, Innovatrix represent
a specific view on innovation management and Living Lab activities. It has been
used in a ‘Living-Labs-as-a-service’-context, but might be applicable in other
contexts as well. We encourage uses and tests in other contexts, other types of
projects and other organizations in order to increase the knowledge on innovation
management in a Living Lab-context and to help building a more structural,
encompassing Innovatrix for all kinds of Living Lab projects and activities to increase
the impact and position of Living Labs as innovation intermediaries.


Almirall, E., & Wareham, J. (2008). Living labs and open innovation: Roles and
applicability. eJOV: The Electronic Journal for Virtual Organization &
Networks, 10.
Almirall, E., & Wareham, J. (2011). Living Labs: arbiters of mid-and ground-level
innovation. Technology Analysis & Strategic Management, 23(1), 87-102.
Baccarne, B., Logghe, S., Veeckman, C., & Schuurman, D. (2013). Why collaborate
in long-term innovation research? An exploration of user motivations in Living
Labs. In 4th ENoLL Living Lab Summer School 2013. European Network of
Living Labs.
Ballon, P., Van Hoed, M., & Schuurman, D. (2018). The effectiveness of involving
users in digital innovation: Measuring the impact of living labs. Telematics
and Informatics.
Bergvall-Kåreborn, B., Eriksson, C. I., Ståhlbröst, A., & Svensson, J. (2009). A
milieu for innovation: defining living labs. In ISPIM Innovation Symposium:
Bergvall-Kåreborn, B., Howcroft, D., Ståhlbröst, A., & Wikman, A. M. (2010, March).
Participation in living lab: Designing systems with users. In IFIP Working
Conference on Human Benefit through the Diffusion of Information Systems
Design Science Research (pp. 317-326). Springer, Berlin, Heidelberg.
Blank, S. (2013). Why the lean start-up changes everything. Harvard Business
Review, 91(5), 63-72. Blank, S. G. (2006). The Four Steps to the Epiphany.
Foster City, Calif.:
CB Insights (2018). 253 Startup Failure Post-Mortems. Retrieved at
Coorevits, L., & Schuurman, D. (2014). Hypothesis Driven Innovation: Lean, Live
and Validate. In ISPIM Conference Proceedings (p. 1). The International
Society for Professional Innovation Management (ISPIM).
D'Hauwers, R., Rits, O., & Schuurman, D. (2015). A hypothesis driven tool to
structurally embed user and business model research within Living Lab
innovation tracks. In Open Living Lab Days 2015.
Gage, D. (2012). The Venture Capital Secret: 3 Out of 4 Start-Ups Fail. Retrieved at
Gourville 2005 - Gourville, J. T. (2005). The curse of innovation: a theory of why
innovative new products fail in the marketplace. HBS Marketing Research
Paper, (05-06).
Herregodts, A. L., Baccarne, B., Conradie, P., & Schuurman, D. (2017, June).
Managing Innovation Uncertainties: a User-Oriented Knowledge Typology. In
ISPIM Innovation Symposium. The International Society for Professional

Innovation Management (ISPIM).
Kristensson, P., Matthing, J., & Johansson, N. (2008). Key strategies for the
successful involvement of customers in the co-creation of new technology-
based services. International journal of service industry management, 19(4),
Leminen, S., Westerlund, M., & Nyström, A. G. (2014). On becoming creative
consumers–user roles in living labs networks. International Journal of
Technology Marketing, 9(1), 33-52.
Leminen, S. (2015). Living Labs as Open Innovation Networks-Networks, Roles and
Innovation Outcomes.
Lovallo, D. & Kahneman, D. (2003). Delusions of success. Harvard business
review, 81(7), 56-63.
Maurya, A. (2012). Running Lean: Iterate from Plan A to a Plan That Works.
Sebastopol, CA: O'Reilly
McPhee, C., Schuurman, D., Ballon, P., Leminen, S., & Westerlund, M. (2017).
Editorial: Innovation in Living Labs (January 2017). Technology Innovation
Management Review, 7(1): 3-6.
Osterwalder, A. & Pigneur, Y. (2013). Business model generation: a handbook for
visionaries, game changers, and challengers. John Wiley & Sons.
Osterwalder, A., Pigneur, Y., Bernarda, G., & Smith, A. (2015). Value Proposition
Design: How to Create Products and Services Customers Want. John Wiley
& Sons.
Ries, E. (2011). The Lean Startup: How Constant Innovation Creates Radically
Successful Businesses. New York: Crown Business.
Rits, O., Schuurman, D., & Ballon, P. (2015). Exploring the Benefits of Integrating
Business Model Research within Living Lab Projects. Technology Innovation
Management Review, 5(12).
Schaffers, H., Cordoba, M. G., Hongisto, P., Kallai, T., Merz, C., & Van Rensburg,
J. (2007, June). Exploring business models for open innovation in rural living
labs. In Technology Management Conference (ICE), 2007 IEEE International
(pp. 1-8). IEEE.
Schuurman, D., & De Marez, L. (2012). Structuring user involvement in panel-based
Living Labs. Technology Innovation Management Review, 2(9).
Schuurman, D., De Marez, L., & Ballon, P. (2013). Open Innovation Processes in
Living Lab Innovation Systems: Insights from the LeYLab. Technology
Innovation Management Review, 3(11): 28-36.
Schuurman, D. (2015). Bridging the gap between Open and User Innovation?:
exploring the value of Living Labs as a means to structure user contribution
and manage distributed innovation (Doctoral dissertation, Ghent University).
Ståhlbröst, A. (2013). A living lab as a service: creating value for micro-enterprises
through collaboration and innovation. Technology Innovation Management
Review, 3(11).

Staw, B. M. (1981). The escalation of commitment to a course of action. Academy
of management Review, 6(4), 577-587.

Steinkühler, D., Mahlendorf, M. D., & Brettel, M. (2014). How Self-justification

Indirectly Drives Escalation Of Commitment - A Motivational Perspective.
Schmalenbach Business Review: ZFBF,66(2), 191.
Wildevuur, S., van Dijk, D., Hammer-Jacobsen, T., Bjerre, M., Äyväri, A. and Lund,
J. (2013), Connect. Design for an empathic society, BIS Publishers,
Amsterdam, NL.
Yin, R. K. (2017). Case study research and applications: Design and methods.
Sage publications.

International Innovation Sprint Bridging the
Sustainability Gap between Metropolitan Core and
Tuija Hirvikoski1, Kaisla Saastamoinen2
and Mikael Uitto3
1,2 Laurea University of Applied Sciences

Category: Innovation paper

Generally, RDI work takes place at the core of Metropolitan areas where the best
innovation resources, Universities, and Research Institutions normally situate. At the
same time, the periphery of Metropolitan that provides the core area with many vital
resources can itself suffer from migration loss, brain drain, loss of jobs, and the many
challenges related to ageing population and long distances or poor logistics. In this
paper we aim to contribute to the discussions on the role of science, technology and
innovation in society and particularly in making cities and communities inclusive,
resilient and sustainable (UN SDG 11). We adopt perspectives and concepts from
innovation literature and policy documents to introduce Innovation Sprint as an
innovation intermediary tool. We explain how the sprints were designed and
experimented first in Taiwan and then in Finland. In both cases, multidisciplinary and
international Sprints were taken to a remote community to observe, understand, and
then to co-create innovative solutions with and for the local stakeholders. We also
discuss the ways the Sprint might bridge the sustainable development gaps between
the urban, peri-urban and rural areas where the intensity of knowledge, technology,
and monetary resources can vary substantially.

Keywords: Innovation Sprint; Living Labs; sustainability gap; science, technology and
innovation in society; ICT solutions making cities and communities inclusive, resilient
and sustainable

Theoretical framework: The role of science, technology and
innovation in society

The role of science and technology in society has been discussed since 1971, when
the OECD report Science, Growth and Society (OECD 1971) was published. Since
then, science and innovation policies have tried to reconcile the curiosity driven
science and its inner need of autonomy with the need driven science (Wessner 2005)
and society’s wish (EC 2014, 2016 & 2018) to enjoy the fruits of science. Decades
ago, the OECD innovation policy stressed that the linear science-technology
innovation (STI) was to be replaced by interactive or systemic innovation (Miettinen
et al. 2006). The systemic nature of innovation refers (e.g. Fagerberg 2006) to the
collective achievement of innovation through interlinking actors, activities, and
innovation system.

As a consequence of the OECD innovation policy, the connections between science,

technology and economy, quality of life, and societal challenges have become
important research topics. In 2018, the role of science and innovation in society is
emphasised with such policies as Responsible Research and Innovation (EC 2014),
Open Innovation, Open Science Open to the World (EC 2016), Sustainable
Development Goals (UN 2015) and Mission oriented research and Innovation (EC

Moreover, examples as the Innovation Manifesto (ENoLL 2018) show how the locus
of emphasis is simultaneously changing from business innovation to social digital

As urban areas expand and interweave, the role of science and innovation has been
discussed in new urban configurations, and urbanization has been considered an
open process, determined by constant innovation and inventiveness (e.g. Diener et
al 2005). In the urban context, interactive or systemic innovation has emerged parallel
to the linear science-technology innovation (STI). Concepts such as Cities as Urban
Labs (Evans et al 2016) have been used to illuminate how the ST–innovation can be
tested and validated in the urban context. At the same time, interactive innovation
has been connected to such notions as Doing, Using and Interacting (DUI) (Lundvall
1985), Mode two of knowledge production (Gibbons & al. 2005), Quadruple Helix
(Curley & Salmelin 2018) based on Etzkowitz’s (2002) Triple Helix of Academia,
Industry and State, or Open Innovation 2.0 (Curley & Salmelin 2018), Living Labs
(The Helsinki Manifesto 2006), and City as a Living Lab (ENoLL 2016), or the city as
an open innovation platform (6Aika).

Finland can be considered an example of sustainable optimization through
technology and Living Labs. The Smart and Clean Helsinki Metropolitan (2017)
claims the capital is the best testbed or living lab in the world for smart and clean
solutions, “providing a unique environment for getting things done”. They aim at
changes that will lead to new, permanent processes that will improve quality of life,
are carbon positive and value adding, and will boost circular economy with the world’s
most resource-wise citizens.

Although such regional policies as the Regional Cities Programme (2018) rely on
effective cooperation between the regional cities and the state, less attention has,
however, been paid on how science, technology and innovation can help to bridge
the sustainability gap between Metropolitan agglomerations and their immediate
hinterlands, where the leading science and innovation organisations seldom exist or
have any activities.

Methods and Case Studies

In this chapter, we will introduce the method Innovation Sprint as a living lab and two
cases bringing international experts with scientific knowledge to the Metropolitan
periphery to collectively solve hinterland challenges with the local stakeholders. The
Sprint also epitomizes how education, science, and innovation in society can be
successfully re-contextualised.

Living Labs with and for sustainable ICT solutions making cities and communities
inclusive, resilient and sustainable

With Living Labs and sustainable ICT solutions making cities and communities
inclusive, resilient and sustainable we refer to the UN Agenda 2030 and the
Sustainable Development Goals (SDGs) ‐ a global sustainability agenda with 17
ambitious goals, agreed in 2015 by 193 countries within the United Nations

Westerlund and Leminen (2011) define Living Labs as “physical regions or virtual
realities, or interaction spaces, in which stakeholders form public–private–people
partnerships (4Ps) of companies, public agencies, universities, users, and other
stakeholders, all collaborating for creation, prototyping, validating, and testing of new
technologies, services, products, and systems in real-life contexts”.

Making ICT deliver sustainability - Co-design for sustainable lifestyles (G-Stick

conference 2017) emphasized the importance of testing, experiencing, and co-
creating innovation through Living Labs as intermediary orchestrators in

multidisciplinary and cross-polluting sectors, by making citizens the driving factor of
innovation creation. The conference defined Living Labs as user-centered, open
innovation ecosystems based on a systematic user co-creation approach. They
integrate research and innovation processes in real life communities and setting,
placing the citizen at the center of innovation. Living Labs have in this way
demonstrated their ability to better mold the opportunities offered by new ICT
concepts and solutions to the specific needs and aspirations of local contexts,
cultures, and creativity potentials.

In this paper, we consider Innovation Sprint as a full-scale urban, peri-urban or rural

living labs, proving ground for learning, inventing, prototyping, analyzing, assessing,
and creating market opportunities for new digital social innovation or ICT technology

Due to the crosscutting nature of ICT and Living Labs, they jointly have the capacity
to create and support systemic transitions leading to an inclusive, safe, resilient and
sustainable ‘smart’ city development. This capacity is based on the fact that Living
Labs are based on trust capital whereas the citizens are perceived as innovation
actors or prosumers, not as research factors. Prosumers refer to consumers who
become involved with designing or customizing e.g. digital products for their own
needs as consumers.

Living Labs capturing the needs of a broader range of users ensures a higher rate of
inclusion of technology. They collaborate with the lead users to face the market needs
months or years before the bulk of that marketplace encounters them. They also co-
create with the vulnerable users or users at-risk parts of the society with a poorer
access to services and technologies and therefore are at the risk of exclusion. Often
Living Labs aim to provide equal learning, innovating, and capacity building
opportunities for all citizens; therefore, they create trust-capital into the society and
local communities.

The original idea of Living Labs was with the help of co-creation and rapid
experimentation to shorten time between digital development and market
deployment. Living Labs can be utilized as a tool for changing behaviour towards
sustainable actions of consumers.

Innovation Sprint as an innovation intermediary tool for Living Labs

Innovation Sprint is based on the idea of Living Labs as an open innovation

ecosystem or a platform allowing sharing and seamless interaction between all
stakeholders such as cities, citizens, companies, and academia (Curley & Salmelin

2017). The Sprint offers a physical place and virtual space bringing together the
different stakeholders. During the Sprint, co-creation and real-time experimentation
is conducted in real-world situations, allowing simultaneous technical and societal

As we explain in this paper, international university students are a crucial element of

the Living Lab based on Sprint. During the Sprint, the students link the scientific
knowledge from the Universities to the market-based knowledge from the hinterland
citizens and local authorities. However, the Sprints sometimes have received
negative feedback from students, when run by big corporations. A Sprint can be seen
as a way of drafting a lot of unpaid talent to solve a problem for the financial gain of
the corporation while leaving the Sprinters with nothing but warm thanks for their

To avoid this problem, our Sprints focused on creating public good. Sprint, when
internalizing the challenges related to making cities and communities inclusive,
resilient, and sustainable and externalizing public good, makes it easier for everyone
to find inspiration without the risk of feeling that one’s intellectual property rights have
been exploited.

Next, we will briefly introduce the two Innovation Sprints, both of which were based
on service design thinking (Ramaswamy and Goiullart 2010; Ojasalo, Koskelo,
Nousiainen, 2015). Moreover, they applied Wenger’s (1998) Social theory of learning
and Living Lab approach to integrate international universities’ scientific and
innovation know-how to the real-live challenges and resources situating in the
hinterlands of Metropolitan areas.

Innovation Sprints as a transdisciplinary and agile development process

An Innovation Sprint is a transdisciplinary and agile development process where

design thinking and co-creation play a key role. In a Sprint, a team or teams are
working on tackling a real-life challenge through rapid prototyping and testing, making
the problem-solving process considerably faster and more human-centric than in
traditional project work. Sprint teams are multidisciplinary to optimise benefitting from
each team members' diverse individual knowledge and skills as well as from the
multidisciplinary co-creation as a team. A Sprint can follow service design, design
thinking, or other agile development and innovation processes. It is important that the
participants will be provided with enough information on both the challenge at hand
and the way of working, as well as the reasons for selecting the process in question,
so that even in a short time of the Sprint they can trust the chosen methods and their
functionality in the process.

The Asian Smart Living Sprint

The first case comes from Taiwan. The first Asian Smart Living Summer School was
organised in 2011 in Taiwan and the Sprint title was “Innovation and Connection:
Crafting Smart Journey in Formosa.” The design of the Sprint was based on the
double diamond method with 4Ds: discover, define, develop, and deliver (British
Design Council).

First, the Sprint took the participants on a Journey to observe and understand the
challenges and opportunities related to the communities situating at the Metropolitan
hinterlands. Three communities were selected to facilitate creative connections
between cultures and technologies, the young and the elderly, and people and the
environment: 1) a Taiwanese community of aboriginals reviving traditional industries
with innovation, 2) an ageing village outside Taipei, offering an ageing life without
limitation on body, mind and spirit, and 3) a Zen Buddhist community integrating
cutting-edge technology and rich cultural content.

In Taiwan, the aim was to create a prototype of a new program modernising academic
education and design-based innovation and to experiment the program in practice.
Therefore, the Ministry of Education of Taiwan sponsored the program. It was jointly
organised by the Talent Cultivation Program for Smart Living Industry and National
Taiwan University, National Chiao Tung University, and Dharma Drum Buddhist
College. Laurea University of Applied Sciences and ENoLL joined the prototype
development and its experimentation.

The first, 5-day edition of the Sprint offered an organised innovation and learning
program for almost 100 people from Finland, Japan, Malaysia, Russia, Singapore,
Taiwan, USA, and Vietnam. Forming six teams, the participants defined and then
worked on the challenge they deemed most important and most viable to be solved
with the resources they had access to during the one-week encounter with the
mentoring university professors and staff, other students, and with the locals. As a
result, six technology-enhanced prototypes and their value propositions were
introduced to the jury and the representatives of the local communities, researchers,
and companies. (Sung & Jou 2011)

The European Digital Wellbeing Sprint

The second case is from Finland, where a small municipality situated at the outer
skirts of Helsinki Metropolitan hosted a Digital Wellbeing Sprint in 2017. In Finland,

the Sprint was initiated by Laurea and jointly organised by the strategic alliance of
three Universities of Applied Sciences (Laurea, Metropolia and Haaga-Helia).

In the 6-day Sprint, students, the municipality at the outer skirts of Helsinki
Metropolitan area, and the organising UASs worked in collaboration to tackle the
municipality’s challenge “Living well all the way”. The purpose was to gather deep
understanding of the every-day life and challenges of the inhabitants, especially those
with memory disorders, and then ideate and prototype proposed solutions based on
the gathered understanding. The teams followed the Service Innovation process
introduced by Ojasalo et al. (2015): Map & Understand; Forecast & Ideate; Model &
Evaluate; and Conceptualize & Influence.

More than 30 Bachelor and Master students from the 3 UAS alliance and partner
universities, as well as exchange students, formed eight Sprint teams. The
participants were from Finland, Romania, Germany, Vietnam, Morocco, Philippines,
Chile, China, Kenya, and Pakistan; with their fields of study ranging from Healthcare
to Service Design, and Business to IT. The Sprint teams had more senior Masters
students as well as staff members of the three UASs as Sprint mentors. Part of the
teams spent the majority of the Sprint in the challenge owner municipality, immersing
themselves in the inhabitants’ life and environment in practice.

At the end of the Sprint, each team pitched their proposed solution to the jury
consisting of members of the municipality administration, a member of staff in a local
care home, and a UAS teacher and facilitator of the Sprint. Each proposed solution
pitch was evaluated on the criteria of 1) the usability, potential of realisation in reality,
and originality of the team’s idea or concept; 2) how the team delivered the pitch; 3)
how the team had used the different methods and followed the Sprint process,
especially paying attention to consideration and usage of Futures Thinking and Value
Proposition Design; and 4) focus and attention on customer, the municipality’s
resident with memory disorder. The scalability of the teams’ propositions was also
considered. As the end result, the municipality gained 8 diverse prototyped service
propositions and concepts that the municipality could further refine.


An Innovation Sprint can work as a means for bridging the sustainability gap between
a Metropolitan area and its peripheries. The Sprint offers an invaluable opportunity
for the periphery to gain access to a skilled, motivated workforce with fresh ideas and
proposed solutions to their challenges. The process being facilitated by a University
ensures the quality of theoretical background and novel approaches. The participants

gain a learning experience in a multidisciplinary team, learning-by-doing in a real-life
setting. The collaboration between the stakeholders enables the periphery to benefit
from the Metropolitan higher intensity of knowledge and technology without
expenditure of their monetary resources.

For an Innovation Sprint to produce potentially usable proposed solutions to real-life

challenges, it is vital that all stakeholders share the mindset of open innovation. The
challenge provider is prepared for collaboration and sharing of information with both
the organisers as well as the participants of the Sprint; the organiser provides a space
and time to connect the stakeholders and offers theoretical and facilitative support on
the process, and the participants do their best in sharing and using their individual
strengths to work together effectively. In our example cases, besides helping
formulate the challenge to suit a short Sprint, the challenge owners provided Sprint
organisers and teams with the time of some of their key staff as well as with the
opportunity for participants to immerse themselves in the local life in the real setting
of the challenge. Part of the teams were working the majority of the Sprint in the
hinterland municipalities where they not only observed the general setting but also
attended visits organised by the municipalities to observe the daily life of e.g. local
service providers and to interview their inhabitants and staff. This provided
participants with invaluable insight into the challenge and led to prominent proposed
solutions for the challenge owner peripheries.

In total, the example case Sprints offered the small municipalities on the outskirts of
Helsinki, Taipei, and Hsinchu Metropolitan areas about 700 full days of work by
motivated, multidisciplinary, and multi-cultural participants. Besides the working
hours, expertise, and insights gained by the municipality during the Sprint, even a
short Sprint can also help build longer relationships between different stakeholders,
benefitting all parties involved. Some of the concepts or ideas that result from a Sprint
could immediately be developed further or spark ideas for prototyping and testing
others, and the Sprint participants – already immersed in the challenge – could be
perfect for the job. Ideas and concepts born during a Sprint can also grow into Start-
ups or improved products or services of an existing organisation. Typically, start-ups
are concentrated in Metropolitan areas, however their operation can benefit a wider
audience and in turn continue to take part in revitalising the periphery.

Innovation Sprints can help universities develop their operation and its effectiveness
by actively facilitating open collaboration and its benefits for the various stakeholders.
As for the roles of universities, Innovation Sprints have clear benefits also on both
pedagogic and regional development level: Sprints help generate opportunities and
obtaining of new competences for both the student participants as well as the region
as a whole. A Sprint also enhances the university’s competitiveness, contributing to
regional development objectives and showcasing the utilizing of methods such as

open innovation, co-creation and rapid prototyping – skills indispensable to modern
working-life and its development. The results imply that innovation-in-interaction
between and within higher education, science, and society can be fruitful to all parties
while supporting positive social and economic links between the Metropolitan and its
peripheries (UN SDG 11).
Benefits for the periphery

Through collaboration with Sprint participants, the municipality providing the

challenge has the opportunity to gain both out-of-the-box new solutions as well as
potential development ideas of current situation with a fresh outlook. At the end of
the Sprint, in Taiwan 6 and in Finland 8 prototypes or concepts were introduced to
the jury and for the stakeholders to utilize.

Besides the presented end results of the Sprint in the form of a pitch or similar, a
Sprint can work as a valuable base for longer, productive relationships between its
stakeholders. In the case of the Sprint 2017, together with the Finnish municipality
two student participants have continued working further on one of the concepts born
in the Sprint. Another student started their Master’s thesis on a topic related to the
Sprint theme: their thesis topic was redefined based on the Sprint experience in the
real-life setting, and the connection with the municipality staff established during the
Sprint enabled them to conduct their thesis research with the actual inhabitants. Also,
the inhabitants of the municipality who were part of the Sprint process e.g. in the role
of interviewees or observees reported gaining a feeling of inclusiveness in being seen
and heard in a process of tackling the challenges that affect their every-day lives.

Benefits for the Metropolitan

Solutions born as a result of innovation Sprints can generate commercial activities

and tax revenue that benefit the society as a whole – both in the Metropolitan area
and its peripheries. Start-ups are born both on ideas sparked during Sprints as well
as a result of the Sprint collaboration: two team members from the winning team of
Digital Wellbeing Sprint 2017 went on to attend an international entrepreneurship and
start-up accelerator bootcamp where teaming up with 2 others they founded a health-
related start-up company.

Method Development

Innovation Sprints as a new Living Lab method integrate University’s curricular and
RDI activities with its stakeholders’ needs and resources. Through Sprints,
Universities can offer their theoretical knowledge and facilitation skills to enable a
meaningful and practical learning experience for students, and at the same time offer

valuable resources and input for the other stakeholders and the region as a whole.
Innovation Sprints can also facilitate longer-term relationships between the
stakeholders, benefitting them and the local area long after the Sprint has finished.

Innovation Sprints create spaces for dialogue and orchestrate co-innovation among
academy, industry, SMEs, Civil Society Organisations (CSOs), and policy makers.
This contributes to sustainable regional development and allows the peripheries of
Metropolitans to benefit from the higher intensity of knowledge and technology
traditionally typical of the core area.

If integrated into Universities’ other Living Lab orchestration activities, Sprints can
effectively explore systematic and institutional adaptation models to future regional
priorities and synthesise new solutions.

Benefit for the participants

Innovation, co-creation, and experimentation with different national and international

stakeholders provide students and other participants with excellent learning and
employment opportunities with a potential of new Start-ups. Participants attending
Innovation Sprints gain skills and practise vital in the modern working-life: open
innovation, rapid prototyping and testing, and co-creation. They can practise these
skills in a real-life environment with a real challenge with skilled facilitators, and take
their learnings – as well as the connections they have gained – into their studies or
working life.

Reconciliation of curiosity driven and need driven science

The Innovation Sprint brings the curiosity and need driven science into mutual
interaction. An Innovation Sprint organised in the outer skirts of Metropolitan area
provides a wide variety of observations and data on inclusion, resilience, and
sustainability, allowing the researchers to decentre their focus of analysis to illuminate
the wider context of an urban territory. The Sprint, intertwining the interactive and
iterative need driven approach with the curiosity driven science is an additional tool
to bridge the sustainability gap between the Metropolitan centralities and peripheries.

Challenges and Limitations

To succeed in producing viable proposed solutions to real-life challenges, innovative

operative methods such as co-creation and innovation Sprints and Living Labs call
for innovative leadership and freedom within framework from all stakeholders.
Additionally, to succeed these methods call for professional facilitation. An academic

institution or another impartial organisation is in a prime position to create trust and
facilitate gathering of relevant stakeholders to be involved, as well as to provide
needed background theory and support on the chosen framework. Whereas some
scholars (for example Kivistö and Pihlström 2018) are sceptical about agile methods
to be used for research and innovation purposes, a viable Business Model can be
created thanks to the fact that the Sprints provide a learning opportunity to the
University students. However, the costs related to trips, accommodation, venues and
equipment is a challenge. Different Business Models has been applied for different
challenges; the most common ones are based on external project funding or the
challenge owners’ compensations.

Conclusions and Further Suggestions for the practitioners,

researchers, and policy makers

During the physical Innovation Sprint, the collaboration between the stakeholders
enables the periphery to benefit from the Metropolitan higher intensity of knowledge
and technology without expenditure of their monetary resources. Today, the multiple
disruptive technologies, all arriving at the same time, can be used in the Living Labs
and to improve the Innovation Sprint’s benefits to all stakeholders. With the following
three suggestions based on the EC (2016) and OECD (2017) policies, we anticipate
to balance the sustainable development between the Metropolitan periphery and core
Firstly, the Sprint stakeholders and participants should be equipped with basic
knowledge on the possibilities provided by different disciplines of computer science
(such as Artificial Intelligence (AI), Data Mining, Bloc Chain, or Cloud Computing)
relevant to the Innovation Sprint. Whilst, during the Innovation Sprint, the computer
scientists carry the responsibility to compose the actual algorithms for digital
innovation, creating innovation in multidisciplinary Sprint teams would become easier
if all the participants shared an intuition of the potentials related to digitalisation. That
is, all the participants should understand what is possible and not possible through
programming, and what are the challenges and major implications of the major digital
innovations such as AI or Block Chain. Therefore, we would recommend such
knowledge to become a prerequisite for all the Sprint participants. Such virtual
courses are available on internet free of charge1.
Secondly, as big data and data-driven innovation is creating significant business
opportunities, data should also be used to solve the sustainability challenges of the
municipalities situating in the outer skirts of the Metropolitan areas. Aligned with the
Findable, Accessible, Interoperable and Reusable (FAIR) Data policy


(Implementation Roadmap for the European Open Science Cloud), the Innovation
Sprints should create and agree on the Data Management Plans, to help the Sprint
participants in different countries and times to collect, share, and use data before,
during, and after the Innovation Sprints. Innovation Sprints and Living Labs sharing
data would leverage better solutions for local authorities responsible for urban, peri-
urban and rural areas to become inclusive, resilient and sustainable. Therefore,
Digital Innovation Sprints should be included into local, regional and EU strategies to
support science based economic, social, and environmental links between the
Metropolitan cores and peripheries and to promote territorial inclusion within EU.
Moreover, FAIR Data provides new Business opportunities for the Living Labs
contributing and sharing data on same phenomenon.
Thirdly, based on the previous two suggestions we recommend transnational co-
creation and experimentation Innovation Sprints to be organised simultaneously in
different countries and to connect them digitally. With the help of internet connections,
the Innovation Sprint participants in urban, peri-urban, and rural areas could create
and exchange more data to develop better solutions and to receive immediate
feedback from the rapid experiments in different legislative, cultural, language,
climate, and logistic conditions. Internationalization would have one further benefit.
The delocalized but simultaneous Innovation Sprits would decrease the travel costs
of the Innovation Sprints without losing the access to cultural diversity. The data
collected during the various Innovation Sprints could operate as an attraction factor
for the world’s leading researchers. Their involvement before, during, and after the
Sprints would create better conditions for the cities and rural areas at the hinterland
to benefit from the combination of the agile and fast face-to-face Innovation Sprints
and their permanent connections to fundamental research and innovation institutions
in any global metropolis. Moreover, continuous research is needed to assess the
impacts of the Digital Innovation Sprints on learning and innovation results.
With these recommendations, we believe the Digital Innovation Sprint will promote
Commissioner Carlos Moedas’ (2017) vision: “The year is 2030. Open Science has
become a reality and is offering a whole range of new, unlimited opportunities for
research and discovery worldwide. Scientists, citizens, publishers, research
institutions, public and private research funders, students and education
professionals as well as companies from around the globe are sharing an open,
virtual environment called The Lab”.


6Aika. Six Finnish cities join forces to become better and smarter.
British Design Council. The Design Process: What is the Double Diamond.
Curley, M. ja Salmelin, B. (2018). Open Innovation 2.0. The New Mode of Digital
Innovation for Prosperity and Sustainability. Springer, Cham.
Diener, R., Herzog, J., Meili. M., Meuron,P., & Schmid, C. (2005). Switzerland –
an Urban Portrait.
EC (2006). The Helsinki Manifesto 20.11.2006. We have to move fast, before it is too
late. Finland EU Presidency
EC (2014). Rome Declaration on Responsible Research and Innovation in Europe.
Italian Presidency of the Council of the European Union
EC (2016). Open innovation, open science, open to the world - a vision for Europe.
Produced by European Commission's Directorate-General for Research &
Innovation (RTD).
EC (2018). Mission-oriented research and innovation policy. A RISE perspective –
Study. Directorate-General for Research and Innovation.
EC Committee of the Regions. Innovation Camp.
ENoLL (2018). The Manifesto for Innovation in Europe.
Evans, J., Karvonen, A. & Raven, R. (Eds.) (2016). The Experimental City. Routledge
research in sustainable urbanism. New York: Routledge.
Etzkowitz, H. (2002). The Triple Helix of University - Industry – Government
Implications for Policy and Evaluation. Science Policy Institute Working paper
Fagerberg, J. (2006). Innovation: A Guide to the Literature. In Fagerberg, J.,
Mowery, D. C. & Nelson, R. R. (Eds). The Oxford Handbook of Innovation (pp.
1-26). Oxford: Oxford University Press.
Gibbons, M., Limoges, C., Nowotny, H., Schwatzman, S., Scott, P. & Trow, M. (2005).
The New Production of Knowledge: The Dynamics of Science and Research in
Contemporary Societies. London: SAGE
G-STIC 2017, key conference findings.
Hirvikoski, T. & Saastamoinen, K. (Forthcoming). Innovation Sprint – Manual for
organisers. Laurea publications.

Kivistö, S. & Pihlström, S. (2018). Sivistyksen puolustus. Miksi akateemista elämää
tarvitaan? Talinna: Gaudeamus
Lundval, B-Å. (1985). Product innovation and producer-user interaction. Industrial
Development Research Series, 31, 79-91. Aalborg University Press.
Miettinen, R., Tuunainen, J., Knuuttila, T. & Mattila, E. (2006). Tieteestä tuotteeksi?
Yliopistotutkimus muutosten ristipaineessa. Helsinki: Yliopistopaino

OECD (1971). Science growth and society: a new perspective. Report of the
Secretary-General's Ad Hoc Group on New Concepts of Science Policy.
OECD (2017). Making Innovation Benefit All: Policies for Inclusive Growth. Paris:
OECD Publishing.
Ojasalo, K., Koskelo, M., & Nousiainen, A. K. (2015). Foresight and service design
boosting dynamic capabilities in service innovation. In The Handbook of Service
Innovation (pp. 193-212). Springer, London.
Ramaswamy V. and Goiullart, F. (2010). The power of cocreation. New York: Free
Regional Cities Programme secretariat and advisory group (2018). Regional cities
programme – Independent development and multilateral partnerships.
Ministry of Finance publications 21/2018. Helsinki.
The Smart and Clean Helsinki Metropolitan (2017).
Sung, T. J. & Jou, S.C. (2011). 2011 Asian Smart Living Summer School Report.
Innovation and Connections. The Talent Cultivation Program for Smart Living
Industry. Ministry of Education R.O.C.
UN (2015). The 2030 Agenda for Sustainable Development.
Wenger, E. (1998). Communities of Practice: Learning, Meaning, and Identity.
Cambridge University Press.
Westerlund, M., & Leminen, S. (2011). Managing the Challenges of Becoming an
Open Innovation Company: Experiences from Living Labs. Technology
Innovation Management Review, 1(2): 19-25.
Wessner, C. W. (2005). Entrepreneurship and the Innovation Ecosystem Policy
Lessons from the United States. In Audretsch, D.B., Grimm, H. & Wessner W.C.
(Eds.). Local Heroes In the Global Village: Globalization and the New
Entrepreneurship policies. New York: Springer science & Business media.

Learning with Community:
Developing Citizen-Led Housing
Shaofu Huang1, Ruth Deakin Crick1,3, Melissa Mean2,
Carolyn Hassan2, Colin Taylor1
1University of Bristol
2Knowle West Media Centre
3 University Technology Sydney

Category: Research Paper

Creating a flourishing city and delivering the Sustainable Development Goals in urban
development is a complex challenge because of the interconnectedness between the
different goals as well as the diverse perspectives and prioritisation of values among
stakeholders. To navigate through this complexity and effectively orchestrate
changes at the scales needed, innovation programmes have to be co-produced by and
with stakeholders. Effective co-production requires mindful leadership and skilful
facilitation. We introduce a learning framework which helps innovators and change
programme leaders to facilitate an integrated approach to stakeholder engagement,
so they can lead their project effectively and stimulate desired, community-led
transformation. We examine the conceptual underpinnings of the framework, drawing
from evidence in the literature about motivation and decision theories, co-production,
learning, and resilient agency. We then describe how this framework has been
developed in a case study of a citizen- led housing project in Knowle West, Bristol,
UK and how the solution developed responds to the urban challenge themes of the
Urban ID project: mobility & accessibility, health & happiness, equality & inclusion,
carbon-neutral city, and environment & ecology. We highlight the importance of
framing co- production as a learning journey and the need for the effective facilitation
of both the behavioural processes of getting things done and the learning facilitation
processes which enable people to change their minds and behaviours. These lead to
the argument that mindful leadership in both thought and action and the development
of a collaborative learning infrastructure are important success factors for co-
production in Living Labs.

Keywords: citizen-led housing, urban diagnostics, learning infrastructure,

stakeholder engagement


Creating a flourishing city and delivering the Sustainable Development Goals (SDGs)
in urban development is a complex challenge which requires a holistic systems
approach to diagnose problems, innovate solutions and sustain community directed
change. This complexity is rooted in the inter-connectedness between the different
goals as well as the diverse perspectives, disciplines, domains and prioritisation of
values among stakeholders. To navigate through this complexity and effectively
orchestrate changes at the scales needed, the change programme has to be co-
produced by and with stakeholders so that there is a sustainable commitment from the
people and groups who will be responsible for delivering it.

There is an established consensus manifested in the SDGs regarding what needs to

be achieved. The pressing question is how we, as individuals, teams and organisations
from all sectors and domains, can deliver these goals in a manner which is itself
sustainable. From the authors’ experience in working with stakeholders in the urban
areas of Bristol and South Gloucestershire, we knew that many people and
organisations are passionate and committed about leading positive change and have
organically developed valuable assets, know-how and networked communities. We
therefore aimed to understand whether, and what, barriers might exist that block this
positive, organic energy for change from developing into authentic outcomes. This
was the challenge of the UrbanID research project, which aimed to develop a
framework to help stakeholders to take an integrated view on the issues or places of
their concern and to surface, and then address, any systemic barriers.

In this paper, we first attend to the conceptual understanding that underpinned the
development of the UrbanID framework, then turn the focus to the challenge and
barriers encountered in community engagement, using the case study of a citizen-
led housing project in Knowle West. We conclude by a reflexive analysis of lessons
learned in the co-production process.

Urban Integrated Diagnostics and Stakeholder Engagement

The UrbanID project was one of the five Urban Living Partnership (ULP) pilots funded
by Research Councils UK and Innovate UK to explore the transdisciplinary research
challenges of future urban living. UrbanID aimed to create a novel Integrated
Diagnostics framework and methods to address these interconnected urban

Whilst there was a serious commitment from stakeholders, the UrbanID project also
recognised that there was a collective ‘business as usual’ mindset amongst a broader
set of stakeholders in the city, which reinforced existing siloed practices and made it
difficult to engage thematically across different domains, disciplines, and commercial
and political interests. The ability to engage stakeholders in the development of a
more user-centred and joined-up view was seen as a critical success factor for the
delivery of the Sustainable Development Goals across the city.

The UrbanID framework was built upon three methodological components: systems
thinking, learning design, and co-production methods. The project design was co-
produced with a high degree of multi-stakeholder participation from the quadruple
helix of public, private, voluntary and academic sector organisations across the City
of Bristol and the South Gloucestershire local authority areas2. To stimulate a holistic,
user-led approach, five challenge themes were identified by the group:

• Mobility & accessibility;

• Health & happiness;
• Equality & inclusion;
• Carbon neutral city;
• Environment & ecology.

The UrbanID framework, as an analytical tool, focused on the challenge themes

through the lens of authentic, user-led case studies, exploring how the challenges
manifest themselves in the life- narratives of citizens, business, government, third
sector, and academic actors and impact on wider ecological systems. The framework
was developed and tested through five case studies, including two transportation
infrastructure projects, two local neighbourhood areas projects, and a leadership
organisation focused on environmental sustainability.

Learning Together to Co-produce

The emergence of co-production as a public service planning and delivery approach

was primarily driven by the need for fiscal austerity. However, co-production has since
been recognised as a more effective way to support citizens to achieve their wellbeing
(Meijer, 2016; Nabatchi Tina, Sancino Alessandro, & Sicilia Mariafrancesca, 2017).
Advocates of co-production believe that all stakeholders will participate and
contribute from their particular viewpoints, drawing on their unique capabilities, if they
can envision some value that they might gain from doing so. This belief is based on

For more details about this project visit

the assumptions that stakeholders can align around a shared goal, are motivated to
work together and are open and ready to engage with relevant diverse perspectives,
data, information and practices and to struggle together to find workable solutions.
These assumptions call for a learning design approach because what is required is a
way of facilitating or ‘scaffolding’ a process of learning together to solve complex
problems when the target outcome is not known in advance (Bauman, 2001).

This process has at least six collective core tasks. Stakeholders need to:
i) Align sufficiently around a shared purpose;
ii) Analyse the problem and any ‘blockers’ to achieving that purpose;
iii) Decide how to go about a change initiative that will achieve the purpose;
iv) Identify what a good outcome would look like and how it might be
v) Select, collect, curate and re-structure relevant data and
information into a new solution which achieves the purpose;
vi) Deliver and evaluate the outcome.

These are key tasks of collaborative learning understood as a process, or a journey,

which operates in authentic contexts, rather than in the ‘training room’ or the ‘lecture
theatre.’ Co-production refers to the involvement of all stakeholders in the process:
the learning design principles provide a ‘context-neutral’ way of moving beyond the
Why and addressing the How of going about achieving a collaborative purpose.

Understanding Motivation and Engagement

Given that collaborative learning takes time and effort and, by definition, entails a
deviation from business-as-usual, it depends significantly on stakeholders’ motivation
and willingness to engage in what is likely to be a disruptive process. A study of
motivation to learn in a performance- oriented culture shows that motivation is
influenced significantly by external factors in the immediate environment and the
cultural context of the person who is learning. Furthermore, people’s sense of agency,
choice, identity and belief structures are four key mechanisms that drive deep
engagement and achievement (Harlen & Deakin Crick, 2003; Sebba et al., 2008).

More general research into human motivation has linked people’s behaviour to their
(unmet) needs, which are understood to have a hierarchy (Maslow, 1943), as well as
being wide ranging, from subsistence and protection to creation, identity and freedom
(Max-Neef, Elizalde, & Hopenhayn, 1991). While the satisfaction of needs explains
the motivation underpinning people’s actions, how those needs are satisfied and the
extent to which those needs have been satisfied are perceived differently from person
to person. These perceptions inform a person’s construction of their goals – their

internal representations of pathways leading towards a desired state (Austin &
Vancouver, 1996). Goal constructs will recursively create a lens through which a
person understands the world and the values they live by. This self-reinforcing process
leads to the formation of a person’s worldview over time and will have already shaped
a person’s identity, relationships and expectations in his or her social network by the
time they encounter a community problem which they want to address collaboratively.

Engagement decisions in authentic contexts

Despite need satisfaction being a fundamental motivator, people do not generally

make direct, or rational, choices about which need they are to satisfy next because
needs are complex and interdependent and sometimes outside a person’s
awareness. The choice is made at a more visceral and practical level that concerns
how a person deploys effort, time or resource. It is the consequences of a person’s
actions that will lead to the satisfaction or deprivation of needs, which means the
behavioural choices are connected to needs in a many-to-many manner (Ballard,
Yeo, Loft, Vancouver, & Neal, 2016).

In authentic, real-life contexts, as opposed to experimental settings or training rooms,

people usually make decisions about their next best action amongst multiple options,
each of which has consequences which relate to multiple needs. People do not have
sufficient mental capacity to assess all the consequences of all possible actions
systematically; they consider the consequences sequentially, one after another
(Ballard et al., 2016), and do not always make deliberate choices but depend on
intuition and routines (Betsch & Haberstroh, 2005; Marchiori, Adriaanse, & Ridder,
2017). Therefore, the attention a person gives to the consequence of a choice
accounts for a significant part of a decision process, while the preferences
underpinning a decision are rooted in agency, goal or belief constructs, and needs.

A layered framework of motivation

These motivation related factors and mechanisms can be grouped into at least four
layers in an inside-outside model of motivation, in that the inside layers are more
strongly driven by internal processes than external ones whilst the outer layers are
more open to external influence factors (see Figure 1).

Figure 1. Layers of sources of influence on participant's motivation
to engage in Living Lab activities

Need Satisfaction represents an underpinning value system, which is largely a result

of the co-evolution of the human species. Fundamental human needs are qualitatively
stable and broadly similar across populations, but need satisfaction is subject to
cultural influences and will be assessed subjectively with reference to the extent and
ways in which needs are satisfied in a specific time and place (Deci & Ryan, 2000).
Most human moral judgements have also been incorporated into our basic
psychology (Greene, 2014) and therefore it is reasonable to consider “value” as an
integrated part of the complex system of human needs (Pink, 2011).

Goal Construct is a person’s internal cognitive, affective and volitional model of how
the world works and thus how a person can satisfy his or her needs. Hence, goal
constructs imply at least two sets of knowledge relating to needs: i) declarative
knowledge which characterises what is needed and the options available, and ii)
procedural knowledge about the set of actions that constitute an option. Even though
people have similar set of needs, they assign differing weights of importance to those
needs and have differing understandings of the options for achieving satisfaction.

Agency is the capability of a person to utilise the structure of a system to achieve his
or her goal (Giddens, 1984). As far as motivation is concerned, it is the person’s belief
in whether he or she has agency that influences the person’s decision to engage with
a goal. There are two agency-related beliefs: self-belief that the person can act him
or herself, and belief and trust in the agents the person needs to work with in order to
act and achieve a goal. Even though a person may over or under estimate his or her

actual agency, their agentic belief will strengthen their actual agency and the increase
of actual agency will strengthen their belief.

Framing is the interpretation of the relationships the person has with other people,
the material environment and their social context, which informs the person’s
attention and temporal importance weighting of goals (Ballard et al., 2016). To give
an example, Steele (2011) describes how female students feel the pressure to prove
their abilities in solving difficult maths problems because there is a stereotype cultural
belief that female students are less able in maths than their male peers. Alternative
task framing can remove the identity threat and nudge the students to attend to the
goal of, say, stretching their ability in solving difficult math problems, instead of
proving wrong the stereotype about their maths ability (Steele, 2011).

The learning framework of UrbanID

Framing co-production activities as learning opportunities is of critical importance to

overcome the siloed and fragmented thinking and practices embedded in
stakeholders’ business- as-usual modes of operation. This is because doing
something differently, by definition, involves learning. If we already know what to do,
we don’t need to learn and ‘business-as-usual-practices’ will be the default modus
operandi. However, innovation and change require moving into the unknown and this
involves learning. To approach new learning opportunities (i.e. change) productively
requires learning power, which is ‘the relational and embodied process through which
we regulate the flow of energy and information over time’ in pursuit of a purpose
(Deakin Crick, Huang, Ahmed Shafi, & Goldspink, 2015, p. 121). Eight dimensions of
learning power emerged from five studies with more than 50,000 cases: Mindful
Agency; Hope and Optimism, Sense-Making, Creativity, Curiosity, Belonging,
Collaboration and Openness to Learning. These are malleable qualities which
emerge from the interactions between the learner and their environment (ibid.).
Learning power is influenced by relationships and social culture – the data suggests
that where a culture is characterised by trust, affirmation and challenge then people
are more likely to move into new ways of thinking and acting, mobilising their learning
power in pursuit of their goal (Deakin Crick & Joldersma, 2007; Deakin Crick,
McCombs, Haddon, Broadfoot, & Tew, 2007). The implication of this for a Living Lab
is that when activities are designed in a way that enable people to utilise and develop
their learning power e.g. creativity, curiosity, mindful agency, sustainable learning
relationships etc, it will not only help people learn better together but it also
incentivises future participation and goal achievement.

The layered framework of motivation clarifies the depth of influence required for

significant behavioural change at scale within a community and highlights the
importance of learning design in stakeholders’ engagement in change and innovation.
Substantive change occurs when a person’s understanding of their needs and their
worldview is modified, and when they have formed sufficient agency to pursue a new
goal; and a person’s agency and goal constructs change only through learning.

This layered framework of motivation, the tasks of collaborative learning, and holistic
systems thinking constitute the learning framework of UrbanID. In the following
sections we describe how the framework was developed and applied through a
particular UrbanID case study focused on citizen-led housing development.

We Can Make: The Citizen-Led Housing Project

Knowle West Media Centre (KWMC) is an arts centre, founded in 1996 and rooted in
the community of Knowle West, Bristol, UK. KWMC works across different disciplines
using art and technology to address issues including health, housing, and smart
cities. KWMC runs Bristol Living Lab and as such fosters civic innovation by
collaborating with residents, artists, local authorities, business and academia to
support people to make positive change and explore new ways of living better

The We Can Make project is a response to community demands and concerns about
housing need in the area. The project team carried out extensive participative
research in Knowle West, a housing estate on the outskirts of Bristol consisting of
5,500 households and 12,000 residents. Knowle West is a low-density estate – below
25 dwellings per hectare which is below the threshold to justify a regular bus service.
There are few sources of local employment, public transport is poor, and the
neighbourhood remains geographically isolated. The low population density makes it
difficult to sustain shops and services, with many closing down or moving elsewhere,
including the cinema, and swimming pool, and presently the library which is under
threat of closure.

Nevertheless, the neighbourhood has a high level of housing need: 522 people in the
Knowle West area are on HomeChoice, the Council’s official housing waiting list.
Knowle West represents 2.9% of the total population of Bristol, and 5.8% of people
registered on HomeChoice are from the area – double its proportionate size.

Co-identification of the need

Over a year, in 2017, We Can Make brought together local people, artists, architects,

policy- makers, academics and industry professionals to collaborate between and
beyond traditional professional and social silos. The aim was to understand the needs
of people from their perspective, identify the nature of the problem, collate and map
the assets and resources the community had available, and co-construct new citizen-
led approaches to enable people to supply their own affordable housing at the point
of need. This process maps on to the six learning design principles described above.
The diagnosis of the issue of housing need was developed through a door-to- door-
survey, narrative interviews, and semi-structured conversations at workshops and on
doorsteps. Four types of unmet housing need were identified:

• Down Shifters: individuals or couples who want to stay in their

neighbourhood but whose house is now too big for them and are looking for
a smaller home. This could be because children have grown up and flown the
nest, changing mobility needs, or a need to reduce costs in retirement.
• New Shoots: families where there is an urgent need for more space because
children are growing up and seeking independence or are having offspring of
their own, or caring responsibilities are expanding to include additional elderly
or disabled family members.
• Better Fits: families where one or more member(s) is experiencing changing
mobility and the family home needs to be adapted to enable them to stay close
to their support networks.
• Making Ends Meet: individuals and families that need extra income or
reducing their housing costs due to changing circumstances.

Co-diagnosis of barriers

The project team researched the assets and resources people had in the
neighbourhood and found there was no shortage of change-ready attitude, know-how
in construction and trade, spare capacity and land ( Figure 2). The project also
identified the barriers to citizens and their communities in applying some of these
assets and resources to how people could go about supplying and securing their
affordable housing at the point of need:

• Conventional strategies to access housing are reductively competitive. They

either require people to divert ever more of their wages and savings to getting
on a property ladder, where the bottom rungs are missing, or compel people to
prove how ‘weak’ or ‘incapable’ they are in order to win eligibility for austerity
rationed social housing. These dominant strategies can cause further damages,
such as diverting capital to be locked in housing stock rather than other forms of
investment that can circulate in the economy more productively, and

undermining an individual’s sense of personal agency.

• An emerging alternative is the “third sector” – the civic or community sector.

However, despite enabling legislation at national level and new forms of low-
interest finance, the take-up so far has been slow – self-build and custom build
account for just 7% of new homes delivered each year in the UK, compared to
80% in Austria and 60% in France.

• A significant barrier is caused by the deliberately veiled nature of the

development process. Access to land, technical know-how, and financial
resources continue to be controlled by professional interests and has
disproportionate costs and barriers to entry; the current housing system is more
often working against than supporting the civic or community approach. For

i) Land: most citizen-led schemes range from 1-10 homes, a size that is
just too small and carries disproportionate transaction costs for most
council site allocation and land disposal processes to bother with. At
the same time the long-term social and economic impact of citizen-led
housing struggles to make its added value count in a highest-sale-
price-wins market. The result is that communities are often left with the
most difficult sites, which commercial developers have rejected or are
overly dependent on increasingly stretched local authorities as a route
to get land at a discounted price.

ii) Finance: at each stage of the development process – from a lack of

pre-planning development finance, the high cost of social investment,
to a lack of capital reserves to accommodate construction cost
overruns – risk and uncertainty accumulates. The high transaction
costs of sticking together small pots of multiple funds each with
different strings attached also creates substantial drag on community-
led projects.

iii) Technical know-how: from planning permission to finding contractors,

the process of making homes is dominated by costly professionals. The
inequity this creates is clear – in Knowle West, planning applications
are twice as likely to be rejected compared to wealthier
neighbourhoods in Bristol.

iv) No repeats: Most community-led projects remain non-replicable and

the experience gained – often through years of ‘hard graft’ – by
individual residents stays stuck to specific sites with few people wishing
to repeat the experience. The result is high per- project overhead costs
in terms of both time and money as each new amateur has to learn and
master the process from scratch. The isolation of individual projects
also means the citizen-led asset base – which might be able to fund the
next project – fails to grow.

The systemized opaqueness and uncertainty mean many individuals and

communities do not know what tools they need, cannot afford them anyway, and
ultimately, do not have the faith that they can achieve the outcome they want – a safe,
secure and affordable home. Using the terms of the layered framework of motivation,
community-led housing was not an option in people’s existing goal constructs nor did
they have the agency required to act upon this option. These findings informed the
project’s next key task which was to co-design the collective tools that could help
overcome these barriers.

Figure 2. Assets the Knowle West people and community had to utilise as captured by the WCM

Learning together to create an alternative approach

To lead the community through the transformative journey from a business-as-usual

mindset to an innovative new approach, We Can Make utilised a mix of open research

and development tools and approaches to stimulate learning:
• Co-design of workshops which mixed residents, artists, and housing
professionals to explore issues and ideas together in an open, learner-
centred and non-hierarchical way;
• The involvement of artists to help open up learning conversations, invite new
perspectives, translate between different communities and types of
participant and create shared learning experiences, spaces, and new
• Using visualisations to unravel and de-mystify complex issues and ideas and
thus empower learning;
• Testing and re-testing emerging ideas and propositions with a wide range of
stakeholders – including residents – learning and evaluating together as the
project developed.

Figure 3. An illustrative graphic which helps citizens understand the community-led housing
process from the perspective of a user whose need is Better Fits, which is one of the four types
of needs identified in We Can Make.

Overall, We Can Make developed a different kind of user-led learning journey for
people to find and realise ways to access affordable homes, attuned to their needs
and resources, and created collective tools to support the individual journeys as a

process of learning. These processes were facilitated and captured using creative
artefacts. For example, the project represented these individual learning journeys
through illustrative graphics which help citizens understand the community-led
housing process from the user perspective (Figure 3).

To make the community-led housing option an authentic experience and thus deepen
the community learning, We Can Make then built a prototype home to pilot the
development process from end to end, demonstrating with the community how some
of the barriers could be overcome in practice. The pilot process involved the following

• Identifying a micro-site next to a community centre;

• Development of community criteria for construction and design including
local materials, local labour, sustainability, and affordability;
• Use of existing planning rules to permit the development;
• Employing local construction teams;
• Artists and local people designing and making fittings and furniture;
• A rapid prototype home built over 3 months;
• Local people experiencing and testing the home by staying in it.

The prototype home created an embodied space where people could see, feel and
help make it happen, learning through experience. This is of critical importance in
helping people incorporate this innovative approach of housing into their existing goal
constructs; it demonstrated that a different way of imagining, making and accessing
an affordable home is possible, as opposed to just a concept or topic for consultation.
The community learning was deep and transformative through ‘playing, making and
doing’ (Thomas & Brown, 2011).

Co-generation of community agency

A co-design approach was used throughout the research project with residents
working alongside professionals to develop and test ideas, which inspired the
community to develop their own ‘agency’ and become the developer. The prototype
home has now had over 100 people staying in it, including locals and people from as
far away as Berlin, San Francisco, and Melbourne. Local people stay for free to test
out living in the home, and non-locals pay through AirBnB. The income is shared with
a community centre who provided the micro-site, and in four months has contributed
10% of the centre’s annual income.

KWMC also seeks funding and support to increase the resourcefulness of the

community and generate the ‘community agency.’ The We Can Make project has
now been identified as a priority pathfinder project by Bristol City Council. The project
has now secured support from Power to Change and the Nationwide Foundation to
develop the collective tools, including a land assembly mechanism, financial tools,
community design code, community supplier’s framework, and digital learning
platform, to support the development and delivery of We Can Make in the pilot
neighbourhood during 2018 and 2019.


We Can Make’s achievements have arisen because sufficient people became

motivated to work together to achieve a mutually beneficial common purpose. Such
motivation is not a given; it is an outcome of a learning process that each individual
participant and organisation goes through. The success of We Can Make is the result
of the creative, purposeful and engaging approach to orchestrate the stakeholders’
collective learning journey, not simply following the script of the UrbanID Learning
Framework presented in this paper. Nevertheless, the framework provides a useful
working theory and set of design principles which explain the stakeholder
engagement demonstrated in the We Can Make project and the quality of its

First, establishment of a shared set of learning design principles helped to direct

stakeholders’ attention to the intended shared goal. Framing the project’s co-
production process – from identifying the problem, diagnosing the ‘blockers’, co-
constructing and achieving an alternative goal – created a social environment that
generated collaborative learning power, motivation and agency and therefore led to
deep stakeholder engagement. Using the challenge themes as an analytical lens also
prompted stakeholders to think systemically, leading to the development of a solution
that is expected to have positive impact on most of the challenges themes. These links
are presented in Table 1 below.
Second, there were three processes running in parallel throughout the We Can Make

i) The identification of the problem and the ‘job to be done’;

ii) The behavioural processes of co-production, including but not limited to

those key tasks of collaborative learning discussed in section 3, which
signpost the pathway forwards for stakeholders to navigate and so
identify, collect and curate the necessary resource, data and information
which will enable them ‘co-construct’ a solution and thus achieve their
purpose – or ‘do the job’;

iii) The intra- and inter-personal learning processes, as discussed in the
layered framework of motivation: a meta-level, internal (and often
invisible) element of co-production which is necessary to sustain
engagement and lasting changes.

The behavioural processes and the learning processes, as we observed in the case
study, were coupled like a double helix around the ‘job to be done’. When both
processes are facilitated and integrated appropriately around the ‘job to be done’ they
are mutually reinforcing. Conversely, if either of the two processes is neglected or if
they are managed separately as in isolated silos, it will limit the transformative
potential of co-production. The job may still get done, but it will be less sustainable
because the learning and knowledge co-production processes will not be embedded
in the hearts and minds of the community and its stakeholders.

Table 1. The expected impact of We Can Make approach on the UrbanID examplar challenge theses

Challenge theme Community-led Increase of local Mobilisation of

housing capability population community
density resource
Mobility & More variations of Higher density makes
accessibility house designs to suit it more cost and
the needs of people energy effective to run
with varied mobility and infrastructure
accessibility services, including
Health & User-centred public transport, and Enhances collective
happiness development means the other locally facilities, and individual
new houses better meet providing easier and agency and
the quality required for a less discriminated relatedness
health and happiness access to essential
services for well-
Equality & WCM Financial Tools Enabling much more
inclusion and Community Supplier diverse ways to
Framework offers job contribute to and
and investment benefit from urban
opportunity for local living than an
people to generate and exclusive mode of
retain value against the job earning and
potential raise of house money spending
price as a result of
Carbon neutral The prototype home Increased
city was built to high resourcefulness
environmental means greater
standards (it is built of ability and
compressed straw); willingness to adopt
Developing carbon latest technologies
saving techniques and which in general are
methods is part of the more environment-
community learning friendly
process and will be
embedded into the
WCM assets
Environment & Better use of land in existing residence
ecology reduces the development pressure on wild
green space

The third reflection is about the role that digital and visual artefacts and assets played
in the We Can Make project. In a related community engagement study with Hunter
Water Corporation3 in NSW, Australia, the components of a digital ‘always on’
community engagement platform have been developed based on the learning
framework identified in this study. In its early stages, based on a data architecture
which offers a ‘single view of the user’ it maps out the key tasks of the behavioural
processes of co-production – or learning with the community - framing these around
the ‘job to be done’, whilst identifying key trigger points for critical learning processes.
The ‘job to be done’ in this example is to secure a resilient water future for the
community – the content-neutral learning framework discussed here provides the
‘scaffolding’ (see Figure 4).

5. How

Create Establish Create Reflect and

Awareness Purpose Take Action Enjoy benefit learn

Purposeful ‘Learning With‘ Journeys

© Decisioning Blueprints Ltd 2017

Figure 4. The community engagement platform architecture

developed by Learning Emergence Lab4

The importance of assets has been well understood by KWMC who have sought
support and investment to advance their development of assets as part of the learning
infrastructure of the community. The notion that this learning infrastructure has to
include both learning facilitation services and physical and digital assets is consistent
with the findings from a study of innovative business model for infrastructure (Crick,
Huang, Godfrey, Taylor, & Carhart, 2017).

While it is widely accepted that co-production is incompatible with managerial control

4 Learning Emergence Lab is the innovation engine of Learning Emergence Partners, a global
collaborative of practitioners, data architects, business consultants and academics and a Community
Interest Company. It is based in Scotland, UK.

exercised through a ‘top-down’ power structure (Meijer, 2016), it takes more than a
simple reverse of power into a ‘bottom-up’ power structure to orchestrate effective co-
production. What we have highlighted in this case study was the Living Lab as an
organisation committed to the provision of a community service, which introduced
mindful leadership in thought, action and relationships to energise and scaffold a co-
production process which led to the emergence of a new social and industrial
structure. This was the ‘job’ they set out to do: enabling the community to solve its
housing problems. They did this through the intentional and visible use of (i) learning
design principles in their mode of communications and relationships and (ii) the
deployment of invited behavioural processes – key tasks that structured the journey
from purpose to performance.


In the paper we presented the Learning Framework of the UrbanID project which was
developed through participatory, user-led research. Through the case study of citizen-
led housing, we illustrated the importance of framing and scaffolding co-production
as a learning journey. We argued for the value of using learning journey design
principles to facilitate the ‘job to be done’ through the interconnected behavioural
processes and learning processes which empower stakeholder engagement in co-
production. An additional insight emerging from our analysis is the critical role digital
and visual artefacts and assets play in harnessing learning across the community over
time. These insights led us to recognise that mindful leadership in thoughts and
actions – leading to the provision of a community focused learning infrastructure - are
important success factors for co-production in Living Labs.


The Authors gratefully acknowledge the financial support of UKRI through the Urban
Living Partnership project, UrbanID, grant reference EP/P002137/1. The UrbanID
project involved a large team of collaborators who worked on other interconnected
case studies. The Authors gratefully acknowledge the contributions of their project
colleagues to the background discussions that informed the learning framework
development described herein.


Austin, J. T., & Vancouver, J. B. (1996). Goal constructs in psychology: Structure,

process, and content. Psychological Bulletin, 120(3), 338.
Ballard, T., Yeo, G., Loft, S., Vancouver, J. B., & Neal, A. (2016). An integrative formal
model of motivation and decision making: The MGPM*. Journal of Applied
Psychology, 101(9), 1240–1265.
Bauman, Z. (2001). The individualized society. Cambridge: Polity Press.
Betsch, T., & Haberstroh, S. (Eds.). (2005). The Routines of Decision Making (1
Mahwah, N.J: Psychology Press.
Crick, R., Huang, S., Godfrey, P., Taylor, C., & Carhart, N. (2017). Learning Journeys
and Infrastructure Services: a game changer for effectiveness (advanced copy).
In T. Dolan &
B. Collins (Eds.), ICIF White Paper Collection (in Press). London: UCL Press.
Deakin Crick, R., Haigney, D., Huang, S., Coburn, T., & Goldspink, C. (2013). Learning
power in the workplace: the effective lifelong learning inventory and its reliability
and validity and implications for learning and development. The International
Journal of Human
Resource Management, 24(11), 2255–2272.
Deakin Crick, R., Huang, S., Ahmed Shafi, A., & Goldspink, C. (2015). Developing
Resilient Agency in Learning: the internal structure of learning power. British
Journal of
Educational Studies, 63(2), 121–160.
Deakin Crick, R., & Joldersma, C. W. (2007). Habermas, lifelong learning and
citizenship education. Studies in Philosophy and Education, 26(2), 77–95.
Deakin Crick, R., McCombs, B., Haddon, A., Broadfoot, P., & Tew, M. (2007). The
Ecology of Learning: Factors Contributing to Learner Centred Classroom
Cultures. Research Papers in Education, 22(3), 267–307.
Deakin-Crick, R., Sebba, J., Harlen, W., Yu, G., & Lawson, H. (2005). Systematic
review of research evidence of the impact on students of self-and peer
assessment. EPPI Centre Social Science Research Unit Institute of
Education University of London, London Google Scholar.
Deci, E. L., & Ryan, R. M. (2000). The" what" and" why" of goal pursuits: Human needs
and the self-determination of behavior. Psychological Inquiry, 11(4), 227–268.
Giddens, A. (1984). The constitution of society: outline of the theory of structuration.
Cambridge: Polity Press.

Greene, J. (2014). Moral tribes: Emotion, reason, and the gap between us and them.
Penguin. Harlen, W., & Deakin Crick, R. (2003). Testing and motivation for
learning. Assessment in
Education: Principles, Policy & Practice, 10(2), 169–207.
Marchiori, D. R., Adriaanse, M. A., & Ridder, D. T. D. D. (2017). Unresolved questions
in nudging research: Putting the psychology back in nudging. Social and
Personality Psychology Compass, 11(1), e12297.
Maslow, A. H. (1943). A theory of human motivation. Psychological Review, 50(4),
Max-Neef, M. A., Elizalde, A., & Hopenhayn, M. (1991). Human scale
development: conception, application and further reflections. New
York: The Apex Press.
Meijer, A. (2016). Coproduction as a structural transformation of the public sector.
International Journal of Public Sector Management, 29(6), 596–611. 01-2016-0001
Nabatchi, T., Sancino A., & Sicilia M. (2017). Varieties of Participation in Public
Services:The Who, When, and What of Coproduction. Public Administration
Review, 77(5), 766– 776.
Pink, D. H. (2011). Drive: The surprising truth about what motivates us. Penguin.
Steele, C. M. (2011). Whistling Vivaldi: how stereotypes affect us and what we can
do. New York: WWNorton & Company.
Thomas, D., & Brown, J. S. (2011). A new culture of learning: Cultivating the
imagination for a world of constant change (Vol. 219). CreateSpace Lexington,
Thomas, D., & Seely-Brown, J. (2009). Learning for a World of Constant Change:
Homo Sapiens, Homo Faber & Homo Ludens revisited. Presented at the 7 th
Glion Colloquium. Retrieved from
for a World of Constant Change.pdf

Participatory software development via Living Labs:
A case of Deajeon
Hee-Sook. Yoo1, Su-jin. Jung2,
Seong-gyu. Park3, Miran. Cho4

1,2 National IT Industry Promotion Agency (NIPA), the Republic of Korea

3,4 Deajeon Information & Culture Industry Promotion Agency (DICA)

Category: Innovation Paper

Over the past few years in Korea (Republic of Korea), demands for welfare policies
to prepare for an aging society and to secure a social security have been grown.
Responding to the growing needs of people, Korea Government is running the
research and development (R&D) projects using Information Technologies (eg.
Software, IoT, AI, big data etc.) to make public services which resolve the social
issues efficiently. The Living labs is considered as one of research and development
methods, which aims to close the gap between technical and social innovation by
enhancing collaboration between different relevant stakeholders (Kopp, Haider, &
Preinesberger, 2017, p.61). Hence, NIPA (National IT Industry Promotion Agency)
has tried applying the Living Labs on software development projects to make software
solutions for local community. Compared to the conventional research and
development projects, Living Labs has a slower pace spending more time and efforts
on communication with relevant stakeholders (issue exploration, demand definition
etc.), but it’s more effective way to reflect concrete demands of users on research
and development projects.

Keywords: Living Labs, software development, co-creation, software solution,



The Ministry of Science and ICT (referred to “MSIT”) is responsible for the policies
on Science and Information communication technology research and development in
Korea (The Republic of Korea). National IT Industry Promotion Agency (referred to
“NIPA”) is a government affiliated organization that makes action plans for the ICT
Research and Development Projects and supports organizations who conduct ICT
research and development projects - such as research institutions, universities,
business entities, local government agencies - to put MSIT policies into practice.

Recently, NIPA has initiated the research and development projects using Living Lab
methodology, which use software technology to resolve issues for a local community.
The projects are in progress as a pilot. Based upon the result of it, NIPA is planning
to expand the adoptions of Living Lab methodology to the software research and
development project. In every project using Living Labs, the participations of different
stakeholders are crucial factors. Among the projects NIPA supports, with the case of
Deajeon, this paper explains how to apply Living Lab methodology to software
research and development project, and how to draw the stakeholders’ collaboration
on Living Lab process.

R&D toward Technological Advancement- Versus Social Innovation

Since 1990s, Korea Government has invested huge amount of funds on Information
Technology (eg. software, big data, cloud, IoT, artificial intelligence etc.) research
and development. Majority of them led by technology professionals - such as
research institutes, universities, corporations etc. - , and aim to produce economic
value by developing advanced technologies dealing with industrial issues.

In the 2000s, people have been concerned about social welfares for an aging society
and a daily living safety. Responding to these national demands, Korea Government
has tried to take advantage of ICT (Information and Communication Technology) to
make a better life of its people. Government driven projects to develop public services
using ICT were carried out in a wide range of areas, such as healthcare, education,
emergency management, food sanitation etc. However, the outcomes of them had
difficulties in applying to people’s real life and commercialization. The conventional
research and development projects are led by technology professionals, such as
research institute, universities, corporations etc. In such way, embracing the opinions
of ordinary people to the development project was limited. The majority of public
service end-users is ordinary people. Many of them are not familiar with cutting-edge
technologies, and reluctant to try new things required much learnings how to use.
First and most important thing to ordinary people is not whether up-to-date

technologies are applied in it, but whether the services (the result of development
project) could resolve their inconveniences or not.

Eriksson, Niitamo, and Kulkki (2005, p.6) and Lepik, Krigul, and Terk (2010, p.1091)
consider the Living Labs as a research and development methodology, which aims
to close the gap between technical and social innovation and to enhance
collaboration between different relevant stakeholders. In this regard, NIPA went into
applying Living Labs to software service development project to resolve social issues
of local governments. Considering that Living Labs was not familiar to employee of
local government agencies, NIPA provided a three-month “Design Thinking” and
Living Labs methodology training program to local government agencies’ employee
beforehand, in 2016. After the training program, NIPA encouraged those trainees to
become Living Lab facilitators and to do applying Living Labs to their software
research and development projects. As a trial basis, Living Labs has been
implemented to a part of their projects.

Deajeon City Case: The Household Wastes Treatment in Old Town

Deajeon is a distinctive area that national research and development institutes are
clustered. People of Deajion have a positive attitude to adopt technologies relatively.
Deajeon City wanted to resolve the inconvenience of citizens efficiently using
software technologies. Deajeon Information & Culture Industry Promotion Agency
(reffered to as “DICA”) is a local government affiliated organization that executes the
ICT Research and Development policies for Deajeon City.
Since 2016, NIPA and DICA have conducted “Soft Town” Program which aims to
enhance Deajeon citizens living by applying software technologies. NIPA and DICA
agreed that the conventional way of research and development was not fit to deal
with social issues, and not suitable for embracing opinions of people sufficiently. In
that regard, they decided to adopt Living Labs to a part of their software service
development project in “Soft Town” Program from 2017.

Software Service Development via Living Labs

To gather the opinions of citizens and made an effective solution to citizen, NIPA and
DICA would like to make process software development project along with local
people with a systematic research and development framework. iLab.o’s
methodology is based on the social construction of technology framework, which
suggests that technology is shaped by the user (in this case, “deajeon citizens”) and
highlights the importance of context in the process of endowing technologies with
social meanings (Almirall, Lee, and Wareham, 2012, p.13-14). Users are considered
the central focus and facts and meanings are the results of social process (Almirall

et al., 2012, p.13-14). The methodology consists of four phases aimed at
understanding the context where the technology will be adopted and emphasizing the
changes in meanings that this adoption will produce.

The object of the project is drawing the positive changes to Deajeon citizens with
software technologies. Going through the Living Labs process with Deajeon people,
NIPA and DICA wanted to understand what kind of problem they really have, and
draw the necessary and effective software solution. In this sense, we decided to apply
adapted version of iLab.o’s methodology to the Deajeon Living Lab projects.

Figure 1. Adapted iLab.o’s methodology


The contextualization phase aimed to get the issue relevant information. The
information was used to organize a group of stakeholders for the project process
(Almirall et al., 2012, p.13-14). To collect the information sufficiently, in this phase,
we conducted ① Documentary Survey, ② Deajeon City officials Interview, ③ Issue-
relevant-spot investigation and Fact-check, ④ 1st Professional Group Discussion
and Citizen Survey, ⑤ 2nd Professional Group Discussion and Panel Group Define.
In this process, we focused on having a good grasp what are the real issues deajeon
citizens have and who are the issue relevant stakeholders.

i) Documentary Survey
To grasp the need of Deajeon Citizen preliminary, survey reports, statistical data, and

policy materials that were published by Deajeon City within two years were reviewed.
As a result of the literature review, sixty two themes of Deajeon City were listed-up.
These were categorized into six fields (industry, culture, transportation, community
welfare, senior citizens, environmental pollution). Then, DICA found out that the six
fields were relevant with twenty two departments at Deajeon City Hall.

ii) Deajeon City official Interview

In accordance with the documentary survey analysis, managers at the twenty two
departments were interviewed. During the interview, we inquired what policies they
conducted, what kind of inconveniences that citizens asked to them, how much the
inconveniences were resolved. Then, we identified the issue of each department and
the stakeholders of such issues.

iii) Issue-relevant-spot investigation and Fact-check

Based upon interviews, people in charge of Living Lab in DICA visited the issue
relevant spots, and did interview with the stakeholders at the spots. Conducting on-
the-spot investigation, they did cross-check the analysis of documentary survey and
official interview with real situation. Through this activity, they checked whether the
information from documentary survey and city official interview was true or not.

iv) 1st Professional Group Discussion and Citizen Survey

The professional group was organized with people from various fields (public
administration, engineering, economist, business administration, statistics etc.). They
discussed about urgency, feasibility, and influence of the issues gathered from the
above a, b, and c phases. Going through five discussions, 4 priority issues -
residential environment improvement, public transportation, school violence, senior
citizen safety - were defined. And the survey to identify the severity and causes of
those issues was designed by the professional group. For two weeks, we conducted
a survey of 1,000 Deajeon citizens aged 15 years or older, and analyzed the specific
issue condition and inconvenience etc.

v) 2nd Professional Group Discussion and Defining Panel Group

As the last phase of Contextualization, based upon the analysis above mentioned in
a, b, c, and d phases, the professional group compared with what kinds of problems
happens now (real situation), and how do citizens think what problem they have
(perception). Based upon this analysis, considering the severity of issues and the
feasibility to resolve, the professional group discussed about the above survey
results, and selected residential environment improvement as a top priority issue to
be resolved. Then, they identified issue relevant stakeholders, and defined the details
of the living Lab panel organization.

Figure 2. Citizen Survey Result


Almirall et al. (2012, p.13-14) state that the key element in this phase is capturing the
issue condition that can be later compared with the changed condition with the
introduction of the new technology or the innovation to be validated. In this regard,
during the Living Lab process, we tried to define the real issue condition that should
be resolved, and draw the positive stakeholders’ participants efficiently.
To make specific issue condition to get solved and to define the problem Living Lab
solve, DICA organized the panel of Living Lab with local residents, local government
associates, NGOs, and professionals from various fields.

At the beginning of DICA Living Lab, when the panels exchanged their opinion and
shared their ideas, they provided much unconfirmed and false information. Some
wanted to bring personal complaints to the table, and some talked about the issue
condition which had already resolved. Caused of the difference of individual view
point, sometime small conflicts were aroused between panels.

To reduce inefficiency of co-works and to make them understand what the Living Lab
is, DICA - as a facilitator - provided a program that introduced the concept and cases

of Living Labs. In addition, to make the panels more easily interchange each thoughts
and opinion to explore the solution, DICA also provided “Design Thinking” - a problem
solving methodology that emphasizes user’s participation on research and
development - training program to them. Consequently, as the Living Labs
processed, the participation of panels was more active, and their communication
became more focused on the issue Living Lab deals with.

Figure 3. Living Labs & Design Thinking Training

Through these efforts, the top priority problem of Living Labs was defined as ‘The
Household Wastes Treatment in Old Town. Household Wastes should be separated
to recyclable, non-recyclable, and food waste. In old town, waste was not properly
separated, and also not collected in time. These make spoil the residential
environment, and cause pollution and safety accident.

Figure 4. The Household Wastes Disposals in Old Town

Panels reviewed the materials from contextualization phase, and shared their
experiences, thoughts, and opinions. Simultaneously, they visited the most serious
spot “Shinsung-dong” to grasp the real specific problem condition. They met 250
residents and interviewed them.

During the visit, professionals on Living Labs, Design Thinking and NGO made
residents actively give their opinion. Local government officials made residents notice
the relevant information they wanted to get. Technicians considered how technology
could be used to solve the problem.

With this visit, panels summarized the causes of the problem that residents pointed
out. Many residents did not have enough information how to separate the waste, and
they did not know where the waste should be put out. Also, they confused the time to
collect the separated waste.

Figure 5. Drawing solution process

As a result, panels agreed that the most effective way to resolve the inconvenience
in short time was distributing software service application that provide the information
– how to separate, where the waste is put out, the waste collection schedule.

DICA has made action plans for Deajeon City ICT research and development
policies. They have a lot of experiences making and conducting software research
and development project. Drawing co-work from panels, they designed requirement
specification for the application that would be able to resolve the problem. A local
software company was selected to develop the application, and the prototype
development was completed in Dec. 2017.

Figure 5. The requirement specification for the application & the prototype application

Implementation and Feedback

The test and validation in real life was carried out in the implementation phase. In the
feedback phase, the ex post measurement was conducted. The results were
compared with those obtained in the contextualization and implementation phases
and used to infer and produce recommendations on the concrete diffusion and
implementation of the technology (Almirall et al., 2012, p.13-14).

Deajeon City and DICA selected Shinsung-dong, as a testbed. Since the first quarter
of 2018, the application which provide the information – how to separate, where the

waste is put out, the waste collection schedule etc. – has been distributed to
Shinsung-dong residents.

As the distribution of the app was done recently, we don’t find rapid changes that the
application makes. However, during this phase, the more feedbacks panels gather
and have a communication with users (residents), the newer service demand and
idea are drawn. As the phase is processed, users’ commitments to Living Lab have
been grown. They positively recognize Living Lab as a place where they are able to
talk about their problems with local government decision makers, and actively share
their experiences and opinions. Residents want that Living Lab deals with other local
community issues.

In addition, in the process of implementation, the wastes collectors asked the

software application for them. This was discussed on Living Labs, and DICA Living
Lab has begun to design requirement specification for the application that senses the
condition of piling up and give the information to the collectors.
Besides the waste issue, DICA also accept the opinions from stakeholders (panles,
residents etc.) that Living Lab should be applied to other issues. They make a plan
to build a big data system to draw social issues and define problem for local
community more precisely.

5 Conclusion

As a result of conducting Living Lab project for one year, we identify that the
followings are important to enhance stakeholders’ commitments and Living Lab

First, the shared concept and objective of Living Labs is the efficient way to give
Living Labs motivation to stakeholders. As mentioned above, at the beginning phase,
inefficient communication and conflicts among stakeholders existed. We resolved
such conditions with Living Labs and Design Thinking training program. With the
program, we tried to make understood stakeholders what they do, what kinds of result
they can draw, how they make changes to their community with Living Labs. The
more clear-cut mission Living Lab facilitators give to stakeholders, the stronger
reasons they commitment to Living Lab are formed.

Second, the on-spot investigation is the way of filtering inefficiency for Living Labs. In
Contextualization and Concretization phase, we visited issue-relevant-spot to define
what condition is real problem by checking whether what we discussed and analyzed
with stakeholders. Going through the on-spot investigation, some distorted or polluted

information can be filtered. It is also very helpful to gather enough resources to decide
for setting priorities among the several issues. Furthermore, by communicating with
people on the spot, many ideas and issues - not be discussed, but should be dealt –
can be gathered. In this regard, the on-spot investigation is a good information
filtering tool, as well as an ideation method.

Third, data is the key to draw real issues. In the processing contextualization and
concretization, DICA Living Lab tried to compare the reality with the perception of
people to define real problem. People would sense their condition more or less
serious than actual condition. Data, such as statistical data, data from e-government
etc., can be the criteria to determine real situation. In this project, we used such data
to check out fact condition and to filter distorted information. Defining real problem is
the phase of building a Living Lab project objective. The use of data can be helpful to
set a direction Living Lab heads to. In recent, big data technology have been rapidly
enhanced, and widely used in various fields. Considering this big wave and the Living
Lab experience, DICA begins to build a big data system for Living Labs.

As a short implementation period, we are not able to find out the drastic changes
Living Lab lead to though, we did many experimental attempts with a systematic
Living Labs framework. As a result, some factors that make enhanced commitments
of each stakeholders and Living Lab Efficiency – sharing Living Lab concept and
cases with panels, problem solving methodology (ex. design thinking) training
program etc. - are identified. To build knowledge to make an advancement of Living
Labs, the details of the above founding should be dealt in later studies.

6 Acknowledgements

This work was supported by National IT Industry Promotion Agency (NIPA) grant
funded by the Korea government (The Ministry of Science and ICT).


Almirall, E., Lee, M., & Wareham, J. (2012). Mapping living labs in the landscape of
innovation methodologies. Technology innovation management review, 2(9).
Dutilleul, B., Birrer, F. A., & Mensink, W. (2010). Unpacking european living labs:
analysing innovation’s social dimensions. Central European journal of public
policy, 4(1), 60-85.
Eriksson, M., Niitamo, V. P., & Kulkki, S. (2005). State-of-the-art in utilizing Living
Labs approach to user-centric ICT innovation-a European approach. Lulea:
Center for Distance-spanning Technology. Lulea University of Technology
Sweden: Lulea.
Kopp, U., Haider, C., & Preinesberger, J. (2017). Systematic tools to better identify
and understand stakeholder roles and relations in Living Labs. In Research
Day Conference proceedings 2017 (p. 60).
Lepik, K. L., Krigul, M., & Terk, E. (2010). Introducing Living Lab's Method as
Knowledge Transfer from one Socio-Institutional Context to another: Evidence
from Helsinki-Tallinn Cross-Border Region. J. UCS, 16(8), 1089-1101.

Transnational piloting for smooth internationalization
of health-tech start-ups
Päivi Haho1, Metropolia and Virpi Kaartti2

1 Metropolia University of Applied Science

2 Laurea University of Applied Science

Category: Innovation Paper

Health-tech business is increasing all over the global markets. Startups need agile
and cost-efficient methods to study the needs and opportunities in foreign markets.
Thus, the purpose of this paper is to advance the understanding of
internationalization of health-tech startups and to discuss the practical issues related
to transnational living lab practices based on a case study of health-tech startups.
Finally, a preliminary model for transnational health and wellbeing living labs is

Keywords: transnational living lab, lean startup, lean global startup,

internationalization, health-tech

1 Introduction

Lean start-up and lean innovation terms have been adapted from lean manufacturing
meaning eliminating waste, the non-value-creating efforts, to emphasize the core
idea behind the lean innovation and lean startup methodology (Rasmussen & Tanev,
2015). Later, lean term has been applied in similar contexts, e.g. in software
development, lean development, and lean enterprise (e.g. Ojasalo & Ojasalo, 2018).
Besides business development, lean startup methodology can be used to support
internationalisation of “lean global startups” (Rasmussen & Tanev, 2015). Living labs,
as local, agile and networked operators in the target market, could support “lean
global startups” to validate their business model, learn from customer experiences
and cultural aspects and identify the right partners and channels for marketing.

This study follows from the authors’ aspiration for practice and preliminary research
into the transnational piloting for smooth internationalization of health-tech startups.
The results and key findings are reported in this stage.

The paper has four sections. The first describes the theoretical framework consisting
of internationalization of startup companies, lean startups, and living labs. The
second section introduces a case study of transnational piloting. The third section
presents the preliminary results and findings. The fourth and final section discusses
the contributions of the paper, notes the study’s limitations, and presents the roadmap
for the future development needs of the transnational health and wellbeing living labs.

2 Theoretical framework

Digital and technological solutions for health-tech business is increasing all over the
global markets. Startups need lean and cost-efficient methods to find out, create and
test market needs and opportunities in foreign markets. Lean startup approach
(Blank, 2007, 2013; Ries, 2011; Maurya, 2012) has been identified as a structured
and efficient process, which may help startups to achieve their strategic and business
goals in internationalization (Neubert, 2017). Many startups aim at internationalizing
early and fast to become profitable and therefore they seek for markets where it is
easy and fast to enter. Based on Johanson and Vahlne (2009), the Uppsala model
can be applied to firms that begin to internationalize fast after their founding, if they
seek low-risk and low-cost market-entry modes such as exporting. However, in many
cases, this is not enough or it is not even an option, and the business model has to
be created within the cultural premises and local market conditions. On the other
hand, internationalizing early and fast is very challenging for startups and
entrepreneurs because it requires specific competences, networks, special

preparation, high experience, and willingness and readiness to enter international
markets (Neubert, 2016, 2017).

Rasmussen and Tanev (2015) linked the two research streams: lean startups and
born-global firms, and introduced the new concept of lean global startup, which is
typically a high-tech startup aiming at creating a new product with innovative solution
for a specific market niche. In lean global startups, the internationalization strategy is
part of the initial business plan (Neubert 2017). The lean startup methodology may
also be used in internationalization by applying incremental and iterative product
development cycles to develop minimum viable products (MVPs) and test them with
quick feedback in the market (Tanev, 2017; Coviello & Tanev, 2017; Neubert 2017;
Blank 2013). According to Neubert (2017) and Johanson and Vahlne (2009) the
speed of learning in small, iterative steps defines the speed of early

In the literature, social networks and networking ability as well as ability to learn have
been recognized as the main drivers of fast internationalization (Coviello, 2015;
Neubert, 2017). Based on Ciravegna, Lopez and Kundu (2014) the social networks
of an entrepreneur as a driver of the speed of internationalization is essential. For the
lean startups, the networking in the internationalization context is especially the ability
to create market opportunities to acquire new clients and distribution partners with
local networks.

In their article, Ojasalo and Ojasalo (2018) formulate how lean service innovation
approach and process can focus and solve the needs of early identification of core
customer value with business potential, especially for new or potential customers
utilizing latent needs in service innovation. Their article build on the idea of lean
innovation (Blank, 2007, 2013; Ries, 2011), and they borrow this idea for the service
innovation process to fulfil the knowledge gap in service innovation research. In
addition to identifying the knowledge gap, the article presents a managerial
framework for applying service-dominant (S-D) logic in practice. Ojasalo and Ojasalo
(2018) argue that the existing lean development models focus on an early
understanding of customer needs and value, and thus the lean approach has a lot to
offer for the research of S-D logic. They also underline that much more than knowing
what presents customer value is required to turn into a profitable business. That is a
scalable and profitable business model. In their framework, lean service innovation
approach is used throughout the service process, and case-specific development
methods are applied for solution development, testing, and experimentation.
Living labs are intermediaries for innovations. They can be characterized in multiple
ways and they serve several purposes. In a Living Lab Methodology Handbook Anna
Ståhlbröst (2017) define: “A Living Lab is an orchestrator of open innovation
processes focusing on co-creation of

innovations in real-world contexts by involving multiple stakeholders with the
objective to generate sustainable value for all stakeholders focusing in particular on
the end-users”.

Based on Westerlund and Leminen (2011) “living labs are physical regions or virtual
realities, interaction spaces, in which stakeholders from public-private-people
partnership of companies, public agencies, universities, institutes, users, and others
that follow the philosophies of open and user innovation to collaborate for improving,
developing, creating, prototyping, validating, and testing of current or new
technologies, services, products, and systems in real-life contexts”. Regardless of the
multiple different definitions and implementations, living labs share certain common
elements that are essential to the approach: 1) multi-method approaches, 2) user
engagement, 3) multi-stakeholder participation, 4) real-life setting and 5) co-creation
(Malmberg & Vaittinen, 2017). In addition, the living labs strives for mutually valued
outcomes that are results of all stakeholders. Schuurman (2017) describes that three
main elements have been distinguished within living lab projects, following the
innovation development stages: 1) exploration; getting to know the current state and
designing possible future states, 2) experimentation; real-life testing of one or more
proposed future states and 3) evaluation; assessing the impact of the experiment with
regards to the current state in order to iterate the future state.

In Living Lab Methodology Handbook (2017) the role of living labs is described as
mediators to build and strengthen the European Open Innovation ecosystem that
enables the internationalization of SMEs supporting validation of products and
services in other markets throughout living labs’ co-operation and consultancy.
However, this role is new and emerging, and the living labs needs to advance their
international collaboration and commercial offering for transnational validation of
services for SMEs and startups.

Living labs has been used specifically by startups and SMEs, and they offer a
structured approach to open innovation (Schuurman, 2015) in user innovation
paradigm (von Hippel, 2009). Schuurman et al. (2016) have explored the value of a
living lab approach for open innovation in small and medium-sized enterprises with
comparative case study in 27 SME projects, and they argue based on the results that
a real-life intervention and a multi-method approach increase the chance of
generating actionable user contributions for the innovation under development.
In this study, a transnational living lab has been defined as a living lab, which serves
companies or other institutions in an international context, i.e. at least in two
countries. In literature, the terms cross-border or transregional living lab have been
used in similar contexts. The mediator living lab, i.e. Laurea, is seeking transnational
health and wellbeing living labs as testing partners to accomplish the needs of local
startups in their internationalization processes. Transnational testing partners are

requested to act in commercial basis performing expertise in their functions, and they
should follow the lean start-up approach in their services.

The overarching aim of this case study is to create better opportunities and efficient
methods and models for health-tech startups to internationalize by following the lean
startup principles in transnational living labs. Finally, the results are described as
preliminary findings and a model for transnational health and wellbeing living labs.

3 Transnational piloting

The living lab practices and experiences presented in this study are based on two
cases. They are a part of the publicly funded project, in which one aim is to create
and pilot a model for the transnational health and wellbeing living lab. In order to
achieve this aim three pilots were planned to be carried out. The piloting focuses on
the service ideas/solutions that are trying to solve one or several of the following
challenges: 1) how might we increase physical activity in everyday life, 2) how might
we enable people to take their health and well-being in their own hands and to
manage them and 3) how might we support active independent living of the elderly.
The challenges were defined based on the needs in the market. First three pilots
were selected but finally two of them continued for actual piloting phase.

Next, the case companies are presented. Company A creates differentiating products
by connecting everyday goods with mobile applications and accessories. They create
tangible digital experiences in everyday products using wireless connectivity, sensors
and cloud services with hardware, smartphone middleware and services platform.
They also offer consulting services. The company is currently developing its own
product portfolio of wellness and rehabilitation products. The company wants to pilot
their product in order to test its suitability to the target market. They are looking for
partners with whom they can develop technologies, services and products focusing
on rehabilitation and wellbeing and gather feedback from local users. They are
interested to work with cities and public sector. In addition, a living lab as a part of
university might be a potential partner for them. They are also interested in to join a
bigger consortium. By doing this testing, they want to develop their service design
further and internationalize their service. For the piloting, there was a need to recruit
end users who represented three different customer segments plus a professional to
support the testing with end users.

Company B has two product lines of which product x is the newest. Several
healthcare providers and companies in Finland use its predecessor, product y. This
has allowed the company to collect data from hundreds of thousands of users over
the years, which led to the development of the product x. While the product y is

designed for occupational health care professionals, product x is created for its end-
users and employers. It is not just a mobile version of its predecessor but rather a
scalable, more intelligent, user-friendly tool with several new features and more
content. The company is heading to international markets and they are willing to test
the service in a company/ies operating in their target market. They want to learn if
there is a need to make significant changes to the product/service and how it works
outside of Finland. The objective is to get a test data of a couple of hundred users in
order to fine-tune the algorithm used in the application for the target market. In this
case there was first a need to recruit a piloting company/ies who was/were willing to
pilot the product with their employees. Thus, the focus is in business-to-business

Before explaining the piloting processes of the case companies further, some
background information is given in order to present the whole development process
of transnational health and wellbeing living lab model. The first draft of the
transnational living lab model was planned based on the workshop done in the Open
Living Lab Days 2017. The main organizers of the workshop were Laurea Living Labs
and Licalab. The goal of the workshop was to move forward in setting up a
Transnational Living and Care Lab as a unique innovation instrument to support
SMEs in developing and scaling up innovations for 'living and care' and 'active and
healthy ageing'. The transnational living lab model was discussed and considered
through the lenses of the customer (e.g. SME). The workshop provided insights how
to promote the transnational living lab, and revealed that there are development
needs for the coordination of the international projects. It was identified that there is
a need to improve the awareness of the healthcare living labs and their services. In
addition, the contact information should be easily available, preferable one contact
point. Further, cultural differences were discussed.

The process of piloting included several phases: call for ideas, selection of the pilots
(inc. selection criteria), invitation to tender, selection of the living lab, planning,
testing, results and next steps. Before explaining the phases in more detail, the
stakeholders and their roles are presented. The project coordinator, Laurea
University of Applied Sciences (Laurea Living Labs), works as a mediator in the
piloting. Laurea is an official contracting body and facilitates the process between the
company and the living lab. The company gives the brief and practical instructions
and support for the piloting. In this pilot support is for example providing the products
for testing and organizing the training for the living lab staff and the users in the target
market. The living lab is responsible of the implementation of the pilot, analysis and
reporting the results.

Call for ideas was a necessary step in this publicly funded project in which all the
organizations who fulfilled the criteria should have a similar possibility to leave their

application and participate. The content of the call for ideas was: description of the
challenges, the focus and the limitations, responsibilities, selection criteria, timetable,
monetary support and contact information. The project partners who had a direct
access to their startup networks were valuable messengers during this phase. In total
nine applications were received before the deadline.

Selection of the pilots were done based on the criteria, which was planned before the
call for ideas were published. First, the best applicants were selected based on the
data they had provided, then the meetings were organized to discuss the piloting plan
and confirm the common interest and understanding. A lot of practical information
and new insights were received in these meetings. The meetings had also an
important role to get acquainted with each other’s and to build trust.

Invitations to tender were planned based on the applications for call for ideas and the
meetings during the selection process. Thereafter it was sent to the case companies
for comments. Some iterations were done, and legal aspects were checked before
sending invitations to tender forward. Simultaneously with this phase, potential
service providers (living labs) were searched. That happened by the help of the
European Network of Living Labs (ENoLL) and its Special Interest Group in
healthcare, project partners from different ongoing or past projects and colleagues.
The name and country of origin of the living labs were quite easy to get but what was
more difficult was to find out their contact information and descriptions of them, their
expertise and services. In some cases, there were web pages describing the living
lab but not always in the language you could understand.

Selection of the living lab was done based on their offers. The selection criteria were
explained in the invitation to tender. Before signing, the agreement there was a
negotiation together with the service provider, Laurea and the startup company. For
company A, the service provider is an experienced living lab having expertise in the
business sector in question. For company B, the service provider is a private
company offering partly similar services than living labs. The first drafts of the
agreements were done by Laurea and they were sent to the service providers and
startups for comments. After finalizing the agreements, they were signed by Laurea
and the service providers. In the case of the company B a separate confidentiality
agreement was signed with the sub-contractor of the service provider. In this phase
the support from the legal services was needed and it was easily available as an in-
house service at Laurea. The time for planning the agreements was not considered
when making the initial schedules for testing.

The initial plan for testing was introduced in the invitation to tender. This plan covered
following aspects: description of the case company and its product/service to be
tested (what, to whom, how), implementation of testing (when, objectives, guidelines),

outcomes and the role of the testing partner (living lab). The plan was finalized
together with Laurea, the startup and the testing partner. The planning was done in
the skype meetings. Some modifications were done later in the process. The
timeframe for the implementation of testing was first between January and May 2018.
Because of the delays in the invitation to tender process, time used for the
agreements and delays in the recruiting phase in the case of the company B the
deadline was postponed until September 2018.

Actual testing phase has now ended with the company A. The testing was
implemented quite independently by the testing partner: there were some
communication between the company A and the testing partner regarding to the
information needs and functionalities of their product and application. Further, Laurea
and the startup visited the testing partner in this phase to discuss the process and
see the testing in action.

The results were presented in the skype meeting. The final report was shared in
advance to be able to discuss in more detail in the meeting. The expected outcomes
in the report were analysis of the usability and the end user experience, needs for
local adaptation and business model (potential for licensing, service as a part of the
package, value proposition, marketing channels). These aspects were discussed and
further information was asked, especially by the company A. For the company it is
crucial to understand the context of testing and how it has been implemented in order
to interpret the data correctly and to be able to apply the new knowledge in the similar
circumstances elsewhere.

Testing phase with the company B proceeded with one piloting company to their
human resource department. The research on that is still in progress and the results
are not yet in use. Thus, the results and findings are based on the experiences so
far. They are presented in the next chapter.

4 Preliminary results and findings

The preliminary results of the ongoing case studies provide guidelines to create a
practical collaboration model for transnational living labs to support startups’/SMEs’
internationalization. These results and findings are presented in the table 1 and
discussed in more detail thereafter.

Table 1: The results and findings of the case studies

Results: Results:
Tasks of the Living Tasks of the Living
Tasks of the Findings
Lab, mediator Lab, testing
(LL1) partner (LL2)

LL services have to
Need to be visible in the
Call for ideas for
internationalize their regional/national/int
international piloting
business ernational startup

Contact point for

Searches for Permanent contact
startups to discuss if
support to test their point for the
their need match
product/service in inquiries of LL
with the service
their target market services is needed

A ready made
Describes a Modifies the brief of
template helps when
preliminary brief startup/SME
preparing the brief

Information needs to
be available in
More information of
Searches for the focus areas
potential living labs Services are (content,
for collaboration visible/known in the geographical area),
from the target living lab network services and
market expertise of LLs is
often needed.
Need to get an
access to B2B

Consultation with
Accepts the the legal services.
invitation to tender Plans the invitation A ready-made
(matches with the to tender template helps when
brief) preparing a call for

Receives the
Gives further Sends the invitation
invitation to tender Timing is critical
information if to tender and replies
and asks further (holiday seasons)
needed to the inquiries
questions if needed

Enough time for
leaving a tender

A detailed
Prepare an offer information of the
Evaluates the offers Evaluates the offers (implementation LL2 is essential in
together with the and asks further plan for testing, the offer because
LL1 questions if needed expertise, costs, otherwise the
contact details) information is not

Negotiates with the Negotiates with the Negotiates with the Readiness for online
LLs LL2 LL1 meetings

A separate
agreement between
the startup and the
Living Lab/s or their
Signs agreement Signs agreement might be needed.
Notify the time for
planning of an
Legal services might
be needed.

Depending on the
testing partner, a
mediator can have a
smaller/bigger role
in facilitation.
Facilitates the
Depending on the
process, Implements the
Support the process brief a startup/SME
quality/cost/schedul testing as agreed
can have a bigger
e control
role in the recruiting
Longer time period
to recruit B2B

Accepts the results

Presents the results,
and pays the
sends the invoice.
Gets the results and invoice. Quality check of the
Discuss the
plans the next steps Discuss the results
possibilities for the
possibilities for the
future collaboration.
future collaboration.

The findings are focusing on roles and responsibilities of the different actors and the
whole process of piloting starting from the identification of the need to the results and
planning of the next steps.

Several roles were identified in the project: living lab as a mediator, living lab as a
testing partner, a piloting company (a company who is willing to test the
product/service), end users, and a funding organization. The roles and
responsibilities may vary depending on the expertise and available resources in the
collaborating living labs (mediator, testing partner). Another thing, which defines the
roles and responsibilities, is a possible project funding which was the situation in this
particular case presented in this paper.

Process of piloting includes several steps: searching for companies, negotiations

between a mediator and a company interested to pilot their product/service,
preliminary plan for testing, invitation to tender, searching for living labs, sending the
invitations to tender, analysing offers, negotiations, selecting a living lab for piloting,
making a contract, modifying the plan for testing, training the living lab to use the
product/service, implementation of the pilot, observing and following the project of
piloting, results, and planning the next steps.

Contracts were done between a mediator and a living lab/s and the legal services of
the mediator organisation were used. When having this service in-house it is easier
and cheaper to use. There might also be a need to make e.g. confidentiality
agreements between the startup/SME and the sub-contractor of the living lab (testing
partner). The needed agreements with the end users and other test participants were
in the responsibility of a testing partner. The process of making and accepting the
contracts may take time, especially if there is not a ready-made contract templates
that can be modified. There are many other moments, which take time in the process
as well, and not all of them have been thought about thoroughly beforehand. Thus,
there is a risk for delays in the process.

Information of the living lab services is not easily available whether it is on regional,
national or international level. Even if the potential for the creation of new international
growth companies has been created through the startup associations,
entrepreneurship societies, business accelerators and incubators they are not aware
of the living lab services and how to use them. If living lab services are not found in
the certain country or region other service provides will be searched. That might be
an only option also in the project where service has to be tested in business

Transnational health and wellbeing living lab model could benefit different actors.

Startups would get an access to the international living lab network, get support from
the local living lab, and utilize easy and agile way to implement piloting in their target
market. Living lab as a mediator or as a testing partner could get new clients and
collaboration possibilities with other living labs where to learn and broaden their

Building blocks for the preliminary transnational health and wellbeing living lab model
consists of the structure of the ecosystem, different roles, process and methodology.
In this case, the living labs had a regional and industry specific (healthcare) focus.
Furthermore, the living lab who worked as a testing partner had a cross border and
international scope in their existing ecosystem. Long-term relationships within the
network offers a platform for testing and validating services in agile manner. The
collaboration between the industry specific, regional and international ecosystems
provide not only the expertise in the substance area (industry) but also the expertise
to analyse the cultural aspects and overcome the language barriers. The role of living
labs are a mediator/facilitator and a testing partner. Besides testing and validating the
product/service, there is a need to act as a business developer. The process of
working can be divided to a pre-testing phase, a testing phase and a post-testing
phase. The methodology follows service design and lean startup approach even if
not always identified as such. Interesting is that the aim of promoting regional
development and industry is strong. Compared to that the business orientation of the
living lab is not so much in focus. Together with business orientation, the
commercialization of the services is still in its infancy. In continuously changing
business environment and rules and legislation, the legal expertise may not be
forgotten of the skillsets of the living lab.

5 Conclusion

The main contribution of this paper is the preliminary description of a model for
collaboration of transnational living labs and startups in the validation, testing and
local adaptation of service innovations of startups. In previous studies of transnational
living labs (e.g. Bódi et al., 2015), description of experiences of living lab methodology
and maturity of innovation and highlights and lessons learned from the
internationalization aspect of transnational cases of startups have been elaborated,
but any assumptions or a model have not been presented.

Based on the learnings and experiences in the workshop a year ago and two case
studies during the past six months there is a clear need to conceptualize the living
lab services and make them visible both regionally, nationally and internationally. This
development is a necessary step towards commercialization and sustainable

business with startups or any other paying customer. The collaboration between the
living labs increases the potential to offer the services to startups in several countries.
Startups could contact transnational living labs by themselves when the contact
information is easily available and they have resources to do that, or through the local
living lab if that is more convenient solution for them.

Transnational context brings new expectations to the living lab that is used to operate
locally. They have to be ready to work in a different language and there is a need of
understanding the cultural aspects in the business relationships as well as when
analysing the test results. Product or service to be tested might have texts that have
to be translated to the local language, thus translation services might be essential
add on in the service package. Furthermore, they have to have enough knowledge
about the legal issues in international context. Transnationality in services brings also
new roles and responsibilities. A living lab can have a role of a mediator when it is
connecting a startup and a living lab abroad. Besides acting as a mediator, it can be
a facilitator and support the process in various levels depending on the resources of
other actors.

In this paper, we have presented through a case study how transnational living labs
can be applied for validation and testing of heath-tech products and services in
international markets, even in commercial bases. However, the role is new and
emerging for living labs, and only few cases (e.g. Bódi et al., 2015) have been realized
in transnational settings so far. Initially, a focus of living labs has been in user
innovation and open innovation, and the validation of business models, business
potential, marketing channels, and needs for local adaptation have not been in the
scope. In lean startup approach, service development occurs cyclically in incremental
steps at the same time when testing and validating a product, thus, transnational
living labs are expected to offer this service as well (compared to the agencies).
Based on this case study, the transnational living labs can serve the startups in their
internationalization process in the means of mediators of lean startup or even lean
global startup approach.

The study is in its’ initial phase, and the conclusions cannot be completed, yet.
Knowing the limitations and benefits of a case study, in the next phase of research a
model for collaboration of transnational living labs and start-ups is demonstrated.


Blanck, S.2007. The Four Steps to the Epiphany – Successful strategies for
Products That Win. Raleigh, NC:Lulu Enterprises.
Blank, S. 2013. Why the Lean Start-Up Changes Everything. Harvard Business
Review, 91(5), 63–72.
Bódi, Z, Garatea, J., Robles, A. & Schuurman, D. 2015. Living Lab Services for
Business Support & Internationalisation. ENoLL.
Ciravegna, L., Lopez, L. & Kundu, S. 2014. Country of Origin and Network Effects
on Internationalization: A Comparative Study of SMEs from an Emerging and
Developed Economy. Journal of Business Research, 67(5): 916–923.
Coviello, N. 2015. Re-Thinking Research on Born Globals. Journal of International
Business Studies, 46(1): 17–26.
Coviello, N. & Tanev, S. 2017. Initiating a New Research Phase in the Field of
International Entrepreneurship: An Interview with Professor Nicole Coviello.
Technology Innovation Management Review, 7(5), 52–56.
Johanson, J. & Vahlne, J. 2009. The Uppsala Internationalization Process Model
Revisited: From Liability of Foreignness to Liability of Outsidership. Journal
of International Business Studies, 40(9), 1411–1431.
Malmberg, K. & Vaittinen, I. 2017. (Eds.) Living Lab Methodology Handbook.
Maurya, A. 2012. Running Lean: Iterate from Plan A to a Plan That Works.
Sebastopol, CA: O’Reilly Media.
Neubert, M. 2016. Significance of the speed of internationalisation for born global
firms – a multiple case study approach. International Journal of Teaching and
Case Studies, 7(1), 66–81.
Neubert, M. 2017. Lean Internationalization: How to Globalize Early and Fast in a
Small Economy. Technology Innovation Management Review, 7(5), 16–22.
Ojasalo, J. & Ojasalo, K. 2018. Lean Service Innovation. Service Science, 10(1),
Rasmussen, E. & Tanev, S. 2015. The Emergence of the Lean Global Startup as a
New Type of Firm. Technology Innovation Management Review, 5(11), 12–
Ries, E. 2011. The Lean Startup: How Today’s Entrepreneurs Use Continuous
Innovation to Create Radically Successful Business. New York: Crown
Schuurman, D. 2015. Bridging the Gap between Open and User Innovation?
Exploring the Value of Living Labs as a means to Structure User Contribution
and Manage Distributed Innovation. Doctoral dissertation, Ghent University,
Schuurman, D. 2017. Living Lab methodologies in Malmberg, K. & Vaittinen, I.
(Eds.) Living Lab Methodology Handbook.
Schuurman, D., De Marez, L. & Ballon, P. 2016. The Impact of Living Lab

Methodology on Open Innovation Contributions and Outcomes. Technology
Innovation Management Review, 6(1), 7–16.
Tanev, S. 2017. Is There a Lean Future for Global Startups? Technology Innovation
Management Review, 7(5), 6–15.
von Hippel, E. 2009. Democratizing Innovation: The Evolving Phenomenon of User
Innovation. International Journal of Innovation Science, 1(1), 29–40.
Westerlund, M. & Leminen, S. 2011. Managing the Challenges of Becoming an
Innovation Company: Experiences from Living Labs. Technology Innovation
Review, 1(1), 19–25.

User needs and expectations as a challenging factor
for successful living lab research initiatives
involving older adults: The DDRI Experience
Tiziana C. Callari*1, Louise Moody1, Nikki Holliday2,
Ed Russell3, Janet Saunders1, Gill Ward4, Julie Woodley5

*Corresponding author
1School of Art and Design, Coventry University
2 Centre for Innovative Research Across the Life Course, Coventry University
3 WCS Care Group
4 Royal College of Occupational Therapists
5 Faculty of Health and Applied Science, University of West England

Category: Research Paper

Despite the number of Living Labs now in existence worldwide and in Europe, there
is still a limited amount of academic study and literature to guide their set-up and
running. We were particularly interested in ethical guidelines and frameworks to guide
research engagement and participation as Coventry University along with partners
sought to develop and establish a new LL. This paper therefore intends to shed light
on the ‘user engagement’ aspect, in particular when older adults, and adults with
cognitive and physical impairments are involved. To do so, we recruited n=6 residents
and n=2 family members from two residential living environments partners of the
DDRI programme, a LL pilot initiative led by Coventry University. Methodologically,
semi-structured interviews were conducted and the Qualitative Content Analysis
(QCA) method used to make sense of the data. A deductive coding frame was built
informed by the study goals, and its categories included (among others) ‘user
engagement’, ‘user commitment, ‘user expectation’, ‘user needs’, ‘nature of
participation’. The results report on the participant views and concerns about these
LL research topics, and provide (ethical) insights for future research collaborations.

Keywords: User needs, User expectations, Nature of participation, Ethical issues,

Living lab, Older adults, Family

1 Introduction

Living Labs (LLs) are defined as “user-centred, open innovation ecosystems based on
systematic user co-creation approach, integrating research and innovation processes
in real life communities and settings” (ENoLL, 2006). Involvement of users is seen as
a “participative force for co-creating value”, and facilitating the creation of an
innovation ecosystem based on a public-private-people partnership in the form of
collaborative processes and knowledge sharing (Pallot, Trousse, Senach, & Scapin,
2010, p. 2). This innovation process becomes user-centred (with user as subject) and
participatory (with users as partners) increasing user-acceptance of new products,
services or processes, and hence reducing the failure rate at the market (Dell'Era &
Landoni, 2014; Habibipour, Padyab, Bergvall-Kåreborn, & Ståhlbröst, 2017). Multi-
stakeholder collaboration and knowledge sharing is a critical factor to successful LLs
(Buitendag, van der Walt, Malebane, & de Jager, 2012; Wenger, McDermott, &
Snyder, 2002). For example, Buitendag et al (2012) argue that knowledge creation
and knowledge sharing within and outside rural LLs motivate and encourage
communication and enhance long-term collaboration between members of different
communities of practice. This also supports the idea that multiple perspectives can
bring value to each of these partners in an integrative way, and contribute to the LL
innovation process and outcome. Ultimately, this becomes a requirement for the
success of LLs (Habibipour, Padyab, et al., 2017; Pino et al., 2014; Ståhlbröst, 2012).
LLs involve a multi-method approach in which the co-design/co-creation criterion can
depend on the degree of involvement and the actual utilization of user feedback and
suggestions for the (re-)shaping of the innovation (Schuurman, De Marez, & Ballon,
2015; Schuurman, Mahr, De Marez, & Ballon, 2013); the process of user involvement
and participation will also impact the ethical and operational concerns of the LL.
However, there is limited academic literature and exploration of how LLs should be
developed through stakeholder involvement, to ensure that stakeholder involvement
is maintained and optimised. Central to participant and stakeholder involvement is
consideration of the key ethical issues as well as how communication is managed
through the lifetime of the LL and individual projects within it.

Researchers agree that there is still scarce literature addressing specifically the ethical
guidelines involved in the design, development, and implementation of R&D projects
(Pino et al., 2014; Sainz, 2012). This is certainly due in part to the nature and
characteristics of LLs and the heterogeneity (e.g. in the way the LL concept and
approach has been understood and applied) of LL projects shared within the scientific
community (Burbridge, 2017; Müller & Sixsmith, 2008; Novitzky et al., 2015;
Schuurman et al., 2015; Yazdizadeh & Tavasoli, 2016). The involvement in LLs of
older adults, and adults with reducing cognitive and physical capacity poses additional
ethical challenges, which include fluctuating capacity or loss of capacity to provide
informed consent, participation opt-in and opt-out, involvement of third-parties as

decision makers e.g. children and carers of participants (Novitzky et al., 2015; Panek,
Rauhala, & Zagler, 2007; Pino et al., 2014; Sanchez, Taylor, & Bing-Jonsson, 2017).
Given this context, this study has been undertaken within the framework of the Data
Driven Research and Innovation (DDRI) Programme, a LL initiative which has involved
Coventry University and sector partners, and two residential living environments. The
first residential environment (later on referred to as ‘Setting A’) offers day care, long-
term residential and short-term respite care for older people and people living with
dementia or a range of different needs. The second (later on referred to as ‘Setting
B’) offers independent living for adults over 55 years, extra care is available for those
who need it. The overarching objective of the DDRI Programme is to use data driven
analytics and insights to support: independent living; healthy ageing; improved quality
of life, well-being and care (self-care or care from others); improved management of
particular health conditions; and enabling people to stay in their own homes for longer.
A number of DDRI/LL projects have been developed and launched within the two living
environments over the last 18 months (see Table 1). One of those projects specifically
focused on co-creating with stakeholders’ mechanisms for engaging older adults in
individual research studies, with a particular focus on how they are approached to take
part, how they are engaged and how they are thanked. The project involved a set of
interview studies, followed by co-design workshops to address some of the issues

Table 1. List of some of the DDRI Projects that the participants were briefed on

DDRI Project Title Objective Residential

Applied sleep To explore ways to improve sleep Setting A
interventions for and innovative ways of responding to
elderly residents in a night-time waking
care home setting
Development of To explore the feasibility and design Setting A & B
wearable technology to of body worn devices to detect
detect dehydration in hydration levels and feed-back
older adults information to care staff
Innovation for To explore the potential for digital Setting A & B
Dementia Care: innovations to improve health and
Evaluation of Digital wellbeing for a frail elderly population,
Health and Wellbeing including people living with dementia
Apps in ‘Real-Life’
Living Labs

The effects of an To trial the use of an acoustic Setting A
acoustic monitoring monitoring system and assess its
system on night time effectiveness in preventing night-time
falls by care home falls within a care home for people
residents with with dementia
dementia and care
staff response

With this study, we aimed to collect the user needs and expectations in relation to
participation and engagement in the above LL projects. Participants in this context
were the older adults living in the two residential living environments, and their family
members. As participation was voluntary, we also investigated the extent to which
rewards/incentives could be a possible additional lever for recruitment in research

2 Literature review

In line with this study objective, the literature review here has focussed on the following
topics only: i.e. (1) user engagement, motivations and expectations, (2) payment and

User engagement, motivations and expectations are critical inputs to create and
sustain a fertile LL context (van Geenhuizen, 2018). User engagement can be driven
by a number of activities, which may include promoting clear communication about the
project objectives and outcomes, shaping research goals based on user needs and
concerns, collaborating with users to help define future research direction, policy or
implementation of research outcomes. LLs could experience different degrees of user
engagement, running from users as leading co-creators at one extreme to users as
passive subjects at the other extreme (i.e. only involved to test/evaluate LL products
and/or services) (Almirall, Lee, & Wareham, 2012; van Geenhuizen, 2014). However,
engaging users throughout the lifetime of a project is not an easy task, as attention,
motivations and expectations tend to decrease over time, and users tend to drop-out
from research activities, regardless of the specific phase on the project development
process, before they are completed (Habibipour & Bergvall-Kåreborn, 2016;
Habibipour, Georges, Schuurman, & Bergvall-Kåreborn, 2017). In particular, user
motivation in LL research activities tends to be higher at the beginning, and decreases
towards the end of the activities (Habibipour, Padyab, et al., 2017). This trend may be
caused by the user’s internal factors (intrinsic motivation) or external factors (extrinsic
motivation) (Habibipour & Bergvall-Kåreborn, 2016). Researchers now agree that

keeping users motivated and having sustainable user engagement during the whole
process of open innovation is of crucial importance. In fact, user drop-out has been
acknowledged as having significant (negative) impact on project time and cost
efficiency, quality assurance, overall loss of trust and motivation with project
participants and stakeholders (Habibipour, Georges, et al., 2017). In their taxonomy
for drop-out in LL field tests, Habibipour and Bergvall-Kåreborn (2016) identify three
main categories: (1) Innovation-related category, (2) Research-related category and
(3) Participant-related category. Within the first category, beside the technological
problems (e.g. stability and maturity of prototype), two subcategories involve
understanding user needs and participant motivation. The first regards the ‘perceived
usefulness’ of an innovation by the target user; the second ‘perceived ease of use’, in
which complexity in user-interaction, and or readiness of a prototype might affect user
motivation. Within the second high-level category (i.e. ‘Research-related’), which
includes ‘Task design’, ‘Interaction’ and ‘Timing’, in the ‘Interaction’ subcategory the
attention is focused on the level of communication established in the LL between
researchers and users and the expectation set. Finally, within ‘Participant-related’
drop-out are listed the personal issues related to the participant – i.e. the attitude, the
context and resources.

LLs, especially with the involvement of older adults, face a range of ethical issues
which need to be adequately addressed. The approach to research in this context
(especially where academic partners or health or social care partners are involved)
will need to undergo a process of formal ethical review and approval. There is clear
guidance of research involving older adults and working in high-risk environment, and
from specific professional bodies (Bollig, Schmidt, Rosland, & Heller, 2015; BPS,
2009; NSW, 2015; Walsh, 2009). We would argue though that the issues brought
together in LLs are complex and multidisciplinary, and navigating the route to clear
ethical principles and practice is an ongoing challenge. One of the challenges for
example faced in LLs is how to handle an appropriate rewarding and incentive system
for users involved in LLs. We are aware of the ethical debate around the use of
rewards and incentives to research participation. Here we report what authors in the
field have commented in relation to ‘rewarding’ LL participants for their involvement in
LL research activities. For example, Dutilleul et al. (2010) argue that as users in LLs
actively involved in co-design and co-creation activities to produce value, there should
be rewards in place that secures pay-back to all the actors involved. Incentives may
be of different kinds (e.g. vouchers, free tickets, etc.), and can be assigned either at
the beginning, after recruitment, or during the trial activities (Georges, Schuurman, &
Vervoort, 2016). Others suggest that an inappropriate incentive mechanism, together
with issues in building a positive relationship with and mutually between the LL
stakeholders, may be among the reasons for participant drop-out (Habibipour,
Georges, et al., 2017). Buitendag et al. include this aspect in their definition of LLs,
as “the active participation by the community of practice (CoP) and other stakeholders

in some or all living lab activities, which may also include in sharing the reward”
(Buitendag et al., 2012, p. 221). They argue that ‘rewards’ do not only refer to tangible
or financial incentives, but can encompass the underpinning concept of open society
for LLs (Buitendag et al., 2012). Along this line, it is argued that LLs support open
innovation built “on voluntary collaboration” in which each participant has a similar role
and relevance (Nyström, Leminen, Westerlund, & Kortelainen, 2014, p. 484;
Ståhlbröst & Bergvall-Kåreborn, 2013). Along with Sainz (2012), we suggest that the
innovative LL approach would benefit from guidelines and regulations on the ethical
aspects inherent to its voluntary and participatory nature, this was the driver for the
study described here

3 Methods

The full study was carried out in 2017 over a 12-month period employing a qualitative
research approach (Blaikie, 2009; Ritchie, Lewis, Nicholls, & Ormston, 2014). The
study received ethical approval from Coventry University Research Ethics Committee.
A letter of support was provided by the residential organisations (i.e. Setting A and B).
The principles of the British Psychological Society Code of Ethics and Conduct and
UK Research Integrity Office’s Code of Practice for Research guided the research.


Recruitment of residents followed two strategies: in ‘Setting A’ participants were

recruited via the recommendation of the staff team, who facilitated the identification of
the interested residents; in ‘Setting B’, coffee morning events were organised, and
flyers distributed to advertise the research initiative with the proposed projects, and
collect resident interests. Overall, n=6 residents agreed to take part in this study, of
which n=5 were female (we only recruited residents who were able to provide their
informed consent). The average age was M=79.5, where the youngest was 56 years
old, and the oldest 90 years old. Two participants currently lived in the care facility (i.e.
Setting ‘A’), whilst the other four have been living in independent apartments in Setting
B for an average period of 17 months. Further, n=2 family members were recruited.
Their parents live in Setting A and at the time of this study were diagnosed with
cognitive impairments. For details see Table 2 below.

Table 2. Participants’ demographics

# Role Male / Setting # Time at residential

Female setting
RE1 Resident - brain damage Female A
age 56, hoping to regain 4 months
RE2 Resident, age 90, no Female A
12 months
RE3 Resident, age 88 Female B 16 months
RE4 Resident, age 87 Male B 10 months
RE5 Resident, age 84 Female B 18 months
RE6 Resident, age 72 Female B 24 months
FM1 Family member - mother Male A 12 months
has dementia, father was
also in Setting A
FM2 Family member - father Male A
5 months
has Parkinson’s


Semi-structured interviews were conducted with residents and family members. The
interview guideline included probing questions to collect opinions (both positive
feedback and concerns) about (1) the ongoing DDRI projects in the two living
environments (namely in Setting A and Setting B), and (2) ethical issues and concerns.
Participants were given short summaries of exemplar DDRI projects by the interviewer
of a and questions explored for example their views on each project, concerns about
getting involved in it if invited, concerns about giving consent, views on ‘thank you’s’
and the use of incentives for participation. Overall, the average interview length was
an hour; resident participants were allowed to take breaks if requested.
The interviews were audio-recorded and transcribed verbatim; all transcripts were
saved, coded and analysed using NVivo (v.11 Plus for Windows, ©QSR International)
(Bazeley, 2007). An NVivo project (entitled ‘DDRI-Driven Research and Innovation’)
was created, which contained all interview transcripts; the selected literature articles;
the memo journal keeping track of all activities and decision-making points agreed
throughout the development of the research.

Data Analysis

The Qualitative Content Analysis (QCA) method was used to make sense of the data
(Schreier, 2012). Critically, the coding frame built to interpret the raw material was
informed by the study objectives, and hence followed a concept-driven strategy
(Saldana, 2012; Schreier, 2012). In the NVivo project a tree-node structure was
created: the parent-nodes (i.e. high-level categories) reflected the relevant themes to
the research. A number of child-nodes (i.e. the subcategories) specified each parent
node. See below Figure 1 for an overview of the coding frame used.

Figure 1. Concept-driven coding frame used to categorise the study data (from the NVivo project)

All interviews were coded using the proposed coding frame. The segmentation
strategy used as coding unit was the ‘meaning unit’ (that is, any portion of text,
regardless of length, to which it was believed a code may apply) (Coffey & Atkinson,
1996; Grbich, 2013; Saldana, 2012).

4 Findings

Involving older adults, and adults with physical (e.g. sight or hearing impairments) or
mental (e.g. dementia) disorders poses additional challenges for the researcher.
Some of these challenges include the design of an inclusive consent form and
participant information; the involvement of family, or others to provide consent should
capacity to consent be diminished. Mechanisms for sharing information and gaining
consent were discussed. Information about the DDRI projects was circulated in the
two residential settings by means of Project Information Sheets (PIS). Colour images
and short paragraphs of text (using Arial 14 font) were used to describe the different

active projects, and assess the resident interest before asking them to sign the
informed consent. Overall, this approach was commented positively. Further,
participants highlighted the need to consider individual needs:

“Sight and hearing are important. We all look normal enough but
everyone’s got some kind of underlying problem. For the people who are
partly blind or blind, it could be read out to them. You would have here a
few with sight problems”.

The involvement of third parties in the process of consent to participate in a study was
also discussed. Involving the family, or a ‘carer network’ is well accepted by all
residents, as they are conscious of their deteriorating physical and mental conditions,
but recognised participant wishes and attitudes should still very much be considered.
This also may include the possibility to withdraw from the LL research initiative due to
health factors.

“Well I mean if you as the researcher thinks that the person is no longer
quite capable of doing it, then I think that’s reasonable [to involve the

“Family or carers then need to know what the person concerned attitude
was in relation to research”.

Challenges around the location of family members were raised and how this might
affect recruitment to studies if they were not easy to get hold of and consult:

“I don’t know my family is fairly well scattered around the country, there’s
none of them that are local. But they are already aware of different aspect
concerning my health, so this involves not only this project”.

All of the presented DDRI projects (see Table 1) were found to be of interest to the
participants’ and indeed the feedback was positive. Reactions like ‘This sounds
interesting’, ‘Well, I think that’s a brilliant idea’, ‘I’d very much welcome something like
that’ were collected. The project related to sleep interventions raised particular interest
among the residents in both settings, as it was commented that sleep is an issue at
their age. Hence, a number of questions in relation to potential side effects, dosage,
and food sensitivities were asked. The project about hydration level detection also
raised attention and the resident comments – particularly in relation to wearing the
proposed technology – were positive. Overall, the discussion about the LL projects

showed how the residents (in particular in Setting ‘B’) can discern their needs and
what could be good (or not) for their health.

“This is another one that’s right in my area of concern. I can’t get to sleep
at night without an intake of zopiclones, […]. I notice that in your project
you’ve got separate ideas, like the milky drink – but I’m sensitive to milk
and any dairy, cheese or anything of that sort so that rules that out for me
but I get the tryptophan from bananas and dates, dried dates. You know, I
got a lot of faith in a nutrition book that I’ve got down here”.

“I recognise the importance of hydration and my general health dictates

that I do have a good intake of, water on one hand but then again I have
prostrate problems so that is quite a problem for me getting it right…but a
research into it is wonderful”.

It was interesting to gain a sense of user motivation to take part in research. The older
adult residents, identified being motivated to be engaged in something challenging
that could help maintain their mental capabilities:

“I like to get involved with these sorts of things because I think it keeps my
brain working. To be honest with you, it’s just like if you just sat here in this
flat and did nothing. I couldn’t do that I have got to be doing something,
and I say it I don’t mean physically, I mean mentally!”.

The intrinsic motivation linked to specific recognition that participation in studies might
support their own health and satisfy their interests. Their perception of the value of the
study and participation encouraged influenced overall resident attitude in relation to
research participation. Residents commented that supporting research and the
‘general good’ are important aspects to them, either because they believe in what the
research is aiming to, or given their personal and educational background.

“Well if they were told by doing research that they were likely to get better,
have better sleep they would, should be taking part. And even if they didn’t
I mean it would help somebody somewhere”.

“See, when you came we didn’t really know what we were taking part in,
we didn’t really understand what it was but it’s really future research that
will help people in the future more than us probably”.

“But then it depends on the background of the person, you know they
wouldn’t normally, well they’ve just never heard of it. I think that’s for your
educational background and what sort of research”

Many factors can affect the residents’ attitudes towards research participation, both
positively and negatively. These may include culture (e.g. what it means to get
engaged in research), period of life (e.g. living in senior living settings), personal
beliefs, etc. For example, in more than one interview it was highlighted that the word
‘research’ seems to convey a negative connotation, and consequently led to a feeling
of distance from the issue and closure.

“I can only tell you the impression I get from talking with people here. I feel
that quite a lot of people, they are not really interested. They got to the
stage in life where they just really don’t want to be bothered”.

“I feel that as soon as you say research they’ll say “Oh no-- I’m not
interested, thank you. I think so because they think of researchers are
really going inside you”.

“I don’t tend to do surveys at all, particularly on the phone. They ring you
up and say we’re just doing a survey. I just say --no. I don’t do surveys on
the phone and put the phone down!”.

“Getting involved with things like these I wouldn’t do it, not knowing your
background, knowing where you came from sort of thing I mean I wouldn’t
do it to anybody who just came to the door and asked me to do it”.

Further, the interviews highlighted that those wanting to involve older adults in
research should also consider how older people can personally manage being part of
a LL research initiative and how researchers should take their needs into account in
study design and delivery, particularly with pace of all procedures involved, etc. One
resident commented about some possible unexpected reactions, pointing at the
potential vulnerability and variation in capabilities of older people.

“You see, there are people with slight irritable problems, get het up very
quickly, the sight of a piece of paper that they have to listen to and do
anything with it is beyond them. Apart from that, don’t put any pressure on
them. You know, it’s how much they can cope with and you don’t know
whether it’s because of their underlying illness or not, you just accept them
as they are and then just work around them. […] You know, refer back to
what you did and if somebody has a better feeling than they might do

The participants had clear views on the benefits to them of participating. They would
like acknowledgement and to know about the impact of the input they made or the
valuable future impact in society. This also included their attitudes toward research
participation. Although during the recruitment process it was suggested to consider

rewards as a motivation factor for participating, all participants were volunteers. When
prompted about the topic, both residents and family members confirmed that
participation should happen without the need of rewards.

“We should all do our bit and not expect a reward”.

“I think if you’re interested, you do it, just do it. I mean I can’t see why we
need to have a reward’.

“No. No rewards. That drives the wrong behaviour doesn’t it?

They do not necessarily expect a ‘personal’ thank you, but they expected to receive
information about the outcomes the project had achieved, any next step, etc. The
same view was shared by the family members

“You know having participated in that little bit of research, it obviously links
into something else and it could be nice if you can hear about it and think:
“well I feel quite proud of that because I helped”.

“It would be nice to get the outcome be it in the form of an e-mail, a

general e-mail to everyone and what the contribution was and how it’s

In line with the above, the interviewed family members were very aware of their
parent’s health condition, and what this entails in terms of communication and
interaction (e.g. the need to repeat a number of times the research questions - posed
in a Yes/No way - to get the answer), and how both physical and mental conditions
may deteriorate as the research proceeds.

“I don’t think this would be a concern, it would be good because I know

how Mum communicates […] by her pointing, so even without saying
anything that’s communicating. […] The only thing that I think that would
be difficult is if you were to spend an extended period of time with her to
get her to do one single thing because of her concentration levels, she’d
get tired very quickly. So, it should be a gently, gently approach really”.

“That’s part of the thing -- if this is no longer suitable for my Dad’s

condition or somebody else’s condition, we just need to step outside the
trial please”.

5 Conclusions

This study was part of a bigger research initiative carried out within the DDRI
Programme that involved a number of LL studies carried out in two residential living
environments with older adults, and adults with physical and cognitive impairments. A
qualitative research strategy was used, and interviews conducted with the research
participants to explore user needs and expectations in engaging in LL research
initiatives, and ethical issues connected to this.

Overall, LL research initiatives leveraging greater interest among older people address
health topics close to the current user needs (e.g. sleeping issues, body hydration,
etc.). This can have an impact on the residents’ intrinsic motivation towards research
participation. Other factors may drive the overall motivation, such as cultural aspects,
which resulted in several valuable feedback responses. One concern was related to
the connotation that the word ‘research’ may convey to older people, and possible
alternative ‘labels’ to define LL ‘research’ initiatives have been advanced, including
“Adult caring research” or “We want your views”. Further, as LL projects involve
medium-, long-term collaborations with research participants, it is argued that it is vital
to maintain the ongoing interest and cooperation, as well as expectations, of research
participants, family and other stakeholders for successful research initiatives.

More light has also been shed in relation to critical ethical concerns when LL initiatives
engage with older adults, and adults with physical and mental impairments. This
included the nature of participation in LL research initiatives (i.e. voluntarily.
rewarded), opting out from research, or involving family in the decision-making
process when the physical and mental condition of elderly persons may put their
participation at risk. The findings informed subsequent co-design sessions to develop
acceptable briefing information and research processes for gaining consent and
meaningful participation within the DDRI model that are reported elsewhere.


Almirall, E., Lee, M., & Wareham, J. (2012). Mapping Living Labs in the Landscape
of Innovation Methodologies. Technology Innovation Management Review,
Bazeley, P. (2007). Qualitative Data Analysis with NVivo. London: SAGE
Blaikie, N. (2009). Designing Social Research. Malden, MA: Wiley.
Bollig, G., Schmidt, G., Rosland, J. H., & Heller, A. (2015). Ethical challenges in
nursing homes – staff's opinions and experiences with systematic ethics
meetings with participation of residents’ relatives. Scandinavian Journal of
Caring Sciences, 29(4), 810-823. doi:10.1111/scs.12213
BPS. (2009). The British Psychological Society Code of Ethics and Conduct.
Retrieved from
Buitendag, A. A. K., van der Walt, J. S., Malebane, T., & de Jager, L. (2012).
Addressing Knowledge Support Services as Part of a Living Lab Environment.
Issues in Informing Science and Information Technology, 9, 221-241.
Burbridge, M. (2017). If Living Labs are the Answer – What's the Question? A
Review of the Literature. Procedia Engineering, 180, 1725-1732.
Coffey, A., & Atkinson, P. (1996). Making sense of qualitative data: complementary
research strategies. Thousand Oaks, CA: Sage Publications.
Dell'Era, C., & Landoni, P. (2014). Living Lab: A Methodology between User-Centred
Design and Participatory Design. Creativity and Innovation Management,
23(2), 137-154. doi:10.1111/caim.12061
Dutilleul, B., Birrer, F., & Mensink, W. (2010). Unpacking European Living Labs:
Analysing Innovation’s Social Dimensions In K. Müller, S. Roth, & M. Zak
(Eds.), Social Dimension of Innovation. Prag; Linde.
ENoLL. (2006). European Network of Living Labs (ENoLL). Retrieved from
Georges, A., Schuurman, D., & Vervoort, K. (2016). Factors Affecting the Attrition of
Test Users During Living Lab Field Trials. Technology Innovation
Management Review, 6(1).
Grbich, C. (2013). Qualitative Data Analysis: An Introduction. London: SAGE
Publications, Ltd.
Habibipour, A., & Bergvall-Kåreborn, B. (2016). Towards a User Engagement
Process Model in Open Innovation

Paper presented at the The International Society for Professional Innovation
Management (ISPIM) Innovation Summit: Moving the Innovation Horizon (4-7
December 2016), Kuala Lumpur, Malaysia
Habibipour, A., Georges, A., Schuurman, D., & Bergvall-Kåreborn, B. (2017). Drop-
out in Living Lab Field Tests : A Contribution to the Definition and the
Taxonomy. Paper presented at the Research Day Conference proceedings
2017: OpenLivingLab Days 2017,, Krakow.
Habibipour, A., Padyab, A., Bergvall-Kåreborn, B., & Ståhlbröst, A. (2017) Exploring
factors influencing participant drop-out behavior in a living lab environment. In:
Vol. 294 (pp. 28-40): Springer Verlag.
Müller, S., & Sixsmith, A. (2008). User requirements for ambient assisted living:
results of the SOPRANO project. Gerontechnology, 7(2).
Novitzky, P., Smeaton, A. F., Chen, C., Irving, K., Jacquemard, T., O’Brolcháin, F., . .
. Gordijn, B. (2015). A Review of Contemporary Work on the Ethics of
Ambient Assisted Living Technologies for People with Dementia. Science and
Engineering Ethics, 21(3), 707-765. doi:10.1007/s11948-014-9552-x
NSW. (2015). NSW Health Code of Conduct. Retrieved from
Nyström, A.-G., Leminen, S., Westerlund, M., & Kortelainen, M. (2014). Actor roles
and role patterns influencing innovation in living labs. Industrial Marketing
Management, 43(3), 483-495.
Pallot, M., Trousse, B., Senach, B., & Scapin, D. (2010, 2010-08-25). Living Lab
Research Landscape: From User Centred Design and User Experience
towards User Cocreation. Paper presented at the First European Summer
School "Living Labs", Paris, France.
Panek, P., Rauhala, M., & Zagler, W. L. (2007). Towards a Living Lab for Old People
and their Carers as Co-Creators of Ambient Assisted Living (AAL)
Technologies and Applications. Paper presented at the 9th Europ Conf for the
Advancement of Assistive Technology in Europe (AAATE).
Pino, M., Benveniste, S., Kerhervé, H., Picard, R., Legouverneur, G., Cristancho-
Lacroix, V., . . . Rigaud, A.-S. (2014). Contribution of the Living Lab approach
to the development, assessment and provision of assistive technologies for
supporting older adults with cognitive disorders. Stud. Inform. Univ., 11(2), 34-
Ritchie, J., Lewis, J., Nicholls, M. C., & Ormston, R. (Eds.). (2014). Qualitative
Research Practice: A Guide for Social Science Students and Researchers.
London: SAGE Publications.
Sainz, F. J. (2012). Emerging Ethical Issues in Living Labs. Ramon Llull Journal of
Applied Ethics, 3, 47-62.

Saldana, J. (2012). The Coding Manual for Qualitative Researchers. Thousand
Oaks, CA: SAGE Publications.
Sanchez, V. G., Taylor, I., & Bing-Jonsson, P. C. (2017). ETHICS OF SMART
LITERATURE REVIEW. Int J Technol Assess Health Care, 33(6), 691-699.
Schreier, M. (2012). Qualitative Content Analysis in Practice. London: SAGE
Schuurman, D., De Marez, L., & Ballon, P. (2015). Living Labs: a systematic
literature review. Paper presented at the Open Living Lab Days 2015.
Schuurman, D., Mahr, D., De Marez, L., & Ballon, P. (2013). A Fourfold Typology of
Living Labs: An empirical investigation amongst the ENoLL community. Paper
presented at the 2013 International Conference on Engineering, Technology
and Innovation (ICE) & IEEE International Technology Management
Conference (24–26 June 2013), The Hague, The Netherlands.
Ståhlbröst, A. (2012). A set of key principles to assess the impact of Living Labs.
International Journal of Product Development, 17(1-2), 60-75.
Ståhlbröst, A., & Bergvall-Kåreborn, B. (2013). Voluntary Contributors in Open
Innovation Processes. In J. S. Z. Eriksson Lundström, M. Wiberg, S.
Hrastinski, M. Edenius, & P. J. Ågerfalk (Eds.), Managing Open Innovation
Technologies (pp. 133-149). Berlin, Heidelberg: Springer Berlin Heidelberg.
van Geenhuizen, M. (2014). Living labs: concepts and critical factors, with case
studies from health care. Paper presented at the The 14th ICTPI Conference -
Building Sustainable R&D Centers in Emerging Technology Regions (9-12
September 2014), Brno, Czech Republic.
van Geenhuizen, M. (2018). A framework for the evaluation of living labs as
boundary spanners in innovation. Environment and Planning C: Politics and
Space, 0(0), 2399654417753623. doi:10.1177/2399654417753623
Walsh, S. A. (2009). Conducting Research with the Elderly: Ethical Concerns For a
Vulnerable Population. Southern Online Journal of Nursing Research, 9.
Retrieved from
Wenger, E., McDermott, R. A., & Snyder, W. (2002). Cultivating communities of
practice: a guide to managing knowledge. Boston: Harvard Business School
Yazdizadeh, A., & Tavasoli, A. (2016). Living Labs as a Tool for Open Innovation: a
Systematic Review International Journal of Humanities and Cultural Studies,

needs and

Living labs as instrument for societal change:
the role of intermediary actors and activities
Jos van den Broek*1, Timo Maas1,
Isabelle van Elzakker1, Jasper Deuten1

*Corresponding author
1 Rathenau Instituut, The Hague

Category: Research in-progress


Not provided

Keywords: Living labs, societal challenges, intermediaries, experiments

1 Introduction

Science and innovation policy are increasingly focused towards societal challenges
such as climate change (SDG 13), sustainable mobility (SDG 11) and clean energy
(SDG 7). Whilst the EU can be seen as a frontrunner in this recent policy shift,
national and regional governments are also refocusing their science and innovation
policy efforts towards societal challenges. This requires not only science to rebalance
their efforts but also a more proactive government (Mazzucato, 2017), as well a
transformation of governance arrangements and policy instruments (Kuhlmann &
Rip, 2014). Here, we are interested in one instrument for such innovation policy for
societal challenges, so-called ‘living labs’. In an earlier study on living labs in the
Netherlands (Maas et al., 2017) we developed a fourfold typology of different kinds
of living labs. We argued only one of these types was especially suitable to contribute
to societal challenges, namely the type which takes place in real-life environments
and focuses on co-creating solutions with a wide variety of actors. However, we also
saw a lack of learning within and between these kinds of living labs and only marginal
attention for upscaling of initiatives and learning experiences. We contend that
upscaling, both in terms of replication of knowledge in other places as well as in terms
of institutionalization of knowledge in new rules and practices, is a crucial element for
living labs to be useful as instruments of innovation policy for societal challenges.
However, someone, both individual actors or organizations, is needed who takes
responsibility for upscaling and is able to move beyond local, context specific
initiatives. By intermediating these actors can play an important role in transitions
towards societal change. As Kivimaa and colleagues (Kivimaa et al., 2017a) show,
transitions require an ecology of different kinds of intermediaries at different moments
in time. These intermediaries take on a whole range of activities, some within a
narrowly defined field or sector, others have broader goals and activities. In this
research we specifically focus on this role of intermediary actors which leads us to
ask the research question of how can intermediary actors contribute to societal
transitions by upscaling local experiments, such as living labs?

To answer these questions, we use a short literature review and we look at ‘historical’
examples in which local knowledge had to be transformed and re-embedded in
different contexts. Both the literature review and the historical examples are still work
in progress and we aim to present our preliminary results.

Later in the process, but beyond the scope of the current paper, we will use an
exploratory qualitative case study approach to look at three domains in which societal
transitions are deemed necessary to take place in the Netherlands: healthcare,
circular economy and a low-carbon economy. Within these three domains we aim to
look at how several living lab initiatives and intermediary activities are operating in
practice. For each case we conduct in-depth interviews with participants, policy-

makers, managers and stakeholders complemented with desk study of project plans,
websites, policy documents and other materials.
At the conference we aim to first discuss our interpretation of intermediation and
upscaling in the other domains and, second, show some first results from the
interviews. As we are in the midst of doing the case studies at that moment, we are
open to suggestions and discussion on both the first results and the research

2 Living labs in theory

Despite the rising attention for living labs, a clear reference point for what a living lab
entails is yet missing from the discussion (Schuurman, 2015). One attempt at a
comprehensive definition is found in Schliwa and McCormick’s (2016) description of
living labs as “physical arena as well as a collaborative approach in which different
stakeholders have space to experiment, co-create and test innovation in real-life
environments defined by their institutional and geographical boundaries”. Crucial
keywords in this definition are collaboration/co-creation and real-life experimentation.
The involvement of citizens is often seen as central to what distinguishes living labs
from other forms of public-private partnerships for research and innovation. The
European Commission has referred to living labs as ‘public-private-people
partnerships (EC 2016, our emphasis), others choose terms such as ‘quadruple helix’
to indicate that the ‘triple helix’ of science, industry and government is supplemented
by citizens as a fourth group (Carayannis and Campbell, 2009). Nonetheless, as
Schliwa and McCormick (2016) note, there is a distinction between people involved
in their capacity of (potential) user of the innovation and people involved in their full
capacity of “citizen”, the latter suggesting a higher degree of co-creation, which in
turn means that laypersons are able to put their needs and preferences front and

Furthermore, living labs are characterized by the fact that the environment in which
this co-creation takes place shifts to the ‘real world’. In this respect, living labs answer
the call for research and innovation that incorporates real-life complexity. Such a role
for what Callon and Rabeharisoa (Callon and Rabeharisoa, 2003) have called
‘research in the wild’ has previously been documented for urban studies (Gieryn,
2006) and economics (Levitt and List, 2009). Bringing together a wider set of
collaborators in a ‘wild’ setting results in an increasingly ‘hybrid’ form of knowledge
that is produced, including attention to non-technological aspects to innovation such
as regulation, public acceptance, and business models. This shift to real-world
research is accompanied by a number of ethical questions. To what extent are
practices of prior informed consent maintained in such a setting? What is a

responsible way of involving citizens in research, respecting e.g. their privacy and
safety? But also, an experiment always involves some intervention in the world, which
means it is important to critically examine their embedded politics (Karvonen and van
Heur, 2014).

This last point is especially important in light of the third aspect: the involvement of
local government. There are two reasons for this. Pragmatically, local government
often bears responsibility for public space, so that living labs’ explicit inclusion of local
context requires some form of cooperation with local government. But recent years
have also seen local government actively take up a role as ‘change agent’ (Barber,
2013; Bulkeley and Castán Broto, 2013). From this perspective, experiments are an
element of urban development strategies (Evans et al., 2016; Evans and Karvonen,
2014). A prominent example are ‘smart city’ experiments employing digital technology
for urban policy; experiments that have been the subject of fierce debate on whose
interests they serve, and how to safeguard public interests (Kitchin, 2014; Townsend,
2013; Viitanen and Kingston, 2014).

3 Upscaling living labs

The fact that living labs take place in a real-life experimental environment with
extensive involvement for local stakeholders means they are very well rooted in the
local context. This is important for ensuring that the solution will ‘work’ in this context,
but poses challenges for wider societal change. Whereas too little attention to local
context means that the solution does not fit the problem, a number of contemporary
challenges are calling for transformative change on the short rather than the long
term, meaning that cross-locational learning is important as a way of accelerating
change. In this section we briefly reflect on some theoretical perspectives on how
living labs can contribute to societal change, which provides the basis for our ongoing
empirical research.

From the field of transition studies, we can gain insight in the way living labs may lead
to change by seeing them as ‘niches’. These are radical innovations that take place
protected from market selection, and are often situated against the dominant socio-
technical system (‘regime’) and factors that are slow-changing and considered
external to this system (‘landscape’) (Geels, 2002). A more transformative shift in the
wider system can then be achieved through aggregated and comparative learning
obtained by linking together various niches with other developments at the regime
and landscape level (Schot and Geels, 2008). An important question is whether a
niche is made competitive within an existing selection environment (‘fit and conform’),
or whether it can catalyse change in this environment that works in the niche’s favor

(‘stretch and transform’), which could open up space for more radical innovation
(Smith and Raven, 2012). From this perspective the coordination between various
levels of government deserves particular attention, because their actions are
influential in both these processes of empowerment.

A difficulty with this transitions perspective is that it has little to say about scale;
implicitly it often seems to take the ‘regime’ to be on the national scale, and ‘niches’
to be occurring at city-level (Bulkeley et al., 2014). However, the question of the scale
on which living labs aspire to create change is an open one. Many of the actors
involved are not per definition motivated by grand society-wide goals but are satisfied
with making adjustments in their immediate surroundings. This is especially the case
for involved citizens and municipal governments. We might expect corporations and
knowledge institutes to have a greater interest in the bigger picture, as it implies new
business and transferable knowledge for these actors, respectively. Learning then
might take place in different networks (e.g. more international) and for different
motives (e.g. global city aspirations). Literature on policy mobilities provides useful
insights by shifting the focus to the work that is done in the process of policy
(knowledge) travelling and being re-embedded in different contexts (McCann and
Ward, 2012; Peck, 2011). This perspective explicitly takes the (micro)politics of these
mobilities into account, for instance by considering the discourses that are being
mobilized to push for the ‘export’ or ‘import’ of certain approaches.

A specific perspective on upscaling sustainability experiments is provided by Naber

et. al. (2017) who distinguish between four patterns of upscaling. The first pattern is
growing the experiment at the specific places and thereby widening the area of the
number of actors involved. Second, they distinguish a replication pattern whereby the
experiment is replicated at different locations or in different contexts. A third pattern
is that of accumulation whereby the experiments are explicitly linked to other
initiatives. Finally, they see a pattern of transformation whereby the experiment
shapes the sets of norms, habits and routines on a regime level.

4 Intermediary activities

A central part of the policy mobilities literature is the emphasis on who does the work
to decontextualize and re-embed the policy. Similarly, within transition studies there
is growing attention for the role of intermediary actors in the process of upscaling.
Although there has been a long standing interest in innovation studies in the role of
actors who develop and maintain networks through which knowledge can be spread
(Howells, 2006), more recently attention has been focused upon the role of
intermediaries that are specifically focused on transitions (Kivimaa et al., 2017a).

These transition intermediaries differ from more traditional ones in having a clear idea
of the direction of innovation. It is not about spurring innovation in general, but instead
they focus upon a specific field for which innovation is needed. Kivimaa and
colleagues (2017; p. 12) differentiate between different kinds of roles that transition
intermediaries can play on different levels:

i) a systemic or strategic intermediary operating on all scales and taking a

system perspective on change;
ii) a regime intermediary that is tied through, for example, institutional
arrangements or interests to the established regime but has a specific
mandate or goal to promote transition and, thus, interacts (often with a range
of) niches;
iii) a niche intermediary typically working to experiment and advance niche
activities as part of a particular niche but also trying to influence the regime
level for that niche’s benefit;
iv) a process intermediary that facilitates a change process or a project without
explicit individual agency or agenda, but may work to advance priorities set
externally by other actors; and
v) a user intermediary translating new technologies to users and user
preferences to developers, and qualifying the value of technology offers

Some intermediaries are established specifically for this transition task, whereas
often these are organizations or individuals who carry out these intermediary activities
from within existing organizations. Furthermore, a transition requires an ecology of
different kinds of intermediary actors and activities at different moments in time
(Hyysalo et al., 2018; Kivimaa et al., 2017a). In this ecology the interconnected actors
each fulfill different functions to contribute to the upscaling process.

The nature of the activities of intermediary actors is diverse. They can be involved in
articulating demands and expectations, building and maintaining networks, and
facilitating learning processes. These activities materialize in activities such as
organizing conferences and site visits, the writing of handbooks, setting standards
and procedures and conducting monitoring and evaluation (Boon et al., 2011; Boon
and Edler, 2018; Kivimaa, 2014; Matschoss and Heiskanen, 2017). One of the central
questions within this discussion about intermediaries is how an ecology of
experiments and intermediary organizations would look like. Which types of
experiments are needed (Kivimaa et al., 2017b)? Which type of intermediary
organizations can contribute to upscaling of experiments (Kivimaa et al., 2017a)? And
what is the role of different kinds of quadruple helix stakeholders in stimulating this
ecology (firms, governments, universities and societal organizations)? These
questions, together with the four patterns of upscaling from the previous paragraph,

provide input to a preliminary heuristic (table 2) to look at how intermediation and
upscaling can take place.

Table1. Heuristic of intermediation per type of upscaling pattern

Type of Role of Activities of

intermediary intermediary intermediary
Niche Scaling the • Network building within
Growing intermediary experiment in a a city
User intermediary specific place. • M&E of experiment
Regime • External networking
Translating the
intermediary • M&E of experiment
Replication experiment to a
Process • Organising and
different place. documenting learning
• External networking
Linking knowledge • M&E of experiment
intermediary • Organising and
and experience
Accumulation Regime documenting learning
with other • Organizing learning
initiatives. visits
• Organizing conferences
• External networking
• M&E of experiment
• Organising and
documenting learning
Shaping wider
Systemic • Organizing learning
Transformation institutional visits
change • Organizing conferences
• Writing handbooks
• Developing rules and

5 Upscaling and intermediation in historical examples

If we want to understand how upscaling and intermediation are taking place within
current living labs we face the problem that many living labs have only recently been
started and the local experiment is still ongoing. In most cases there is not yet much
to scale-up, although there are always exceptions. But scaling up and the role of
intermediaries is not an issue that is exclusive to living labs, there have been earlier
domains and areas where the issue of scaling up local knowledge and experiments
played a role and where solutions have been found to deal with this issue. Particularly
we have looked at three domains in which these issues have been at stake: in the
agricultural sector in the Netherlands, in the Water sector in the Netherlands and in
development studies at large. For each of these domains we have documented what
role scaling played here and how intermediary activities have played a role in

fostering this scaling process.


The importance of a self-sustaining agricultural sector became an important policy

goal in the Netherlands in the years after the second World War. For this goal to be
realized the competitiveness and efficiency of production of Dutch agriculture needed
to be improved. This led to the development of the so-called OVO-triptych in which
Research (Onderzoek), Counselling (Voorlichting) and Education (Onderwijs) came
together in which new knowledge was developed by research institutes, for example
in experimental gardens and experimental farms, then this knowledge was actively
disseminated by counsellors to the farmers. These counsellors functioned as an
intermediary between research and the workplace whereby they also collected
problems and issues from farmers that could inform researchers about their next
projects. This model functioned very well until the 1980s after which there came more
attention for issues such as environmental degradation and animal welfare together
with an improvement of the education levels of farmers. This led on the one hand to
a diversification of policy goals and more discussion about the direction of knowledge
an innovation for agriculture and at the same time farmers where now better educated
and better able to critique and discuss research results, In response to this
development one of the projects that were initiated was called Transform in which a
more co-creative approach was taken whereby researchers, farmers, government
representatives and societal organisations together worked on, for example, new
stable concepts for pigs and environmentally friendly and animal friendly stables for
chickens. Within this new approach a kind of counsellors 2.0 were appointed who
functioned as knowledge intermediaries between these different stakeholders.
Rather than being counsellors bringing new research results to farms, these
knowledge intermediaries orchestrated the coming together of different stakeholders
around an innovation challenge.

Upscaling within the agricultural sector thus developed from a linear approach
towards a more co-creative endeavour whereby in both cases there was a role for an
intermediary actor, but with a very different role. These intermediary actors were
dedicated professionals that were specifically trained and appointed for this role.


At the end of the last century it became clear that the water management and coastal
defences of the Netherlands were not future-proof. Through a mix of different
research and innovation programs, sometimes running in parallel, sometimes
consecutively, a knowledge base has been created for technical and organizational

improvements in the water chain (sewers, drinking water, water treatment); renewal
of river management, so that rapid water rises cause less nuisance and the ecological
value of the rivers is increased; development of new forms of coastal defense and
integral and climate-robust water management in cities.
The experiences of the programs in joint knowledge development and innovation are
continued in the National Knowledge and Innovation Program Water and Climate and
the Knowledge Network Delta Program of the Delta Program Commissioner. The
program forms a bridge between science and practice and brings together knowledge
about a large diversity of water-related tasks. This ranges from research lines aimed
at refining climate models to lines that are aimed at disseminating knowledge about
climate-proof cities.

Within each research line, knowledge sharing takes place via, among other things,
knowledge newspapers, study days and project visits. In addition, there is an active
knowledge sharing program at the level of the Delta Program, in which, among other
things, knowledge is made available through so-called 'Delta Facts'. The Delta
Program also has a knowledge agenda with questions that are, among other things,
being made available to partners such as Rijkswaterstaat and Deltares. Once a year,
a broad knowledge conference is organized at the level of the Delta Program. In
addition, the NKWK is also part of the international Delta Alliance, an international
network of areas located in deltas, aimed at knowledge exchange with the aim of
strengthening the resilience of delta areas. This alliance consists of 18 territories in
15 countries.

The water domain is an example of a domain in which knowledge sharing and

learning from each other are actively organized from a broadly felt urgency. The
national government has taken a leading role in this, but the implementation takes
place in a broad partnership.

Development studies

Within the development sector many projects take place on a small, local scale.
External financiers or development organizations are often involved in this. Even
when these projects prove successful, this does not mean that this success
automatically spills over to other local contexts, or is picked up by politicians and
policymakers. In addition, support from external financiers and development
organizations is often temporary. Because of this, both spreading the success and
sustained existence is a challenge.

The above observation has led to a lot of attention within the development sector for
how successful projects can be scaled up. Although there are different ideas about

what this scale up entails, a shared urgency is felt to increase the potential impact of
small-scale interventions. Instead of continuously adapting one-off (possibly
successful) projects and interventions, upscaling should contribute to making better
use of successes by broadening and sustaining existing interventions ("multiplier
effect"). The link is made with the major societal challenges, as formulated in the
MDGs and now the SDGs, which call for joint action and an acceleration in its

One of the most important messages from the development studies community is
that the successful scaling up of an initiative requires that structural attention be paid
to possible upscaling from the outset. This does not mean scaling up or becoming an
end in itself, but it does mean that from the start of a project possibilities for upscaling
and creating the conditions under which upscaling can take place are considered.
Another consequence of this structural attention for upscaling is the role of knowledge
building and reflection in enabling possible, successful, upscaling. Crucial to the
success of upscaling efforts is learning. The starting point here is that upscaling is an
iterative process. This means that there must be continuous questioning whether the
project achieves the intended goals, but also what is needed to scale up any success.
This requires a monitoring and evaluation strategy. Results can be used to reflect on
visualized scaling-up pathways and to adjust these where necessary.

A final lesson from the development studies community is the attention for building
capacity and institutional support. This is related to the other lessons: both a well-
founded strategy and the involvement of stakeholders from the beginning (who are
not directly involved in the project but can be mobilized at a later stage) contribute to
creating support for the concrete scaling up process. . In the development sector,
building capacity also means: ensuring that innovative processes can be
institutionalized in existing policy structures and pass through any political changes.


Barber, B.R., 2013. If Mayors Ruled the World: Dysfunctional Nations, Rising Cities.
Yale University Press, New Haven.
Boon, W., Edler, J., 2018. Demand, challenges, and innovation. Making sense of new
trends in innovation policy. Sci. Public Policy 1–13.
Boon, W., Moors, E., Kuhlmann, S., Smits, R., 2011. Demand articulation in emerging
technologies: Intermediary user organisations as co-producers? Res. Policy
40, 242–252.
Bulkeley, H., Castán Broto, V., 2013. Government by experiment? Global cities and
the governing of climate change. Trans. Inst. Br. Geogr. 38, 361–375.
Bulkeley, H., Castan Broto, V., Maassen, A., 2014. Low-carbon Transitions and the
Reconfiguration of Urban Infrastructure. Urban Stud. 51, 1471–1486.
Callon, M., Rabeharisoa, V., 2003. Research “in the wild” and the shaping of new
social identities. Technol. Soc. 25, 193–204.
Carayannis, E.G., Campbell, D.F.J., 2009. “Mode 3” and “Quadruple Helix”: toward a
21st century fractal innovation ecosystem. Int. J. Technol. Manag. 46, 201.
Evans, J., Karvonen, A., 2014. “Give Me a Laboratory and I Will Lower Your Carbon
Footprint!” - Urban Laboratories and the Governance of Low-Carbon Futures.
Int. J. Urban Reg. Res. 38, 413–430.
Evans, J., Karvonen, A., Raven, R. (Eds.), 2016. The Experimental City. Routledge,
Geels, F.W., 2002. Technological transitions as evolutionary reconfiguration
processes: a multi-level perspective and a case-study. Res. Policy 31, 1257–
Gieryn, T.F., 2006. City as Truth-Spot. Soc. Stud. Sci. 36, 5–38.
Howells, J., 2006. Intermediation and the role of intermediaries in innovation. Res.
Policy 35, 715–728.
Hyysalo, S., Juntunen, J.K., Martiskainen, M., 2018. Energy Internet forums as
acceleration phase transition intermediaries. Res. Policy 47, 872–885.
Karvonen, A., van Heur, B., 2014. Urban Laboratories: Experiments in Reworking
Cities. Int. J. Urban Reg. Res. 38, 379–392.
Kitchin, R., 2014. The real-time city? Big data and smart urbanism. GeoJournal 79,

Kivimaa, P., 2014. Government-affiliated intermediary organisations as actors in
system-level transitions. Res. Policy 43, 1370–1380.
Kivimaa, P., Boon, W., Hyysalo, S., Klerkx, L., 2017a. Towards a Typology of
Intermediaries in Transitions: a Systematic Review. Swps 17, 2017–17.
Kivimaa, P., Hildén, M., Huitema, D., Jordan, A., Newig, J., 2017b. Experiments in
climate governance – A systematic review of research on energy and built
environment transitions. J. Clean. Prod. 169, 17–29.
Levitt, S.D., List, J.A., 2009. Field experiments in economics: The past, the present,
and the future. Eur. Econ. Rev. 53, 1–18.
Maas, T., Van den Broek, J., Deuten, J., 2017. Living labs in Nederland: Van open
testfaciliteit tot levend lab. Den Haag.
Matschoss, K., Heiskanen, E., 2017. Making it experimental in several ways: The
work of intermediaries in raising the ambition level in local climate initiatives.
J. Clean. Prod. 169, 85–93.
McCann, E., Ward, K., 2012. Policy Assemblages, Mobilities and Mutations: Toward
a Multidisciplinary Conversation. Polit. Stud. Rev. 10, 325–332.
Naber, R., Raven, R., Kouw, M. & Dassen, T. 2017. Scaling up sustainable energy
innovations. Energy Policy 110, 342-354.
Peck, J., 2011. Geographies of policy: From transfer-diffusion to mobility-mutation.
Prog. Hum. Geogr. 35, 773–797.
Schliwa, G., McCormick, K., 2016. Living labs: users, citizens and transitions, in:
Evans, J., Karvonen, A., Raven, R. (Eds.), The Experimental City. Routledge,
London, pp. 163–178.
Schot, J., Geels, F.W., 2008. Strategic niche management and sustainable
innovation journeys: theory, findings, research agenda, and policy. Technol.
Anal. Strateg. Manag. 20, 537–554.
Schuurman, D., 2015. Bridging the gap between open and user innovation? Exploring
the value of living labs as a means to structure user contribution and manage
distributed innovation. Universiteit Gent, Gent.
Smith, A., Raven, R.P.J.M., 2012. What is protective space? Reconsidering niches
in transitions to sustainability. Res. Policy 41, 1025–1036.
Townsend, A.M., 2013. Smart cities: Big Data, Civic Hackers, and the Quest for a
New Utopia. WW Norton, New York.
Viitanen, J., Kingston, R., 2014. Smart cities and green growth: outsourcing
democratic and environmental resilience to the global technology sector.

Environ. Plan. A 46, 803–819.

Models for Living Lab’s sustainability. Evidences
from Italy and the Netherlands
Edoardo Gualandi1 and Luca Leonardi2

1 Technical University of Eindhoven

2 Consorzio ARCA, Palermo

Category: Research in-progress

Living Lab is either part of, or constitutes an Innovation Network of people, private
firms and public institutions. The Living Lab has the potential to convey value to the
entire network but it requires the support and commitment of its stakeholders to
operate at full regime and ensure long term viability. For this reason, financial
sustainability is one of the main challenges for any Living Lab. With the current paper,
the objective is to understand how long-term viability can be reached through a
sustainable business model based on the right collaboration mode in a multi-
stakeholder environment. This research proposes a theoretical framework in which is
stressed the importance for the Living Lab of the innovation network and of the
creation and demonstration of value to the stakeholders. Then, based on this
framework, three practical cases are analyzed: Stratumseind Living Lab in Eindhoven,
the AUAS field labs in Amsterdam and the ARCA Textile and Clothing Living Lab in
Palermo. From the observation of the three Living Lab, this paper proposes a
distinction between product-oriented and policy-oriented Living Lab and presents
some meaningful implications of the two approaches to sustainability.

Keywords: Living Lab, Sustainability, Innovation Network, Value creation, Social

Innovation, Stakeholder Commitment.

1 Introduction and objectives

Living Lab is either part of, or constitutes an innovation network of people, private firms
and public institutions based on knowledge and observation, and guided by a practice
driven approach to co-creation. The entire ecosystem built around the Living Lab
concurs in the realization and implementation of user- or community-driven innovative
solutions. The Living Lab conveys value to the entire network but it requires the
support and commitment of several stakeholders to operate at full regime and ensure
long term viability. Indeed, sustainability is a key challenge for any Living Lab. Building
mutual trust and identifying a set of shared objectives with the actors of the
surrounding ecosystem is crucial for the sustainability and success of the Living Lab.
To this purpose, it’s not sufficient that the Living Lab creates value, but it must also
demonstrate it across the network, and make both methods and results tangible to
every innovation partner. For this reason, it’s fundamental for a Living Lab to deeply
understand its network and a comprehensive multi-stakeholder analysis is necessary
to identify the way to produce value and deliver it to each actor.

This paper has the objective to identify the strategies developed by three different
Living Labs to address the problem of sustainability: Stratumseind Living Lab (SLL) in
Eindhoven, AUAS field labs in Amsterdam and Textile and Clothing Living Lab
(TECLA) in Palermo. Creating an economically viable Living Lab means aligning
internally consistent processes with the needs of external stakeholders so that
revenues are generated and funding are ensured (Katzy, 2012). This research
investigates the relationship between Living Lab’s activities, innovation network, value
creation, delivery and demonstration to the stakeholders. Developing a clear view over
the complex interrelations among the several actors is fundamental to understand the
mechanisms that allow a Living Lab to generate revenues and thus ensure

2 Literature framework

Living Lab as an innovation network

The concept of innovation network is crucial to understand Living Lab. In fact, Living
Lab is not only a network of infrastructures but especially a network of people with
their experiences. (Mulder, 2008). The Living Lab indeed serves as connection
between research, citizens and the actual living environment (Franz, 2014). Living
Lab is a network of different stakeholders in which the existing relationships are
supported, while it’s given the opportunity to new partners to meet and collaborate to
develop new products and services (Bergvall-Kåreborn et al., 2009). The Living Lab

has a role of innovation intermediary in a complex network of different stakeholders: it
functions as an aggregator of various external inputs that are then translated into
requirements and design parameters for valuable social innovation (Mention &
Torkkeli, 2015; Almirall & Wareham, 2011).
Living Lab is a form of Public-Private-People Partnerships (PPPP) in which, not only
users and producers, but also many other heterogeneous stakeholders, such as firms,
public agencies, universities, and institutes build an innovation network and
collaborate at the creation, prototyping, testing and evaluation of new products and
services (Leminen et al, 2012; Niitamo et al., 2006).
Finally, it’simportant to notice that the innovation network around the Living Lab is
constituted of parties with different backgrounds and interests; moreover, these
stakeholders often come from a history of interrelations in which roles and
expectations are already defined (Eskelinen et al., 2015). The elements above stress
the importance for a Living Lab to understanding the complex nature and the dynamics
of the innovation network to be able to develop adequate context-specific strategies.

The need for Sustainability

Sustainability is crucial for future feasibility of Living Lab. A significant number of Living
Labs present an unintended temporary nature since they stop their activities when the
funding ends (Leminen et al., 2012). Many researchers advocate for developing a
strong model based on long-term strategy and suggest that a more detailed analysis
of this aspect is required (Bergvall-Kareborn et al, 2009; Mastelic et al., 2015)

Sustainability, in the context of Living Lab, refers to two aspects (Bergvall-Kareborn et

al, 2009). First, it’s considered viability as a pure economical aspect. It refers to the
possibility to finance every-day activities, and have the financial availability to provide
a service respecting the planned standard. The second aspect is the Living Lab’s
responsibility to its wider community. This quality must be supported and ensured
during the entire life of a Living Lab. Thus, it’s fundamental for a Living Lab, not only
to sustain economically its activities, but also to continuously expand its network and
strengthen the partnerships with crucial sources of innovation and knowledge.

Living Lab’s outcome

If the Living Lab is adequately financed overtime, its activities allow the several
stakeholders to benefit from:

• Valorization of knowledge: several research components of Living Lab allow the

transformation of knowledge into models, methods and theories (Bergval-

Kareborn et al., 2009; Veeckman et al., 2013). New products, services and
policies can be developed based on this knowledge generated.

• Exploitation of network: through the interaction inside a Living Lab, every actor
can draw from a wide range of competences and combine disparate sources of
knowledge to implement better products and services. Therefore, every partner
is an added value for the network and at the same time can benefit from it
(Veeckman et al., 2013).

• Social impact: The output of the development process is a product or a service

which, not only brings a direct economic benefit to the commissioner, but also
improves living conditions and the social environment.

Value creation

It’s evident that the value delivered by Living Lab varies in terms of nature, effected
actors and implications. Thus, this research provides a classification of value into three
different categories:

• Economic Value covers economic metrics that are highly tangible from different
stakeholders’ perspectives (Baccarne et al., 2014). Thus, these results can be
assessed and evaluated from an economical perspective. (Bergvall-Kareborn,

• Economic value can be extended to include other forms of value such as

employee value, customer value, supplier value, managerial value and societal
value (Bergvall-Kareborn, 2009). This is referred as Business Value. The
generated business value can bring relevant benefits to the various actors of the
innovation ecosystem.

• Finally, Public Value is delivered by Living Lab when it supports and promote the
implementation of solutions that respond to local challenges and opportunities,
and contribute to the achievement of public policies (Cosgrave & Tryphonas,

3 Research questions

The literature framework developed in the previous section constitutes a starting point
for researching how Living Lab should address the challenge of financial sustainability.

The framework shows that the activities of the Living Lab produce a certain outcome
which translates in three different categories of value. The process of value creation
is enabled only if there is complete commitment by all the actors of the Living Lab’s
innovation network. Literature advocates for the development of a business model that
can ensure financial sustainability. This paper wants to contribute to current research
by investigating in practice what different Living Labs do to reach sustainability. In
particular, this research focuses on how the nature of the network is linked to the
outcome of the Living Lab, and how the Living Lab creates and demonstrates value to
its stakeholders.

Consequently, the following section of this study are guided by the following research

• Which stakeholders constitute the network of the Living Lab and how do they
participate to the outcome of the Living Lab?

• What kind of value does the Living Lab create and how is the value delivered and
demonstrated to each stakeholder?

• Which business model, forms of funding, and other revenue streams ensure Living
Lab’s sustainability?

4 Methodology

The chosen research strategy is a general discussion of a more detailed case study.
The adequateness of this method was analysed according to the principles expressed
by Yin (2003). For the purposes of this research, three successful cases are analysed
in detail: Stratumseind Living Lab and AUAS field labs represent two valuable
examples that reached a phase of maturity. Despite operating with diametrically
opposed strategies, they developed a sustainable business model, which ensures
long-term viability and allows them to have a concrete impact in the innovation
ecosystem and in the wider society. Then, it’s presented ARCA Textile and Clothing
Living Lab which is still in a start-up phase. For each case, it’s first provided a short
overview and contextualization. Then, it’s described their innovation network. Finally,
it’s presented the way they deliver economic, business and public value to their
stakeholders, and how this value is translated in findings and revenue streams. The
information was collected through qualitative methods: interviews with several
stakeholders, observation of daily activities directly on the field, and specific

5 Empirical research

Stratumseind Living Lab

Context The municipality of Eindhoven in the last years developed various living lab
initiatives with the support of several actors from diverse sectors. SLL, which is part of
the Stratumseind 2.0 & Smart City project, is one of the most successful examples.
Stratumseind is a pub street about 300 meters long that hosts around 50 pubs in the
centre of Eindhoven. Every week more than 20.000 visitors enjoy the nightlife but also
around 800 incidents a year are recorded.

Description SLL is an instrument to measure the influence that can be created on

behaviour through the application of diverse solution based on light, fragrance and
design. The Living Lab is a test facility for new sensors, data harvesting, privacy, smart
interfaces, smart lighting systems, design and gaming.

Challenge The mission of SLL is to support companies and institutions in the

development of innovative products, services and policies which can structurally
improve the economic and social functioning on the Stratumseind.

Network Thanks to the highly technological content and to fervid innovation

environment of Eindhoven, SLL involves a massive number of partners.

• Governmental institutions: SLL involves local administrations such as the

municipality of Eindhoven, Tilburg and the province of Noord-Brabant; public
organizations, like the Police Department; also, other Public-Private-People-
Partnerships (PPPPs) like Brainport Eindhoven, which is a triple helix

• Educational institutions: The Eindhoven University of Technology represents one

of the main sources of knowledge, but several other universities, not only from the
region, are involved to a different degree.

• Businesses: many private companies take part to the activities of SLL. The degree
of maturity of such organizations is various: technological start-ups as well as
multinational companies, can find the right compromise to use the services of the
Living Lab.

• Citizens: The involvement of the citizens is mainly passive. In fact, they serve the
Lab as a subject to observe. In addition, they allow experimentation and evaluation
of new technologies. The vast majority of the observed citizens, isn’t aware to be
part of the Living Lab.

Outcome SLL facilitates the combination of knowledge, experience, competences
and technologies of different actors to develop better co-created products and
services. The resulting new products, services and policies not only have substantial
competitive advantages, but also improve living conditions in Stratumseind.

Value creation Economic value is delivered to private businesses and organizations

which can ideate better products and services. Business value is delivered to
companies and organizations which benefit from the experience of the SLL and of the
other parties. They gain new knowledge, establish relationships with research
institutions and improve their business model and innovation capacity. Public value is
delivered to governmental institutions that obtain the knowledge necessary for the
development of better policies. Citizens obtain a safer and more liveable street for

Funding and revenue SLL began by a small funding by the municipality of Eindhoven,
then it obtained subsidies at provincial and national level, and ensured the support of
educational and governmental institutions. That was made possible by the direct
involvement of some members of the public administration in the activities. Moreover,
the Lab was clearly linked with the concepts of Smart City and social innovation which
made the public value more tangible. At the same time, SLL receives economic
support by several companies that take advantage of the experience, competences
and unique setting of SLL to co-create innovative products. The development of better
products is already a direct demonstration of economic and business value. To its
business partners, SLL offers different collaboration and payment options based on
the maturity of the firm and the objective of the product or service to be developed

Amsterdam Lab

Context In the last years, the municipality of Amsterdam started an ambitious plan of
reorganization with the objective to operate in a “locality-oriented” manner, with a
prominent focus on social innovation. Amsterdam University of Applied Sciences
(AUAS), together with the city council, has engaged a creative process to address
diffused social issues. With this purpose, AUAS as part of its department of Urban
Management developed local innovation environments, called “field labs”. In the
period 2012-2016 three field labs were established in different neighbourhood of the
city: Amsterdam Nieuw-West, Amsterdam Oost and Amsterdam Zuidoost.

Description The three fieldlabs realized several projects with the objective of gaining
fundamental knowledge for the solutions of recurrent social problems. The
neighbourhood is seen as a local level playing field formed by several already present

actors with their own needs and interests, which get together to translate complex
urban issues in context-specific solutions.

Challenge The mission of AUAS fieldlabs is to deliver new governance arrangements

and social policies which enable the development of context specific solutions to
complex social, sustainability and economic issues that are faced in the city.

The Network AUAS provides an interesting distinction of the main stakeholders

between strategic partners and project partners. The project partners participate only
to one or few specific initiatives of the fieldlabs while the strategic partners also share
the organization of the network of the fieldlabs at a higher level.

• Educational institutions: the AUAS is the owner of the fieldlabs activities and main
educational institute involved. Professors, researchers and students from several
departments of the university are involved with different roles and responsibilities.

• Governmental institutions: the fieldlabs involve primarily two layers of

governmental institutions. The municipality of Amsterdam is the main strategic
partner and financer, while the local administrations of the three neighbourhoods
that host the fieldlabs are involved in a more operational manner.
• Business: there is a limited number of business involved, mainly as project
partners. Only big social associations and utility companies, such as social
housing companies, are interested in the knowledge valorised by AUAS fieldlabs.

• Citizens: a sample of citizens is involved in every initiative of the fieldlabs. They’re

empowered of the role of co-creators of the social innovations and policies
resulting from the activities of the fieldlab. Often citizens are involved as
associations which are extremely important project partners: in fact, associations
represent the needs and perspectives of specific categories of citizens and user
groups in a structured manner.

Outcome AUAS fieldlabs allow the application and valorisation of previous knowledge
in real-life setting, the development of new knowledge and its combination with
different sources. Knowledge can be used by institutions for the development of
services and policies.

Value creation Because of the primary interest of the fieldlabs in generating

knowledge for the development of social policies, it becomes hard to produce easily
sellable solutions. Accordingly, the fieldlabs mainly create public value in the form of
knowledge and awareness about diffused social issues and set the bases for future
improvements of the urban context. Therefore, the main financers of the labs are the

governmental and educational institutions which aren’t interested in the realization of
tangible products but rather in the implementation of innovative social policies. For the
fieldlabs it’s hard to obtain monetary support from private firms and small businesses.
Thus, it’s fundamental for AUAS fieldlabs to deliver and demonstrate the generated
public value. Accordingly, the process of building trust with every stakeholder is a
priority for the fieldlabs. To this extent, AUAS presents research results and concrete
suggestions for innovative policies to the municipality and to the University board.
AUAS uses a full set of tools and visual supports, such as Value Design Canvas, to
help every stakeholder to realize the outcome and potential benefits deriving from the
intended learning processes.

Funding and Revenue Every fieldlabs has its own financing scheme. The strategic
partners provide a shared budget: It means that no specific project is funded in
advance, but the budget must be allocated where needed. In addition to that, other
public organization or governmental funds contribute to the fieldlabs budget, while the
project partners finance specific initiatives. An ideal scheme of the itemized budget for
a single subproject would be financed in equal parts by four categories of actors:
AUAS, the urban district, the project partners, and project specific sources. At the
moment of research, the AUAS Lab was entering a transition phase to a new
positioning with the respect of the municipality of Amsterdam. In fact, thanks to the
good results obtained in the experiences of the fieldlabs, AUAS is gaining a stronger
position with the central administration of the municipality of Amsterdam.

Textile and Clothing Living Lab

Context TECLA represents one more component to the vivid innovation environment
of Palermo, one of South Italy’s biggest cities and crucial node of the Mediterranean
basin. The Living Lab started in the context of Horizon 2020 Textile & Clothing
Business Lab project. TECLA is hosted by Consorzio ARCA which manages a
university business incubator and has already ran other Living Labs in Sicily, such as,
the Solar Living Lab and Madonie Living Lab.

Description TECLA is an initiative of industrial (micro and small enterprises)

development to exploit the endogenous potential of Palermo region reinforcing the
innovative capacity of many actors to accelerate the creation of new user-centric
solutions. TECLA is a physical space that encourages to discuss ideas and projects,
meet partners, develop cooperation methodologies where textile and clothing
manufacture meets technologies and advanced multimedia tools. TECLA supports
the entire process and the actors involved developing co-creative models for the
design of new products and services and linking them with successful business

Challenge The main challenge for TECLA is to implement a systemic and integrated
approach to local sustainable and inclusive development to reduce the gap between
SMEs and research, stimulate entrepreneurship and enlarge the community of
artisans. The end objective of TECLA is to foster an integrated co-design approach to
produce a lasting impact on the territory.

Network TECLA can count on two main forces in the development of a solid network.
On the one hand, it can attract many diverse stakeholders closely linked to the theme
of the Living Lab, the textile. On the other hand, it can rely on ARCA and its
stakeholders from the quadruple helix.

• Governmental institutions and public organizations: TECLA inherit the innovation

ecosystem of ARCA and thus can count on the collaboration of policy makers at
regional and municipal level, local development agencies and NGOs. Moreover,
TECLA physical location is inside the community of Cre.Zi. Plus which gathers a
broad ecosystem of cultural foundations and public associations. TECLA is also
part of TCBL network of labs across Europe.

• Educational institutions: The Consorzio ARCA itself is a partnership between

business group and the University of Palermo. In TECLA are involved several
professors, researchers and students with different roles and tasks. Minor
research institutions are also involved.

• Businesses: TECLA has a strong focus on establishing a concrete partnership

across the entire supply chain. To this extent, it involves SMEs and start-ups not
only from the textile industry, but also in complementary fields, from IT to design.
Family workers and traditional artisans also have a big part in the innovation

• Citizens: The citizens are actively engaged in the activities of the Living Lab. User
groups, potential entrepreneurs, fashion amateurs, young creatives, members of
cultural associations are involved in the activities of the Living Lab through
workshops and seminaries.

Outcome TECLA supports mutual learning and re-connects the capacities and
knowledge of old artisans and fashionistas with the one of the players of the global
supply chain, professionals in technology fields, design and fashion students and new
entrepreneurs. Thanks to the collaboration of multiple stakeholders and the use of
common sophisticated assets the opportunity for new businesses can emerge. Thus,

new employment opportunities and an improved community of artisans can have a
positive impact in the region.

Value creation TECLA conveys economic value to private businesses which can
ideate better products and services based on new knowledge and advanced
technologies. Also, it delivers business value to companies which not only earn new
competences, a new co-creation approach and use shared assets, but also improve
their business model. Private companies also get the support of TECLA and create
synergies to apply to European programs and financing. Educational institutions
become more integrated in the local textile industry and provide students with
opportunities for traineeship. Public value is also a main interest for TECLA. There,
key social players and innovators are involved to generate environmentally and
socially sustainable products, services and processes which fit crucial societal
challenges. Moreover, Palermo region suffers of endemic unemployment problems
and TECLA’s projects can result in new start-ups.

Funding and revenue The Sicilian Living Lab whose funding is from Horizon 2020
TCBL project is located in a territory without large companies and where the public
financiers, present in other territorial contexts, don’t always respond quickly to the
challenges of territorial innovation. Therefore, the sustainability of the Lab necessarily
passes through a funding mix given by daily activities, call to action of all the
components of the ecosystem and EC funding. TECLA can count on the capacity of
ARCA in fund raising, both through its investor network. To demonstrate the public
value created TECLA wants to develop a common language and shared
understanding of the potential of the Living Lab with its stakeholders, and make explicit
these shared values to ensure the support of the main institutional partners. Moreover,
TECLA wants to assess the outcome in a measurable way based on traceability of the
process, the quality and effectiveness and the relevance of the multi-actor target. The
business value created can be accounted and made explicit through ARCA’s Open
Innovation platform, in which the several partners can be put in the position to generate
new links and partnerships, start a new venture or sign R&D contracts and license
agreements. The development of better product and services is already a direct
demonstration of the economic value generated. Among the other revenue streams
on which TECLA can count, it’s possible to notice that there is the plan to develop a
catalogue of services such as support packages for innovative SMEs, technology
transfer methods, access to EU finance, training modules for entrepreneurs,
internationalization etc.

6 Observations and Discussion

The two cases from the Netherlands already operated for several years. From their
experience it’s possible to obtain valuable insights on the relationship between Living
Lab’s activities, innovation network, value creation and the resulting business model
that ensures long term sustainability. These insights are then compared with the
strategy developed by TECLA which is still in a start-up phase.
The observations of SLL and the AUAS fieldlabs propose two different declinations of
the concept of Living Lab and introduce two prevailing orientations. On one extreme,
SLL can be seen as a product-oriented Living Lab in which the outcome is a set of
sellable products and services co-created with other companies from a specific
industry. On the other extreme, AUAS fieldlabs can be seen as policy-oriented Living
Labs in which the outcome is a set of policies co-created with the citizens. The
difference between these two models isn’t limited to the outcome, but the entire
structure of the Living Lab reflects the diverse orientation. In the product-oriented
Living Lab the innovation network is highly dynamic and business driven while the user
involvement is mainly passive. The focus is on the creation of economic and business
value, while public value is more an intended effect rather than a scope. There, the
economic and business value is easily demonstrated to the stakeholders which
guarantee the sustainability of the Living Lab through direct revenue streams. In the
policy-oriented Living Lab the network is more static and institutions driven, while the
users are actively involved. The focus is on the creation of public value, while
economic and business value is hard to seize. Building mutual trust with key
stakeholders is the main option for demonstrating the public value created and ensure
adequate funds to the Living Lab.

TECLA represents a promising trade-off between product-orientation and policy-

orientation. Indeed, TECLA developed its model as a combination of these elements
that can ensure sustainability in a characteristic context like Palermo. To this extent,
TECLA identified a particular field of application, the textile industry, which sees the
presence of a consistent number of diverse actors that still haven’t established
important synergies, and where the potential impact of new technologies and
innovative business models is still to be exploited. In this context, TECLA aims at
enabling the realization of an extremely wide set of sellable products and services,
which can translate in substantial revenue streams. At the same time, Palermo region
has a strong determination to support initiatives that foster entrepreneurship and
employment, valorize traditional craftmanship and spread concepts such as open
innovation and co-creation across the territory. Consequently, TECLA will be able to
count on the support of public institutions for additional funds.

7 Results

Analysing the observations collected in the current case study, it’s possible to obtain
several meaningful insights which answer the research questions. The two
approaches presented in the previous section aren’t mutually exclusive but rather two
extreme positions on the same continuum. Accordingly, the business model which
gives the best chances to ensure the necessary funding should be designed starting
from the choice of the most suitable positioning in such a continuum. Indeed,
determining the right trade-off between product orientation and policy orientation is
fundamental to design a Living Lab that can effectively pursue long term sustainability.
The adequate trade-off should be determined, not only based on the objective and
intended outcome, but especially on the context of application and the surrounding
network. In fact, long-term viability is enabled by the funds and revenue streams that
can be obtained from the several actors constituting the innovation network. From the
case study emerged that the recurrent stakeholders for a Living Lab can be divided in
four macro-categories: Governmental Institutions, Educational Institutions,
Businesses and Citizens. Two of them are particularly relevant to the purpose of
financial sustainability: on the one hand, the presence of a vivid network of businesses
and private firms in a specific industry gives the possibility to a Living Lab to focus on
the co-creation of new products. On the other hand, if the public institutions share the
innovative vision of the Living Lab then it is easier to obtain the needed support for the
development of policies and public services. Indeed, based on an exhaustive network
analysis a Living Lab should identify the core outcome, and thus the value, to be
produced. Finally, the case study provides concrete testimonies on how Living Labs
create value (Economic, Business and Public) and how they succeed in delivering it
across their innovation network. Indeed, Living Lab should systematically link its
outcome and created value with the way of making it tangible to every stakeholder so
that an adequate financial return can be ensured.

8 Conclusions

This research suggests that the choice of a suitable business model depends from the
context of application: Living Labs with a prevailing orientation towards the product
have higher chances to meet sustainability when the chosen theme is central also in
the local environment, and thus the Living Lab can count on a large and dynamic
network of businesses. In this case, the stakeholders give the necessary financial
support when they are capable to seize the Economic and Business value created.
Alternatively, more policy-oriented Living Labs require the support of local public
institutions. Thus, a policy-oriented Living Lab has better chances to meet
sustainability if concepts like social innovation and policy co-development are part of

the strategies of public administration. In this case, it’s fundamental to develop trust
and a stable relationship with the most strategic stakeholders. Moreover, availability
of dedicated public funds and access to European programs and subsidies is another
crucial driver for the success of a policy-oriented Living Lab.

9 Limitations and future work

The results present a promising research stream and a novel approach to Living Lab’s
sustainability, but they are far from being reliable, generalizable and transferable.
Therefore, these results require a further validation: the dualism between product- and
policy-orientation is inferred by the analysis of only three cases and, to pursue
generality and develop an adequate theory, these insights need to be tested on a
larger sample of Living Labs. Only via broader mapping it can be eventually possible
to apply these findings in different thematic sectors of Living Lab. Moreover, this
research was mainly based on qualitative methods, while the employment of
quantitative techniques might result in more consistent and robust sets of theories.


Almirall, E., & Wareham, J. (2008) Living Labs and Open Innovation: Roles and
Applicability. Electronic Journal for Virtual Organizations and Networks. 10,
pp. 21–46.
Almirall, E., & Wareham, J. (2011) Living Labs: Arbiters of Mid- and Ground-Level
Innovation. Technology Analysis & Strategic Management, 23(1), pp. 87-102.
Baccarne, B., Mechant, P., Schuurman, D. (2014) Empowered Cities? An Analysis of
the Structure and Generated Value of the Smart City Ghent. In R. P.
Rosenthal-Sabroux, Smart City, Progress in IS. Springer International
Publishing, Switzerland .
Bergvall-Kåreborn, B., Ihlström Eriksson, C., Ståhlbröst, A., Svensson, J. (2009) A
Milieu for Innovation: Defining Living Labs. Proceedings of the 2nd ISPIM
Innovation Symposium: Simulating Recovery - The Role of Innovation
Management. New York City.
Cosgrave, E., Tryphonas, T. (2012) Exploring the Relationship between Smart City
Policy and Implementation. Proceeding of Smart 2012: The First International
Conference On Smart Systems, Devices And Technologies. Stuttgart,
Eskelinen, J., Garcia Robles, A., Lindy, I., Marsh, J., Muente-Kunigami, A.
(2015). Citizen-Driven Innovation: A Guidebook for City Mayors and Public
Administrators. World Bank, Washington, DC, and European Network of
Living Labs. © World Bank and ENoLL.
Feurstein, K., Hesmer, A., Hribernik, K. A., Thoben, K. D., Schumacher, J. (2008)
Living Labs – A New Development Strategy. In Schumacher, J. & Niitamo,
V.P. European Living Labs – A New Approach for Human Centric Regional
Innovation. Wissenschaftlicher, Berlin.
Franz, Y. (2014) Chances and Challenges for Social Urban Living Labs in Urban
Research. Conference Proceedings of Open Living Lab Days 2014, pp. 105-
Katzy, B. (2012) Designing Viable Business Models for Living Labs. Journal of
Organizational Virtualness. 2(9), pp. 19-24.
Leminen, S., Westerlund, M., & Kortelainen, M. J. (2012) A Recipe for Innovation
Through Living Lab Networks. Proceedings of ISPIM Conferences
Mastelic, J., Sahakian, M., Bonazzi, (2015) How to Keep a Living Lab Alive?
info®,17(4), pp. 12-25
Mention, A.-L., Torkkeli, M. (2015) Open Innovation: A Multifaceted Perspective.
World Scientific Publishing Company Pte Limited.
Mulder, I. (2008) Living Methodologies: Understanding the Social Dynamics of
Innovation. In J. Schumacher, V. Niitano, European Living Labs – A New
Approach for Human Centric Regional Innovation, 6, pp. 31-38. Berlin:
Wissenschaftlicher Verlag.
Niitamo, V., Kulkki, S., Eriksson, M., & Hribernik, K. (2006) State-of-the-art and Good
Practice in the Field of Living Labs. Unpublished
Retrieved from:
Tukiainen, T., Leminen, S., Westerlund, M. (2015) Cities as Collaborative Innovation
Platforms. Technology Innovation Management Review, 5(10), pp. 16-23.

Veeckman, K., Schuurman, D., Leminen, S., Lievens, B., Westerlund, M. (2013)
Characteristics and Their Outcomes in Living Labs: A Flemish-Finnish Case
Study. Proceedings of the XXIV ISPIM Conference – Innovating in Global
Markets: Challenges for Sustainable Growth. Helsinki, Finland.
Westerlund, M., Leminen, S. (2011) Managing the Challenges of Becoming an Open
Innovation Company: Experiences from Living Labs. Technology Innovation
Management Review.
Yin, R. K. (2003). Case study research: Design and methods. Thousand Oaks, Calif:
Sage Publications.

Motivational Goals for using Electronic Health
Record Applications
1 2
Rachel Burrows , Sonja Pedell ,
2 1 1
Leon Sterling , Tim Miller , Antonette Mendoza

University of Melbourne, Australia
2 Swinburne University of Technology, Australia

Category: Research in-progress

The deployment of Electronic Health Records (EHRs) across Australia has provided
a platform of possibilities for new technology developers. However, the lack of
enthusiasm by multiple health-care providers and patients reveals how many existing
applications fail to address the needs of these stakeholders. We conducted interviews
with GPs and software company representatives (being partners of our Living Lab)
to understand their goals of engaging with EHR-enabled technology, with a focus on
their emotional goals. We then constructed a motivational goal model to represent
the technology goals that are relevant to our case study. This approach to gathering
technology requirements enabled us to elicit, organise and communicate the
functional, quality and emotional goals for EHR-enabled technology. Results illustrate
our living lab process and reflect on a variety of goals that may be supported by EHR-
enabled technology. The goal model will be evolved and utilised in the planning of
future phases of this living lab project.

Keywords: Goal Model, Living Lab Case Study, Electronic Health Record, Emotions

1 Introduction

The role of healthcare providers as ‘information gatekeepers’ is changing (Bomba,

2001). The ever-widening access to our own health information brings a shift in power
from health care providers to their patients. These changes are often met with great
resistance (McGinn, 2011; Barrett, 2018). One particular shift in power has
accompanied the deployment of Electronic Health Records (EHRs), which has
brought new ways for patients to access their health information online. This new
dynamic---i.e. people being able to access their health information at will---provides
a platform of possibilities for novel telehealth applications. A broader question
surrounding EHRs is therefore: How can we utilise this platform in order to improve
how people manage their own health?

Significant governmental funds are continually being directed towards the

deployment and appropriation of EHRs into health care facilities. However, many
stakeholders have concerns about using such technology; especially, within a
traditional GP consultation time slot (Linder, 2006; McGinn, 2011). The increased
pressures on health care facilities to adopt EHRs have been met with resistance on
part of health care providers and patients alike. In particular, emotional goals are
frequently overlooked during development; technology developers often focus on
functional goals (i.e. what a solution does) as opposed to quality and emotional goals
(i.e. how a technology solution should be and feel to the user) (Miller et al, 2014). The
emotional aspects affecting the uptake of EHRs are not thoroughly understood and
instead stakeholders are blamed for lack of engagement.

In this paper, we report on work-in-progress towards the design of an app that

supports patients to self-manage their health. This project is in association with the
Future Self and Design Living Lab and Elthi, a software company developing an app
to support people in self-managing their health. We conducted interviews with seven
GPs and two software developers to gain insights into their emotional perceptions
associated with EHR- enabled technology. The findings are then used to construct a
motivational goal model (extended form the work of Sterling and Taveter, 2009 and
Marshall, 2018) to represent the design challenges and complexities of this (and
similar) applications when considering the different living lab parties. The motivational
goal model represents the elicited functional, quality and emotional goals that need
to be incorporated in the design of EHR- enabled technology.

2 Background

The uptake of EHRs across Australia is faced with disparate challenges related to

social aspects, workflow, technical and professional issues (Linder, 2006). In spite of
the commendable goal of improving health and wellbeing, many believe that bringing
additional technology into the patient–GP consultation as a basis would result in the
natural conversation being interrupted (McGinn, 2011). In particular, the loss of eye
contact during a consultation may negatively impact the conversation. Similarly, some
patients feel that having a computer in front of them would be ‘rude’ (Linder, 2006).
Other issues affecting the uptake of EHRs include the introduction of further
inefficiencies into an already short consultation timeslot. For instance, there would be
a risk that the retrieval of relevant information may be too slow, or that the patient
themselves would spend too much time typing information (Linder, 2006; McGinn,
2011). A recent report outlines common goals that may help future technology design
overcome these barriers to engagement. These goals include improved
communication, improved health education for patients and, ultimately, improved
ability for patients to self-manage their health (AH&C Report, 2017). The focus of
existing work places great emphasis around a GP consultation, yet these goals do
not necessarily need to be supported from within a GP consultation. Instead, EHR
applications may be used before, during or after a consultation which significantly
increases the possible number of use scenarios.

One major factor that influences engagement with new technology lies in the
emotional goals of the stakeholders (Norman, 2013). A number of existing techniques
can be used that allows ethnographers to elicit and represent emotional goals of
technology stakeholders in the form of a goal model (Marshall, 2018; Sterling, L., &
Taveter, K., 2009; Mendoza et al, 2014; Miller et. al, 2014, Pedell et al, 2017). Goal
models can subsequently be used throughout the design process to steer exploration,
experimentation and evaluation strategies (Lopez-Lorca et. al. 2014). In the context
of health technology, positive emotional experiences during visits to the clinic were
shown to visibly increase patient motivation to self-manage their health (AH&C
Report, 2017). Supporting positive emotional experiences is not easy. Patients
sometimes feel anxious because they cannot remember what they wanted to
communicate to their GP. These issues were frequently reported by elderly patients
who, as a result, felt stressed and dissatisfied with the overall consultation experience
(Bomba, 2001). In such cases, EHRs could provide an additional relevant information
to support a positive emotional experience.

More generally, the emotional perceptions of technology stakeholders need to be

thoroughly considered to ensure that engagement is sustained in the long-term and
an impact through technology can be perceived. Emotional perceptions may refer to
primary emotions (Ekman, 1992) such as the desire to have fun while playing a game
(e.g. Callele, 2006), while other emotional perceptions may refer to less-immediate
reflective emotions that rely on higher levels of reflective, cognitive processes
(Plutchik, 2003) such as the desire to feel trust in shared information (Saffarizadeh,

2017). Unfortunately, the emotional goals of EHR stakeholders are not fully
understood. This is at least partly due to the complexities surrounding the health care
system in Australia, and a separation between wellbeing, quality of life and health –
the latter being assessed at distinct points of time instead of being monitored long-
term. Like most countries, the Australian health care system is a socially- complex
problem domain with competing views on how to best support people in managing
their own health and even views that this should be left to health care professionals.

3 Case Study and Method

In this project we focus on My Health App. My Health App is an application owned by

a software development company. The aim of this application is to provide patients
with an improved ability to self-manage their health. Unfortunately, My Health App
faces similar challenges to other applications that aim to build on top of EHRs and
therefore the emotional goals of stakeholders need to be carefully considered during
design. Our aim is to alongside the software development company, health care
providers and patients during the full timeframe of this living lab project. By
negotiating all stakeholder views, we aim to design a solution that can reconcile
different points of view towards an integrated health care pathway. Currently, our
research is in the first phase of the process where activities aim to explore the
problem domain and emotions.

We interviewed nine participants and elicited their emotional perceptions related to

EHR technology. Each interview was semi-structured and lasted about 30 mins. GPs
were selected to represent a range of situations including urban and rural clinics,
single and multiple clinics, shorter and longer consultations, short-term versus longer
term patients. The two members of the software development company had a wealth
of knowledge working with their target user base over a number of years. We
discussed topics spanning: (1) their health- related activities, (2) what information
they would see as important inside and outside of a GP consultation, (3) how new
technology may improve health outcomes, and (4) how they would want it to feel. If
they had negative perceptions of EHRs, we asked for their reasons. Results were
analysed following a thematic analysis approach to find common themes in the data
(Braun and Clarke, 2006).

The results were used to construct a motivational goal model. The use of goal models
has been refined in a variety of applications over recent years. They contribute a
practical way of communicating (visually and verbally) the functional, quality and
emotional goals that need to be addressed in the design of new technology for
adoption. Further details of their applications, notations and construction processes

are available (Marshall, 2014; Sterling, L., & Taveter, K., 2009; Pedell et al, 2017;
Miller et al 2014). In short, the entities shown in goal models represent what a system
should do (parallelogram), how a system should be (cloud), and how a system should
feel (heart) These goals are associated with roles (person icon) of people in the
surrounding socio- technical system.

4 Results and Analysis

The interviews were transcribed and a goal model was constructed by two authors of
the paper. After discussion we merged results into a single agreed model. Figure 1
presents a high-level view of the goal model, while Figure 2 presents a more detailed
view of the functional goal to Improve Consultation Outcomes. This model will be
used to summarise and design challenges that were elicited in the interviews. It will
also be used to steer exploration, experimentation and evaluation strategies in
subsequent living lab activities.

The top-level functional goal was labelled as Improve Health and this encompasses
all goals that were elicited. Both patients and GPs aspired to achieve this goal and
they wished to feel in control, resilient and capable while doing so. The remaining
goals are presented in the following text. Based on the interviews, we divided the
core motivations for utilising EHRs into five main functional goals. These goals are:
(1) Access Health Data, (2) Improve Consultation Outcomes, (3) Understand Health
Situation (4) Stay Informed, and (5) Improve Health Care Provision across Australia.
Starting with the first functional goal on the top left (Figure 1); the ability to access
health data is something that GPs already have the capability to achieve, although it
is new to many patients. A participant from the software development company
advocated for patients to own and access their health data, regardless of its’ further
utilisation. It is possible that this goal could have been considered as part of a larger
goal; yet the need for patients to feel that they owned their own health data was
considered a significant goal in its’ own right. Creating this sense of ownership was
therefore added as an emotional goal to this functional goal.

The second goal is to improve consultation outcomes for both GPs and patients.
Further details of this goal are shown in Figure 2. Patients may wish to prepare
relevant information on their device prior to a consultation. Also, they may wish to
refer to their health information during a consultation to support communication.
Finally, they may wish to record advice and outcomes of their consultation so that
they can refer back to it afterwards.

Figure 1. Motivational Goal Model for My Health App

Face-to-face communication is important and it is valuable for a GP to hear patients

explain things ‘in their own words’. Therefore, there needs to be a balance on the role
that any new technology takes to ensure the communication within a consultation
remains effective. The need for information to be organised, accurate, relevant, be a
historical perspective and efficient are the elicited quality goals that are associated
with one sub- goal of support communication. Participants also referred to key

emotional goals; they wanted to feel organised, relaxed, professional and to not feel
perplexed. One GP discussed the opportunity of technology to support patients in
becoming more organised. If the information that is accessible via the application
supported existing conversations in an organised way without detracting from the
natural conversational flow then this could reduce the anxiety within a consultation.
This GP stated:

“it’s very confronting when someone comes and they are all over the
place” [...]

“They assume you have some magical power for you to decipher [their
medical problem without the relevant information]”


“The more organised a client is, the more organised I feel, and that makes
for a better consultation.”

Figure 2. Detailed view of one goal

Both GPs and the software company described their experiences with different
cohorts of patients. Based on these experiences they explained how their patients
seemed to desire different levels of responsibility when it comes to their health. For
those with lower health literacy they thought that they wanted to feel listened to and
for the GP to take greater responsibility for their health decisions. Other participants
proposed that patients wanted to feel empowered to make these decisions
themselves. Conflicting goals for software applications are not unusual, however, in
the realm of emotional goals, they are perhaps even more common due to the very
nature of reflective emotions belonging to individual people, rather than the
application. These goals will need to be validated in future interviews with patients

The third goal was for patients to better understand their current health situation. This
is a necessary step to improve the ability for them to self-manage their health. For a
patient, this could include the ability to explore their health information with or without
a specific question in mind. Visualising and discovering relationships in health data
is believed to be a core motivation for patients; Patients want to feel empowered while
navigating their information. While objective data (e.g. blood pressure measures) are
frequently cited as being important to decide on a specific treatment, the ‘soft
information’ often guides questions to provide the initial understanding of the problem.
Soft information may be mood, pain, lethargy, anxiety or energy levels and can help
the GP to structure and navigate the conversation towards an understanding of the
problem. One GP stated: “‘How is your mood?’ That is my most highly weighted

Next, participants noted that it would be useful for patients to stay informed about
important health events and information. While staying informed, participants
discussed an underlying motivation of feeling connected to others while sharing
health news. They may wish to share health information with others, including friends
or family. Consequently, an appropriate level of privacy will need to be carefully
considered in the next phase of the project. Finally, the broader goal to improve health
care provision was articulated by the software company themselves. Achieving this
goal may involve contributing to research in health care and contributing new
technological solutions for the Australian Healthcare Service.

5 Conclusion

This paper presents an emotion-led exploration of the design challenges for new
technology that utilises Electronic Health Records. The aim is to understand the
emotional goals of EHR-enabled technology. We conducted a series of interviews

with GPs and representatives from a software company. Results from the interviews
were analysed to construct a motivational goal model that structures the elicited goals
that should be considered throughout the design process. The model maps
emotional, quality and functional goals in order to represent this information visually.
The model outlines five overarching goals. This approach to gathering technology
requirements enabled us to elicit, organise and communicate the functional, quality
and emotional goals effectively.

The five high-level goals elicited were: (1) Access Health Information, (2) Improve
Consultation Outcomes, (3) Understand Health Situation, (4) Stay Informed and (5)
Improve Health Care Provision across Australia. For each of these functional goals
we described the emotional and quality goals that need to be considered during the
technology design process. Emotional goals included the need to feel in control,
organised and to have a sense of ownership, amongst others. We propose that
realising these emotional goals will lead to shifts in the health care system as patients
will get a better understanding of the impact of their actions on their own health. It is
possible to see how certain emotional goals span many of the desired functionalities.
In other cases, it is possible to see that some emotional goals are only associated
with one isolated aspect of functionality. The goal model will be evolved and utilised
in the planning of future phases of this living lab project and their main stakeholder.

6 Acknowledgements

This research has been conducted in collaboration with Nigel Ball, Founder of Elthi
Solutions and funded by the Australian Research Council Discovery Grant
DP160104083 Catering for individuals’ emotions in technology development.


AH&C (2017). Access Health & Community. Patient Information and Data Flow.
Barrett, A. K. (2018). Electronic Health Record (EHR) Organizational Change:
Explaining Resistance Through Profession, Organizational Experience, and
EHR Communication Quality. Health communication, 33(4), 496-506.
Beirao, G., Den Ambtman, A., De Pourcq, K., De Regge, M., Simões do Carmo
Dias, J., & Kandampully, J. (2016). The healthcare experience: the
supportive role of technology and people in the elderly care sector. In
Frontiers in Service 2016.
Bomba, D. & de Silva, Ashley.(2001). An Australian case study of patient attitudes
towards the use of computerised medical records and unique identifiers.
Studies in health technology and informatics, 84(Pt 2), 1430-1434.
Braun, V., & Clarke, V. (2006). Using thematic analysis in psychology. Qualitative
research in psychology, 3(2), 77-101.
Callele, D., Neufeld, E., & Schneider, K. (2006). Emotional requirements in video
games. 14th IEEE International Conference in Requirements Engineering,
(pp. 299-302).
Ekman, P., & Friesen, W. V. (1976). Measuring facial movement. Environmental
psychology and nonverbal behavior, 1(1), 56-75.
Likourezos, A., Chalfin, D. B., Murphy, D. G., Sommer, B., Darcy, K., & Davidson,
S. J. (2004). Physician and nurse satisfaction with an electronic medical
record system. Journal of Emergency Medicine, 27(4), 419-424.
Linder, J. A., Schnipper, J. L., Tsurikova, R., Melnikas, A. J., Volk, L. A., &
Middleton, B. (2006). Barriers to Electronic Health Record Use during Patient
Visits. AMIA Annual Symposium Proceedings, 2006, 499– 503.
Lopez-Lorca, A. A., Miller, T., Pedell, S., Mendoza, A., Keirnan, A., & Sterling, L.
(2014, November). One size doesn't fit all: diversifying the user using
personas and emotional scenarios. In Proceedings of the 6th International
Workshop on Social Software Engineering (pp. 25-32). ACM.
Marshall, James. Agent-based modelling of emotional goals in digital media design
projects. Innovative Methods, User-Friendly Tools, Coding, and Design
Approaches in People-Oriented Programming. IGI Global, 2018. 262-284.
Mendoza, A., Miller, T., Pedell, S., & Sterling, L. (2013). The role of users’ emotions
and associated quality goals on appropriation of systems: two case studies.
In 24th Australasian Conference on Information Systems.
Miller, T., Pedell, S., Mendoza, A., Keirnan, A., Sterling, L., & Lopez-Lorca, A. A.
(2014). Emotionally-driven models for people-oriented requirements
engineering: The case study of emergency systems. IEEE Transactions on
Software Engineering.
McGinn, C. A., Grenier, S., Duplantie, J., Shaw, N., Sicotte, C., Mathieu, L., &
Gagnon, M. P. (2011). Comparison of user groups' perspectives of barriers

and facilitators to implementing electronic health records: a systematic
review. BMC medicine, 9(1), 46.
Norman, D. (2013). The design of everyday things: Revised and expanded edition.
Basic Books (AZ).
Pedell, S., Keirnan, A., Priday, G., Miller, T., Mendoza, A., Lopez-Lorca, A., et al.
2017. Methods for Supporting Older Users in Communicating Their Emotions
at Different Phases of a Living Lab Project. Technology Innovation
Management Review, 7(2): 7-19.
Plutchik, R. (2003). Emotions and life: Perspectives from psychology, biology, and
evolution. American Psychological Association.
Saffarizadeh, K., Boodraj, M., & Alashoor, T. M. (2017). Conversational Assistants:
Investigating Privacy Concerns, Trust, and Self-Disclosure.
Sterling, L., & Taveter, K. (2009). The art of agent-oriented modeling. MIT Press.
West, V. L., Borland, D., & Hammond, W. E. (2014). Innovative information
visualization of electronic health record data: a systematic review. Journal of
the American Medical Informatics Association, 22(2), 330- 339.

Smart Campus: a route using 4G and 5G
to serve the Smart City
Esmat Mirzamany1 and Joe Barrett2

1JointInformation Systems Committee (Jisc)

2 Global Mobile Suppliers Association (GSA)

Category: Innovation Paper

Not provided

1 Smart Campus: a mini Smart City

“Smart City” has become a hot topic in recent years. While still in its infancy, the new
and extreme broadband connectivity and computing capacity unleashed by the next
generation (5G) mobile networks, is bringing the power of Smart City solutions to our
cities. The success is driven by the fact that Smart City and its services have the power
to support necessary utility functions in today’s cities, and also creating completely
new business models and value propositions while enhancing safety and comfort of
its users; both citizens and visitors. Based on Deloitte5, a city is smart when
investments in (i) human and social capital, (ii) traditional infrastructure and (iii)
disruptive technologies fuel sustainable economic growth and a high quality of life,
through sustainable management of natural resources, as well as through
participatory governance. They are other definitions for Smart City, however; one can
say a smart city is a city with more contented and involved citizens, thanks to the
digitalisation of different aspects of their life. During recent years there have been
several smart city projects, with different business models and planning. Nevertheless,
apart from a high cost of running such projects, one thing common to all of these
activities is a “use-case driven” nature of such activities. That means use cases
chosen for the smart city project make the foundation of its success and are the
essence of a viable business model. So, the ability to choose the best use cases with
the best technology solutions- given the circumstances- is the key to the success.


Looking at different Smart City activities, it seems behind most, if not all, is a university
working side by side with other stakeholders, if not leading it. Research groups in
universities play a significant role in driving innovation, however, the role of campus
as a living laboratory - where not only researchers but citizens, city officials, and
businesses can experiment with smart technologies that aim to make everyday
life more comfortable, enjoyable and sustainable – has been overlooked. Both city and
campus have their own goals, challenges, and stakeholders, yet they share a lot of
common issues. Citizens’ care and wellbeing, estate, energy, transport and mobility,
are just a few that can be named. Therefore, campus can be seen as a mini small city
- with lesser risks, cost and better mitigation plans- with its own security system, retail
outlets, waste collection, sport facilities, car parking, etc. That makes a campus an
ideal vehicle to trial smart city initiatives prior to scaling up and get insights about
users’ behaviour toward this transformation and the effectiveness of different

2 Problems of Today’s Smart City Projects

Some of the problems of smart city projects are:

• Cost: The cost of implementing new services or rectifying the existing ones is
high, making it difficult to justify any changes with medium to high risk.
• No one-size-fits-all solution: While cities all have more and less the same
elements and infrastructure, each may have different problems and/or priorities.
Therefore, there is no solution which can be applied to all. Each solution should
be customised and tailored for each city.
• Business Model: Most services applicable in the smart city concept lack a viable
business case to sustain them on a long run.
• Data Sharing: There is a problem with city systems communicating with each
other in an effective way.

3 The Proposed Solution

Living Labs are playing a significant role in providing a co-creation, user engagement,
test and experimentation facilities targeting innovation in many different domains such
as energy, media, mobility, healthcare, agrifood, etc. However, they are not present in
every city and cannot offer a big scale pilot facility for potential services. University
campuses can be seen as a mini small city - with lesser risks, cost and better mitigation
plans while they still can give a taste of++=1 hosting city priorities and problems. This

can make them as a good complement to the existing chain of all the stakeholders
involved in Smart City activities i.e., City governors, Living Lab6, Solution providers,
etc. While Campuses are rich in their similarities with cities, they lack the required
experience for such activities and dealing with all the stakeholders involved. The gap
can be filled in with Living Lab, bridging between city stakeholders and campus ones.
By piloting new services in the campus, not only the students’ campus life can improve,
the outcome can be used as a good indicator of success or failure of a service in a
large scale like city, making it easier to convince decision makers.

4 Synergy between Smart Campus and Smart City use-cases

Some potential use cases for Smart City which can be scaled up from Smart Campus
are listed as follows:
• Smart transport: While cities are a complex environment, campus environment
can provide a good testbed for testing intelligent transport systems, evaluating
their impact on solving density problem and helping users move more quickly,
cheaply, and efficiently. Intra-campus mobility can be used to:
§ Test autonomous buses without the official approvals and formal need to
have all the sensors and intelligent decision making to adapt to any road
traffic situation, but adapted for a restricted but useful case;
§ Trial different operation models to tackle rush-hour problem e.g., demand-
based operation using App vs fixed time/fixed route operation, different
payment models etc.
§ Explore the idea of moving away from mass transits by introducing and
assessing disruptive services such as Mobility-as-a-Service (MaaS) transit,
a system which relies on a digital platform that integrates end-to-end trip
planning, booking, electronic ticketing, and payment services across all
modes of transportation, public or private.

• Smart building: United Nations Environment Program estimated that buildings

consume 40 % of global energy7. In England alone, annual energy costs for the
education sector total more than £200 million, resulting in the release of at least 3
million tonnes of carbon dioxide (CO2) into the atmosphere. Annual energy costs
per institution range from under £200k to over £3 million, and generally account


for around 25 % of building-related revenue spend8. This is a sizeable portion of
any estates budget. It is also potentially one of the most controllable cost. Green
Energy Ensemble estimated that smart buildings save 40 % energy and reduce
building operational costs up to 30 %9. Smart building can help minimise the
manual check and maintenance overhead of various systems of the building. This
can include fire, smoke, HVAC (heating, ventilation and air condition), elevators,
and other facilities which need periodic checks and maintenance. Capabilities
such as fine-grained temperature control, auto tinting of windows for optimal
lighting, finding empty desk in library or empty class rooms by using location-
based services in the buildings, are just a few examples of how smart building
could help improve student and employee focus and productivity, while reducing
the operation cost and improving the space efficiency. Such solutions can be
tested and tuned through series of trials in the campus and then be scaled up.

• Smart parking: Both cities and campuses are suffering from the lack of enough
parking spaces, leading to an ever-pressing demand of reducing inefficiency in
parking systems and payment. In a campus, available solutions can be
benchmarked, new features can be tested and their impacts can be evaluated in
a small scale. Features such as multi-tier users e.g., showing last available spaces
only to staff rather than students, can be tested and refined in a campus before
being rolled out in a city.
• Smart water: Water system is one of a city's most important parts of critical
infrastructure, yet one of the oldest ones. Providing effective and economical
solutions for measuring the water quality, water loss management, leakage
detection, and predictive maintenance planning are just a few examples of how
camps can serve as a cost-effective testbed for this purpose.

• Big data analytics: Data, the lifeblood of smart solutions, is the “oil” of the 21st
century, and is considered as a viable asset within and beyond the actual smart
city opportunity. The challenge is how to use the power of data to create smart
solutions that meet real needs of city users. And for that, big data analytics has,
undeniably, a very important role in having a sustainable smart city/campus. In
smart campus/city data is created from people, systems and things, and comes
from various environments such as traffic, transport, water and sewage, energy,
and buildings, with minimal or no coordination. While data is a big asset, its
heterogeneity makes it challenging to discover, combine, organise, interpret,
consume and publish information. Campus, as a mini city, can provide a living
laboratory to try novel and sophisticated techniques and test their effectiveness.


• User education: The transformation from today’s cities to smart ones doesn’t just
happen overnight. Smart solutions should address needs of their users, be utilised
in their full potential and be perceived as useful. Only this way can they be adapted
naturally and stimulate changes in users’ behaviour. And that is the key for
success of Smart environment. Education of smart city users and raising
awareness can start from campuses. Students, as the most technology-friendly
citizens, can act as early adapters and a good advocate for smart services. They
can then educate their family and friends and raise awareness among them.
• Smart people: Disruptive technology and big data are gaining huge attention due
to their role in developing Smart City. However, they are just two out of three pillars
of any smart environment. The third one, and the most important one, is “smart
people”. These people are essential for a constant renewal of the economic
infrastructure through creative destruction and innovation. Education is critical for
development of talent that is motivated and enabled to drive innovation.
Tomorrow’s resources are in today’s classroom. Therefore, providing them with
an experience of living in a smart environment is a start of laying the foundation
for new initiatives, start-ups and a climate in which innovation can flourish.
• Crowd management: Crowd management - the ability to monitor/support and,
where necessary, direct a group of people to ensure their safety- is one of ever
growing needs in today’s societies, with some similar challenges found in
campuses. Enabling technologies can be assessed in campus to see how they
help move people to their destination more efficiently. The aggregated and
anonymous geo-location data, available through monitoring and analysing
crowds, can be used to identify how they move, how environments can be planned
around them, how to get some insights about crowd densities, travel patterns and
the group profile of the crowds travelling. It also can be used to identify the source
of any issue or unusual behaviour and try new features based on that. For
instance, new features to detect and localise an abnormal behaviour in crowd
scenes, such as detecting violent in a stadium, can be tested in campus
environment before being scaled up.
• Safe campus: The cornerstone of all smart developments is security. While data
leakage from different smart services used can cause serious concerns, smart
safe campus/city is not just about IT (information technology) and cyber security.
It is also about how security and emergency situations can be organised and dealt
in a smart way. Today’s technology can provide users with an app that can inform
authorities of an alert as and when needed. Using an intelligent work flow,
authorities can process an incident based on different priority levels and provide
voice and/or video communication with users i.e., a victim or witness with one
click. Depending on the incident category and location, the system can
automatically recommend the dispatch units to assign based on their skill sets,

location, availability, current work load, available assets and traffic conditions to
the person in need. Such services can be very effective in big campus, with the
ability to subsequently serve vulnerable citizens, elderly people, etc., in big cities.

• Wayfinding and Digital Signage: One of primarily outcomes of smart

city/campus is to improve communication with users. The ability to change all
signs through the central platform, not only can provide better user experience, it
can immensely help authorities in a case of security and emergency events.
Possibility of updating relevant signs based on path closures, events on a day,
emergency incidents etc., and providing what users need to know at the given
time and location can have a very positive impact on users’ day to day life.

• Experimenting the future of SIM: The arrival of embedded SIM (eSIM)- or more
accurately eUICC- will add a new level of flexibility to mobile access. eSIM will
remove the physical SIM and replace it with an embedded chip on the phone. This
will enable users to download and store different mobile operator profiles on their
phone and select the one they want to use without the need to change out the SIM
– although they can only use one profile at a time. This will enable smaller devices
such as smart watches, fitness bands and small IoT devices to be freed up of a
physical SIM and make connectivity of these devices much simpler. With eSIM,
new opportunities for the use of SIM cards and mobile devices arise, allowing
MNOs to customise their products for different customer groups. One example of
such opportunities is to install private business applications on the SIM. For
example, adding building access control, or cooperate Wi-Fi profile, are just few
examples of such applications. eSIM will also support access to a Neutral Host
Network in a stadium, event, campus or any location where a Neutral Host
Network is provided. A university site would be a good test bed for this use case
where only registered students could access the network and access rights for
entry to buildings, photocopying, and student services was managed by eSIM.
After testing in campus, the concept can be tailored for other communities, e.g.,
Heath services or facilities
• Heterogeneous connectivity: Similar to smart city solutions, smart campus
solutions contain usually both the “service” and “connectivity” parts, two domains
that need to be bridged based on the requirement of each service. As the range
of services and their connectivity requirements is very broad, there is no one-fit-
all solution. Smart Campus/City connectivity may come through a cluster of
cellular4G and 5G, Wi-Fi, and LPWA networks sites and all wireless technologies
will continue to evolve to meet the full needs of the smart city and smart users. A
Smart Campus can therefore provide a good testbed to try different combinations
before implementing them in a city-wide application.

5 Business case for 5G smart cities

One of the key questions around smart cities is the business case, and specifically,
where does the revenue come from to support the investment needed to build out the
city-wide 5G network. There are multiple service suggestions, but what is ideally
needed is a main use case that goes some way to answering the business base
revenue question and then a range of adjunct services that contribute to the overall
business case. If this could be established, there is no reason that a business
proposition could not be developed to stimulate investments and a suitable
commercial vehicle established to deliver on the smart city vision. What then could be
the key use cases?

It is generally agreed that areas such as traffic management and road traffic flow are
a key benefit of smart cities, but would drivers pay extra to avoid congestion and a
faster less stressful journey? One option is to not offer a choice. A congestion or
pollution zone charging scheme, already used in many cities around the world, could
indeed be applied to the Smart Cities that adopt a 5G infrastructure. Vehicles could
be charged a variable daily rate for entering the city area, the cost dependent on time
of day and type of vehicle – electric vehicles could be subject to reduced fees to enter
the city for example – and high polluting vehicles charged more. The smart city app in
the vehicle’s built-in 5G system or on the driver’s smart phone would suggest available
parking, and even reserve a spot for premium app users. Public car parks could be
automatic, based on number plate recognition, charging only for the time parked, etc.
This level of “city smartness” would require extensive video coverage from networked
cameras. As an only candid, this can be achieved over 5G – with AI and machine
learning to reduce the need for massive human intervention and the associated costs.
Once this video coverage is established, a number of supporting smart city services
become apparent.
• Smart city lighting: Using city video surveillance and motion detection it would be
possible to intelligently manage street lighting and traffic lights. There can be long
periods of time when street lighting is not needed – during the night for instance –
and street lighting could be dimmed or turned off by location based on need. Traffic
lights could automatically engage based on approaching traffic reducing energy
costs for the city. These services are predominantly night-time based and the 5G
mobile network would be less busy during these periods so gaining maximum use
out of the network at minimal cost.
§ A smart campus could also utilise the same smart lighting solution to reduce
energy costs.

• City road tax: An alternative to a congestion or pollution zone charge could be a
city road tax to cover the cost of maintaining the smart city network and supplement
the maintenance of highways. Vehicles, including electric cars – which may not pay
fuel tax – could be charged for the distance travelled in the Smart City. Different
rates could be applied to different types of vehicles, with the least polluting vehicles
being charged the least per km travelled.
• Night time security: An out of hours surveillance service could monitor movements
– people or vehicles – and if certain pre-defined criteria were met, video of the area
could be packaged up and sent to a central control centre for human analysis and
action as required. A mechanical digger entering the city at 2am for instance would
trigger the surveillance system since this would be an unusual event for this time of
night and could be an intended smash and grab – theft of a cashpoint for instance.
The cost of this service could be added to the business rates of those in the city
area protected and should actually have a positive impact on business insurance
rates so potentially negating the cost to any business.
§ A smart campus solution would probably be focused late at night covering
people’s movements and also consider noise – identifying a scream for
instance – as part of the incident monitoring criteria.
• City road block: Following on from the above surveillance service if a vehicle was
used to smash into a store and drive off late at night, bollards could be raised across
access streets trapping the vehicle in a defined zone.
• Parking compliance: Illegal parking can be a major headache for cities where a
vehicle like a delivery truck is parked outside of the agreed time period for deliveries.
This inconsiderate parking can cause traffic to back up. If a vehicle is illegally parked
a tiered parking fine can be instantly raised based on vehicle number recognition
and its owner can be informed via text, asking them to remove the vehicle to avoid
further charges. The service revenue here is quite clear.
§ A smart campus could also implement the same parking compliance service
without the need for expensive human monitoring and intervention.

• Residential security: The smart city video surveillance service could be expanded
to cover residential areas of the city providing a percentage of the street signed up
to an annual service contract. This would supplement any private house CCTV and
again could positively impact the cost of insurance.

Compliance was one of the interesting business case proposals that came out of
Mobile World Congress 2018 to fund the cost of smart cities. Whether it is charging
for travelled distance, excessive speeding, jumping red lights, vandalism, graffiti or
ant-social behaviour, using smart city video surveillance to instantly spot the

infringement and deliver an appropriate fine was suggested as a viable revenue

Delivering the smart city solution ideally needs a commercial partner, potentially via
a private public partnership or a standard commercial agreement with a private
enterprise building the network and sharing any revenue generated with the city.

A smart city video surveillance services should not be viewed as big brother in
current times of increased security threats and the rise in petty crimes that often go
unpunished due to a lack of policing resources. Technology, including 5G and AI
can improve city life considerably. The key issue is the vast amount of video data
that would be collected. The solution is to use AI to only save information that meets
specific incident criteria and then automatically save the video file as evidence of
the event for future use if needed – question over a speeding fine for instance.
Urgent incidents would be instantly brought to the attention of those that needed to
take any action.

Smart city business case conclusion

The business case for smart cities can be challenging and identifying how these
networks will be funded has delayed the wide introduction of smart city networks.
Traffic congestion and air pollution reduction could be central to the smart city revenue
plan with additional less profitable services complementing this revenue stream. Now
could be the time for those enterprises with the vision to recognise the business
potential of smart cities and the benefits such services can deliver to engage with the
broad smart city ecosystem to make this a reality and take benefit of using smart
campuses as a living laboratory for their ideas, if applicable.

6 Spectrum considerations for the Smart City/Campus

Mobile network operators (NMO) are usually the focus for providing a smart city
network since they usually have spectrum that would support efficient and affordable
4G or 5G technologies for the services envisaged and have the core network available
to manage the connectivity to multiple smart city devices or end points. Their ability to
deliver an Internet of Things (IoT) solution also suggests MNOs are ideally positioned
to deliver on the smart city/campus initiative.

However, if the main underlying revenue solution driving smart cities is video
surveillance that could require high quality imaging and massive bandwidth, new
spectrum and a 5G radio link along with massive MIMO may be the only solution.

MNOs could acquire new 5G spectrum as could other agencies who wanted to focus
on the 5G smart city opportunity.
The advantage of using current mobile spectrum, sub-6 GHz, is that the propagation
is generally good and the cell size results in fewer sites. Sub-1 GHz bands are even
better for demanding coverage requirements, but there is less bandwidth impacting
the peak data rates and the capacity at these lower mobile frequencies; so high
bandwidth video is probably not a use case for this band.
Considering the need for extreme throughput and the amount of data that could be
needed for smart city video it will also be necessary to consider the expected 26 – 28
GHz and 37 – 43.5 GHz ranges for smart city connectivity.
The main advantage is the large amounts of bandwidth that is available – well in
excess of 1 GHz in the early stages of 5G deployments.

In a campus situation large cells may not be an issue and this could be an ideal
proving ground for these bands.

When considering the design of a smart city network and what devices need to initially
be connected – video cameras, lighting and traffic lights for instance – many of these
are in what could be referred to as city canyons with clear line of sight to required
service points (CCTV cameras). Servicing these connections with a fixed wireless
access (FWA) point to multipoint 5G network in the mmWave bands using MIMO
antennas would potentially support the capacity requirements for smart city video.

7 Further considerations

• Smart city/campus is, in its nature, use-case driven. Having use-cases with
perceivable values is the foundation for its success and key to viable business
model. That makes the transformation toward smart environment evolutionary
rather than revolutionary; change will not occur overnight. Different use cases
need to be identified, their impacts on different parts should be assessed,
people should be educated and involved and their feedbacks need to loop back
in from very early stage of project.

• Sharing the ownership of a Smart City project with different stakeholders not
only can provide a unique aspect into issues, it can open to new ideas and
increase the likelihood of success. As different parties feel being part of the
project, they will try their best to avoid failure. Involving universities can also
help stimulate start-ups by providing incubators where new ideas can be
shared, and problems can be solved through collaboration between
entrepreneurs and students.

• While the foundation is the same, every city has its own priorities and
challenges. Attracting and retaining high-tech and creative talents who are
familiar with Smart City environment is very crucial to its existence. Today’s
students equipped with the experience of smart environment can be the best
resource for tomorrow’s need.
• NREN10 (National Research and Education Network) can play a significant role
in involving universities. NREN is a specialised Internet service provider
dedicated to supporting the needs of research and education communities
within a country and usually has peering with mobile operators and service
providers. It is usually distinguished by support for a high-speed backbone
network, often offering dedicated channels for individual research projects.
Apart from connectivity, most NRENs offer digital services to their members in
pursuit of the vision of having a digitally advanced education and research
community. Involving NRENs in smart campus/smart city activities can:
§ Provide the best platform to raise awareness among Education sector by
disseminating Smart Campus project’s outcomes through case studies,
conference presentations, workshop, etc. It also helps to explore and
identify the best possible use cases;
§ Help to utilise the concept of Network Slicing to meet the need of different
Smart Campus data traffic through having totally independent and isolated
virtual networks within NREN infrastructure, and providing different level
of latency, reliability, availability and security for each use case.
§ Facilitate an inter-country collaboration. Different NRENs across Europe
are interconnected with each other via GÉANT11. Together, GÉANT
connects over 50 million users at 10,000 institutions across Europe.
Operating at speeds of up to 500Gbps, and offering unrivalled
geographical coverage, GÉANT is considered the most advanced
research network in the world. Apart from connectivity, GEANT
Associations opens a door for a Pan-European deployment of successful
services through other NRENs, enabling collaboration on projects over
dedicated infrastructure.

10 National Research and Education Network



Not provided

Spectrum Analysis
Digital Arts-and-Culture Assets and the Future of
Co-Creating Decentralised Technologies
Olivier Zephir1, Soenke Zehle1, Olivier Buchheit1

1 Technoport SA, Luxemburg

Category: Research-in-progress

The rise of decentralized ledger (bookkeeping) systems that operate as public
transaction archives to store value records (such as payments in a cryptocurrency)
and maintain consensus about agreements also seems to promise solutions for the
problem of scaling trust-based decision-making processes. Once linked almost
exclusively to finance (“cryptocurrencies”), the “blockchain space” has become a
terrain of social and technological experimentation. While these changes are likely to
substantially transform the way, we organize our collective actions, few digital content
creators; artists, designers, and other actors not directly linked to processes of
technological innovation are currently involved. In this essay, we describe how such
a co-creative approach to such involvement can be developed through the public
prototyping of Spectrum, a blockchain-based tool for digital creators. What we found
is that such an approach offers opportunities to more fully comprehend value chains,
but also highlights need for co-creation approaches that cut across existing
audiences and bring in new actors. This matters because blockchain-based
strategies for technology design are about to transform the way we create, share,
and cooperate. Given the promise and potential stakes of shifting the creation of
digital arts-and-cultural assets to infrastructures based on such distributed ledger
technologies, it is crucial that artists and other digital creators engage with the design
both of these infrastructures and the systems of governance structuring their

Keywords: Digital arts-and-culture assets, blockchain, decentralised technologies,

co-creative design, open data

1 Introduction

In this paper, we engage with a specific technology development process, whose aim
is the creation of a digital assets catalogue that operates as a tool for artists as well
as tool of analysis, facilitating the comprehension of how actors across value chains
approach digital assets creation. Spectrum analysis describes an applied arts-and-
technology research project that aims to involve users in a concrete design process
to engage with questions and processes that in turn frame our (co-creative) activity
across micro-, meso- and macro-scales. Over a two-year period, we have worked
with Spectrum to offer exemplary engagement with this dynamic. Spectrum aims to
offer a blockchain-based digital assets catalogue for digital creative industries. After
several iterations it is now available for user testing. In these use scenarios, we
showcase the project’s history of software development and involve actors from
government, research, enterprise, and society in user testing. In this essay, we focus
on music as a well-developed use case with a strong commitment from many actors
across these four sectors; we have also identified the field of open cultural data as
exemplary transfer dynamic that raises additional issues involving the question of
continued enrichment of existing digital assets through co-created metadata. These
two complementary concerns reflect our interest in comprehending the way in which
blockchain-based approaches can transform entire value chains.

Through the design of specific tools, blockchain-based approaches might help us

address broader issues of trust in processes of tracing and attributing contributions
in co-creative and open innovation contexts (Shavit 2016). Our own research is
framed by related work across this space of social and technological experimentation.
While industry actors are already exploring the future of royalty collection (cf. URights,
a research initiative of IBM and the French collecting society SACEM), artist-coders
such as Imogen Heap have been among the pioneers in this space (Howard 2015)
and initiatives such as “Artists Re:Thinking the Blockchain” affirm the need to
broaden the range of people and perspectives involved in these experiments beyond
a small core of software developers (Catlow et al 2017). Explorations of value
systems in collaborative contexts serve as reminder that blockchain-based
approaches build on a much broader dynamic of peer-to-peer cultures and
collaborative economies (Pazaitis et al 2017).

1 Methodology

Spectrum Analysis

With Spectrum, a DLT (Distributed Ledger Technology) platform, a proof of concept

exists, a prototyping dynamic has been initiated and a validation phase has been

organized. In a next step, the aim is to elaborate a corresponding co-creation approach
with a focus on the involvement of artists / digital creators. What follows is an account
of how we approach this iteration of co-creation methods.

Project History

05/16 24 Hrs Electronic, Luxembourg - Rocklab - Cultural institution - From

awareness to education

11/16 Sonic Visions Panel presentation: Creativity + Technology = new

opportunities? Luxembourg

12/16 EU Creative Hubs network (Formalisation of initial digital assets life

cycles), Edinburgh Music - creators

04/17 10th edition of Luxembourg intellectual property day, Smart royalties topic,

05/17 24 Hrs Electronic demo, Luxembourg

09/17 1535 creative hub demo, Luxembourg

11/17 Sonic visions demo, Transforming DRM, from sound to blockchain,


02/18 Spectrum - A D-DAC (Decentralised Digital Assets Catalogue)

transforming digital assets administration, Luxembourg

05/18 24 Hrs Electronic demo, Luxembourg

06/18 Sonar Market Lab installation, Barcelona

Current status

Spectrum is an exploratory project that aims to test how blockchain-based

technologies can simplify the administration of digital creative assets. Spectrum is a
D-DAC (Decentralised Digital Assets Catalogue) that explores sustainable
alternatives for resolving digital media industry complexities at the various stages of
digital assets life cycle: creation, production and distribution, archiving and reuse.
The core aim of this project is to provide a technical test platform to digital creative
industries players allowing them:
1. To identify and validate candidate blockchain fields of use;
2. To get an acute understanding of whether (and if so, how) their products and
related technical as well business operations can benefit from an added value with
the Blockchain technology (on the basis of relevant minimal testing).

Focus areas are:
• Digital assets
• User community value exchange
• Blockchain based operations benchmarks
• Public and private permission blockchains
• Smart contracts
• Tokens

At the current stage, the Spectrum “D-DAC Decentralised Digital Assets Catalogue” is
a consolidation of players in the Luxembourg blockchain ecosystem, and the test use-
cases are being developed in the digital entertainment and creative industries across
Europe. This collaborative initiative, supported by the Intrachain network and the
Digital Luxembourg initiative, is carried by:

• Technoport SA – a technology-oriented business incubator promoting and

supporting the development of innovative companies in Luxembourg
• Sonopraxis – a digital art & science consultancy company
• Bitbank – a Luxembourg Blockchain Start-up that builds decentralised
applications and provides consultancy services
• APLA – a blockchain platform for organising economic, public and social
activities through smart laws and smart contracts

Conceptual Issues: New Stakes, New Stakeholders

If these developments introduce new stakes, we also have new stakeholders, some
of whom are already engaging with these dynamics but many are not. The task for
co-creation approaches is fairly large - it ranges from new ways to “map” stakeholders
and value chains to finding ways to actually engage them across these value chains.

Blockchain is a technological enabler depending on a distributed governance model.

The design of governance model can and must be co-creative for facilitating
interoperability. When we speak of governance, we assume that we can comprehend
the point and process of creative production (example: Spectrum), but also that we
can follow (and track) the digital object throughout its lifecycle across complete value
chains. This extends far beyond a revival of digital rights management; instead, this
is a question of how we comprehend the operation (and governability) of digital

While technology development seems to just happen and it is all around us, one

12While such conceptual issues are beyond the methodological scope of this essay, we think it
important to address the question of digital objects in distributed environments. For the idea of an
“entropology” addressing the condition of distribution, see Rossiter & Zehle 2017.

question for co-creation methods is whether and how arts and culture can become
key drivers and even (re)claim the lead in tech development. Possible audiences
include: People who love blockchains. People who have no idea what a blockchain
is. People who are worried about blockchains. People who love music and digital
creation. People who care about how artists can make a living in digital economies.
Artists /Digital creative interested in co-creating technology. We want these different
audiences to meet and mix, and we need a co-creation approach that can work with
the differences and commonalities of these audiences.

Conceptual Issues: New Value Chains, New Data Models

- The Examples of Music

The internet has been made for digital data transfer along a community of
researchers; the network used by this community has been the seed for digital data
transfer at a worldwide scale. Nowadays, digital data becomes more and more
valuable, and Blockchain allows secure peer-to-peer transaction of digital asset. In
this global context, Blockchain appears to be a powerful tool for digital value
exchange in a secure and fully trusted way, providing the following technical
• Decentralised infrastructure, spread over a network;
• Peer-to-peer model for direct operation;
• Build-in trust;
• Resilience, high security;
• Historical conservation - capacity for audit trail and business intelligence.

The initial challenge for the digital media and entertainment industry players to exploit
decentralised technologies capabilities is to validate blockchain best candidate use-
case along collaborative value chains. The music industry is very dynamic in
embracing new technologies from creation to distribution. Significant DLT
(Decentralised Ledger Technologies) research for decentralised rights management
have been operated since 2015 (Fujimura 2015). This research focused on the
technological capability of adding the rights related information within a blockchain
transaction. On an operational perspective what still needs to be investigated is the
end to end value proposal of decentralised technologies for a musical digital asset.
Within the spectrum project we have mapped representative operational processes
involved in the generation of a digital asset in the music industry. We have designed
a musical digital assets life cycle to point out the different processes where
decentralised technologies can potentially bring an added value. We have illustrated
5 processes of musical digital assets generation, 1. Creation, 2. Production, 3.
Artwork, 4. Distribution and marketing, 5. Investments and revenue management.
Illustrated in the figure below are the activities where operating actors of the music

industry are expecting a facilitated process with decentralised digital assets
management technologies.
Creation process: Digital music creators are seeking for solutions to administer
rights repartition/recognition in the early phase of musical digital asset generation.
Ex. Collaborations across musicians, singers, vocalists, writers through smart
Production process: Recording professional music requires studio resources.
Music professionals are seeking for indirect facilitation from decentralised
technologies to leverage the necessary funds to book studio capabilities. The
expected solution by music processionals is to use the smart contracts to raise funds
under different modes; Love money, Crowdsourcing, label support.

Figure 1. Digital assets life cycle model and decentralised technologies impact (Technoport,

Assets & artworks: Musical recordings release impact is nowadays much related to
graphical and video related content. The expectations of music professional is to be
able to include artwork professionals in smart contracts for potential future share of
revenue. A kind of media for equity deal.
Distribution and marketing: The expectation of music professional is to be able to
use a smart contract and related content to facilitate organisation of concerts and for
example to deal real time revenue sharing on ticket sales.

Investment & revenue management: Music professionals’ expectations is heavily
related to the capabilities of Decentralised Digital Assets Management to facilitate fund
raising, royalties’ collections, direct retribution from unit sales activities.

2 Assessment / State of Research

With the spectrum project we have demonstrated how to operate the formalisation of
a digital asset at the early collaboration phases of creative digital asset generation.
Digital creatives can access and use a Decentralised Digital Assets Catalogue (D-
DAC) to register their creative works and trigger collaborations across the various
parties involved in the digital assets value chain. Although we have a robust
technological layer enabling Decentralised Digital Assets Management, we still need
to formalise and validate swift collaboration processes and related technological
interfaces to involve all the contributing parties of the digital asset life cycle to exploit
the full capability and benefits of decentralised technologies. Our latest development
scope of the spectrum project was to add the archiving activities and related private
and public parties involved.

On the basis of existing digital assets generation life cycle, various areas of trade,
financial related transactions supported by fintech technologies currently exist. From
the initial collaboration to co-create an asset to production and distribution there are
multiple existing trade touchpoints that are administered by rights/financial
uncorrelated third parties. Financial trade technologies of digital creative assets are
emerging and are facilitating the tack of digital creators to trade their creative assets,
request crowdfunding, monetize production. Operating these trade processes
requires much technological and human effort that can be simplified with
decentralised technologies. We believe that adopting a living-lab approach will
support us to generalise the use of Decentralised Digital Assets Catalogue across
existing collaboration processes and not only re-defining collaboration processes for
technology adoption. Industry standards such as DDEX in the music industry and
fintech industry standards such as FIBO (Financial Industry Business Ontology) have
shown how defining governance models of technological capabilities is relevant to
ensure technological adoption. Our next development step of the Spectrum project
will focus on the definition of governance models with the support of organisations
such as Infrachain in Luxembourg that supports community-driven governance for
operational Blockchains.

At Technoport, we have set with the Spectrum project the initial technological and
inclusive operational layers (integrating independent digital creative assets creators,
private production companies and content aggregators, public cultural heritage
archiving organisation and art & culture research organisation and academic bodies)

providing a test platform to address the mentioned challenges by using a
Decentralised Digital Assets Catalogue (D-DAC). In future collaborations (including
K8), This D-DAC based on DLT (Distributed Ledger Technology) principles will evolve
to reach an optimal fit to various categories of digital assets life cycles: creation,
production and distribution, archiving. We currently have the technical test platform
to invite digital creative industries players to test and co-design Prosumer
functionalities. This operated under a living lab mode will support the definition of
governance models for Decentralised Rights Administration under collaboratively
approved data ontology to reach shared standards.

3 Perspectives: From Creation to Co-Creation - The Example of

Open Cultural Data

To focus on the generation of digital creative assets is to focus on the moment of

creation itself. In all philosophies of art and culture, this moment is elusive, and we
tend to resort to ideas of inspiration, or the intensity of collaborative engagement that
somehow defies description. At the same time, the data-based processes of digital
creation provide a wide range of information. Leaving aside the question of whether
something like a “decisive moment” defining the moment of creative expression even
exists (and thus whether it can be attributed to a specific creator at all), this means
that someone (or some process) has to decide which of these data points “marks”
the moment in the flow of data which then becomes the reference for the digital object
- the version of the image, the movie, the song. However, all of this only refers to the
moment of creation.13

If we shift the focus of analysis toward the potential of co-creation, it matters to what
extent we can also consider moments of co-creation across the lifecycle of a digital
object, and how we inscribe the traces of (co-creative) use into the metadata of digital
objects. For the creation of arts and culture, this may or may not be a central
concern.14 But for arts and culture institutions, this is becoming a key issue - from the
co-curation of exhibitions to the integration of co-created metadata into the archive of
digital objects. And many of these are potential application fields for blockchain-based

13 In a 1952 collection of his works, the photographer Henri Cartier-Bresson famously defined the
“decisive moment” as follows: “To me, photography is the simultaneous recognition, in a fraction of a
second, of the significance of an event as well as of a precise organization of forms which give that
event its proper expression.” See Cartier-Bresson 2014.

14 In the realm of literature, co-created material is sometimes referred to as “fan fiction”, see Burt 2017;
in gaming, discussion has revolved around the roles (and possible remuneration) of “modders” (fan
coders), since mods extend the life of games and play a key role in the digital asset management
strategies of game publishers. For an early contribution to this ongoing controversy, see Kücklich 2005.


To explore these possibilities, it makes sense to move from mapping collections to

mapping the lifecycles of the digital objects in the archives that make up these
collections. Seemingly static terms like “storage” may be misleading, as is the idea of
the archive itself - unless we think of it as a living archive whose contents continue to
be enriched and linked throughout a lifetime of use.

Recent efforts to address this include a series of initiatives by Mediachain Labs to

create an “attribution engine” (a search engine for images with open licenses) and a
“social impact” cryptocurrency without any transaction fees for end-users.
Mediachain Labs offers a useful example because they started to address the key
role of users in value creation: “Users generate the value of these networks, but
individual users must cede ownership of their contributions, losing even basic
attribution as their work is shared and propagated. ... We believe that the only way
out of the incumbent-dominated internet is for emerging platforms to cooperate,
leveraging open data and decentralised applications to distribute the “network effect”
across a community of creators, users, developers and organizations.” Their solution
was the mediachain, “a collaborative federated media metadata protocol that allows
parties to make statements about creative works” (Nazarov 2016a).

Mediachain Labs also addressed the potential of open cultural data. They noted that
“[i]nstitutions generally publish their open data through their proprietary APIs or as
static data dumps on Github. The datasets live in disparate silos, unaware of each
other’s existence. ... Furthermore, fractured datasets lead to fractured APIs —
 developers hoping to use open data must accept the burden of interacting with
multiple incompatible APIs. This friction inhibits reuse” (Nazarov 2016b). So, for
“openness” to mean “lower thresholds for use and reuse”, appropriate data models
and APIs are needed. Instead, reuse tends to create even more silos: “If a developer
creates an application that generates valuable new metadata supplementing an
original dataset, it’s typically placed in a new silo since contributing it back to the
source presents technological and curatorial challenges” (ibid.). Aware of the painful
process of establishing new standards (such as JSON-ID), Mediachain’s open source
solution (based on the peer-to-peer hypermedia protocol IPFS) was not a new
standard but a “reconciliator layer” that automatically linked disparate data sets
describing the same object (ibid.). Mediachain Labs was acquired by Spotify in 2017
(following a series of lawsuits over unpaid royalties for many of the songs available
for streaming), and the team officially turned over code (and responsibility) to the
open source software community.

All of the issues defined in the process remain, and distributed ledger technologies
remain at the center of ongoing efforts. The focus on use and reuse complements

interest in digital creation. The trend toward open cultural data is likely to continue;
see, for example, Cultural Places, describing itself as “the first holistic platform in the
cultural sector that connects visitors, institutions, artists, content creators and donors”
(ibid.). But cross-platform solutions to the identification and sharing of data sets play
a key role as decentralised technologies have the potential to build on the work of
open cultural data pioneers (Sanderhoff 2014, Europeana 2018). Given the societal
stakes of new data models and data-driven strategies for the creation of value, it is
important to ensure that such developments are not simply considered issues of
technology design (Rouvroy 2015, MSI-Net 2018). Initiatives such as “Artists
Re:Thinking the Blockchain” highlight the urgency and richness of such
multidisciplinary engagement, but also the complexities of reclaiming even parts of
the terrain (and terminology) of technology development (Catlow et al 2017). But if
the rollout of distributed ledger technologies is to amount to more than a new
generation of digital rights managements tools, co-creative processes to foster the
emergence of a truly open web will have to be organized.

4 Conclusion and Outlook: SPOT or Truth in the Era of

Decentralised Technologies

The blockchain - both as the technology of distributed ledgers and the broader socio-
technological dynamic emerging around this infrastructural core - is about the internet
of value, facilitating transactions, applicable to a wide range of digital creative
activities, promising significant support to digital entrepreneurial activities. Open
“living lab” / prototyping methodologies to focus on the co-design of emerging
technological systems.

Only a few years old, the idea of the blockchain as a stand-alone technology (rather
than a data structure) is already undergoing conceptual and technological revision.
Several communities are working on “blockless” distributed ledgers, so we may not
even speak of “chains” in the near future. The issues remain, calling for artists and
digital creators more generally to engage with the co-design of the decentralised
technologies likely to shape their work in the future.

Current Focus in Co-Creation Future Focus in Co-Creation

non-commercial > commercial non-commercial <> commercial

linear business model development non-linear business model development

stakeholder identity defines stakeholder stakeholder action facilitates flexible
contribution attribution of roles

The search for the “single point of truth”, i.e. a comprehensive data model in digital
creation is on. How do we - artists / digital creators - engage with this process? Can
arts and culture create developments that are more than secondary to the dynamics
of finance? Along with the Digital Single Market EU directive on the topic and the
launch of a Blockchain Observatory and Forum, 22 European countries have already
signed a Declaration on the establishment of a European Blockchain Partnership.
This means that we also need to engage with this policy process that will soon frame
co-creative activities in this area of tech design. Getting involved in a concrete design
process offers us an opportunity to comprehend the stakes of this policy dynamic,
generate a better understanding of how such technological developments frame our
own activity, and explore how different trajectories of policy development affect our
ability to engage in such activity. Spectrum can contribute a case study and co-
creation approach to this process.


Burt, Stephanie (2017). “The Promise and Potential of Fan Fiction”, The New Yorker
(August 23).
potential-of-fan-fiction. Accessed July 15, 2018.
Catlow, Ruth; Garrett, Marc; Jones, Nathan; Skinner, Sam, eds. (2017). Artists
Re:thinking the Blockchain, Liverpool: Liverpool University Press.
Cartier-Bresson, Henri (2014). The Decisive Moment, Göttingen: Steidl.
Cultural Places. Accessed July 15, 2018
DAOstack (2018). “An Operating System for Collective Intelligence”. Whitepaper
v1.1. Accessed July 15,
DDEX Accessed July 15,
de Vries, Alex (2018). “Bitcoin's Growing Energy Problem”. Joule Volume 2, Issue 5,
801–805. Accessed July 15, 2018.
digital luxembourg Accessed July 15, 2018.
EU (2015). “Digital single market: Bringing down barriers to unlock online
opportunities”. European Commission. Accessed
July 15, 2018.
EU (2018a). “European Commission launches the EU Blockchain Observatory and
Forum”. European Commission (Feb 01).
release_IP-18-521_en.htm. Accessed July 15, 2018.
EU (2018b). “European countries join Blockchain Partnership”. European
Commission (April 10).
market/en/news/european-countries-join-blockchain-partnership. Accessed
July 15, 2018.
Europeana (2018). “‘A call to culture’ - Europeana 2020 Strategic update”. Accessed July 15, 2018.
FIBO (Financial Industry Business Ontology) Accessed July 15, 2018.
Field, Matan (2016). “Blockchain and a new paradigm of collectivity”. TEDx Talks
(12/05). Accessed July 15,
Fujimura, Shigeru & Watanabe, Hiroki & Nakadaira, Atsushi & Yamada, Tomokazu &
Akutsu, Akihito & Kishigami, Jay. (2015), “BRIGHT: A concept for a
decentralized rights management system based on blockchain”; 345-346.
Howard, George (2015). “Imogen Heap's Mycelia: An Artists' Approach for a Fair
Trade Music Business, Inspired by Blockchain”. Forbes (07/17).

blockchain/. Accessed July 15, 2018.
Infrachain Network Accessed July 15, 2018.
Kücklich, Julian (2005). “Precarious playbor: Modders and the digital games
industry”, The Fibreculture Journal 5,
precarious-playbour-modders-and-the-digital-games-industry. Accessed July
15, 2018.

Mediachain. Accessed July 15, 2018.

Mediachain Labs (2017). “The next chapter for Mediachain Labs”, Mediachain Blog
(April 26)
8aef3eedd729. Accessed July 15, 2018.
MSI-Net (2018). “Algorithms and Human Rights - Study on the human rights
dimension of automated data processing techniques and possible regulatory
implications”. Strasbourg: Council of Europe,
rights-a-new-study-has-been-published. Accessed July 15, 2018.
Nazarov, Denis (2016a). “Introducing Mediachain: Shifting the balance of network
effects with decentralized, open data”, Mediachain Blog (Jan 15), Accessed
July 15, 2018.
Nazarov, Denis (2016b). “Bringing Cultural Metadata to Life: How shared blockchain
infrastructure can foster innovation and realize the promise of open cultural
data”, Mediachain Blog (March 3),
metadata-to-life-12cc118b2298. Accessed July 15, 2018.
O’Dwyer, Rachel (2018). “Calculated Risk:
Ooki, Maki (2018). “IPFS and Ethereum ERC721 token Helps Claiming Ownership of
Digital Art”, (May 20).
Accessed July 15, 2018.
Pazaitis, Alex; De Filippi, Primavera, Kostaki, Vasilis (2017). “Blockchain and value
systems in the sharing economy: The illustrative case of Backfeed”.
Technological Forecasting and Social Change 125, 105-115.
Prvulovic, Mark (2018). “An Overview of the Different Blockchain Consensus
Algorithms”. kapitalized (06/16).
the-different-blockchain-consensus-algorithms. Accessed July 15, 2018.
Rossiter, Ned; Zehle, Soenke (2017). “The Experience of Digital Objects: Toward a
Speculative Entropology”, Spheres #3 Unstable Infrastructures,
speculative-entropology. Accessed July 15, 2018.

Rouvroy, Antoinette (2015). “‘Of Data and Men’ - Fundamental Rights and Freedoms
in a World of Big Data”, Strasbourg: Council of Europe,
?documentId=09000016806a6020. Accessed July 15, 2018.
Sanderhoff, Merete, ed. (2014). Sharing is Caring: Openness and sharing in the
cultural heritage sector. Kopenhagen: SMK.
smk/smks-publications/sharing-is-caring. Accessed July 15, 2018.
Shavit, David (2016). “Re-inventing co-creation with Blockchain”. Medium (02/16).
5268f1525e5f. Accessed July 15, 2018.
Skella, Jamie (2017). “A blockchain explanation your parents could understand”.
LinkedIn (06/02).
mum-could-understand-jamie-skella. Accessed July 15, 2018.
Zemel, Josh (2018). “An Explanation of DAOstack in Fairly Simple Terms“. Medium
simple-terms-d0e034739c5a. Accessed July 15, 2018.

The Austrian Real-World Laboratories -
How to manage multi-stakeholder engagement in the
wide field of mobility research
Lina Mosshammer1, Doris Wiederwald1

1AustriaTech GmbH Raimundgasse, Vienna

Category: Innovation Paper

Pursued by the Austrian Federal Ministry for Transport, Innovation and Technology,
the Urban Mobility Lab (UML) initiative was created in order to complement mobility
research with a Living Lab support structure to bring findings of research and
development into real-life practice. Regarding the Sustainable development goals the
laboratories help facilitating sustainable cities and communities (SDG 11) and foster
innovation (SDG9) in the field of mobility. Five Urban Mobility Labs are funded and
supported for four years, starting in 2017. Each UML has a specific regional trait -
from a port area, to a city district to whole cities and their surroundings. Topics
covered by the UMLs range from active mobility, passenger mobility, to freight
transport. The support of open innovation in a mobility living lab with a range of
services and thereby speeding up the innovation process requires the exchange and
coordination of multi- actor cooperation frameworks. The Urban Mobility Labs bring
together stakeholders from all different levels starting from the user level to regional
and national policy level. The close collaboration with local, regional and national
authorities to provide real-life test environments as well as to strategically steer and
embed projects in fast track innovation is one of the unique selling points of the

Keywords: living labs, mobility labs, co-creation, stakeholder engagement, public

partnerships, test environments

1 Introduction

Current transport challenges require innovative solutions as the demand is constantly

changing in terms of technological developments, organizational and legal aspects in
the society. Research often fails when it comes to putting their concepts into practice.
This is due lack of knowledge, acceptance, as well as embedding in specific regional
contexts and missing transition processes established. The mobility sector in general
is characterised by shared governance approaches, leading to a wide distribution of
tasks between public, semi- public and private actors, which poses a significant
challenge to bring research and development projects into real-life application.
Therefore, the UML in their mission to support innovation projects and bridging the
gap towards implementation need to pursue a multi-stakeholder approach in order to
provide and integrate all ideas, requirements and framework conditions in a timely
Pursued by the Austrian Federal Ministry for Transport, Innovation and Technology,
the Urban Mobility Lab (UML) initiative was created in order to complement mobility
research and innovation activities in Austria in this sense. During a first exploration
phase eight feasibility projects covering a variety of mobility topics had the chance to
gain practical experience and deliberatively test different approaches to generate a
broad range of experiences and insights for a potential Urban Mobility Lab in the
respective region. An accompanying study collected the different experiences from
the exploration-phase-projects and fed them into a structured pool of knowledge.

Figure 1. Austrian Mobility Labs (own representation)

Subsequently five Urban Mobility Labs were chosen for an implementation phase
funded for four years starting in 2017. A perspective of 10 years in operational terms
was required for each UML. Currently the UMLs are establishing real-world
laboratories in five different Austrian locations comprising districts, cities and regions.
Each laboratory sets several thematic foci (outlined in figure 1), aiming to facilitate
research projects in those fields by speeding up the implementation process. The
UMLs provide a toolkit of services to accompany research projects during the
innovation process and enhance their impact. As the concept of open innovation is a
core element of the living labs, a broad integration of diverse stakeholder and users
is strongly required.
The facilitation of an exchange and the coordination of multi-actor and multi-
stakeholder frameworks are therefore pursued. The Urban Mobility Labs bring
together stakeholders from different levels starting from the user level, to research,
enterprises and to public policy level. The close collaboration with local, regional and
national authorities allows strategic embedding and real-life testing of innovation
projects embedded in the UML/Living Lab environment. This allows bringing research
faster to implementation by providing test infrastructure not only in private
surroundings, but also in public space.
This paper sets a focus on the development and continuity of a successful multi-
stakeholder network in the field of mobility living labs with a special focus on
integration of public authorities and how two facilitate collaboration between all types
of stakeholders.

2 Organisational structure
The stakeholder involvement within the Urban Mobility Labs comprises different
levels of integration. They depict the intensity of cooperation, ranging from core team,
close partners to further contacts in UML-related fields and sectors.

Operative level
• Management team
• Core team

Strategic level
• Steering Committee
• Advisory Board

Additional contacts
• Further not formally established contacts
• Funding bodies

User level
• User groups

Figure 2. Organisational structure of the Urban Mobility Labs (own

As shown in Figure 2 on the operational level the management team is the core of
the Urban Mobility Labs and responsible for the coordination of all living lab activities
including the establishment and further development of a knowledge hub, open
innovation methods as well as services for research and development projects. This
organisational part is mostly covered by a management team from one institution.
The core team includes additional partners establishing the content of the UML. On
this level a variety of institutions closely work together in each lab, taking over various
responsibilities or work packages. The composition differs between the labs, as they
are facing different challenges regarding research topics and regional traits and
requirements. Partners within the teams of the UMLs are:

• Research institutions • Private companies

• Universities and colleges • Service provider
• Public transport providers • Public consulting firms
• Public authorities • Further partner

On a strategic level most UML include a steering committee that provide important
inputs regarding the embedding of the UMLs and their supported project in regional
and national strategies. The organisation of the knowledge exchange differs, but
usually takes place on a regular basis. Additionally, further advisors including interest
groups, experts, research etc. can give important inputs on this level and are
addressed in regular joint meetings or one-to-one contacts.
Further stakeholders are activated to complement the UML team in order to e.g. bring
in additional knowledge, provide further test environments or support in the transfer
of the outcomes. The collaboration is usually based on a Letter Of Interest (LOI).
Further contacts include a variety of stakeholders such as user groups or interest
groups. Funding bodies, notably the Austrian Ministry of Transport, Innovation and
Technology, enable the implementation of the Urban Mobility Labs facilitating the first
phase till the labs can work in a cost- covering manner.
As an example, the UML Salzburg’s organisational structure consists of an operative
and a strategic level. On the operative level a core group is composed of one main
management institution, the SIR Salzburg Research, which is responsible for the
organisational aspects and further institutions, including public authorities and
research companies/institutes, which offer functional and thematic input. The
strategic level includes a steering group consisting of the same institutions as the
core team overlooking the compliance of policy goals, as well as an advisory
committee comprising representatives of the federation of regional industry,
research, interest groups, political stakeholders, administration and further
stakeholders. (Salzburger Institut für Raumordnung und Wohnen, 2018)

Partner and stakeholder environment

As shown in Figure 2 the labs cannot cover all the services that may be needed to
support research projects along the full innovation cycle and to build up a laboratory
structure in the field of mobility. Therefore, stakeholders are a crucial part of the
Mobility Lab community. This is applicable to the structure of all five Urban Mobility
Labs. The co-operation can either be formally prepared by a Letter Of Interest (LOI)
or in a non- formal manner e.g. via different communication channels such as
newsletters and mailing lists. The degree of integration into the UML can vary
depending on the nature of cooperation. (Salzburger Institut für Raumordnung und
Wohnen, 2018).


SMEs Public
& institutions

Research L Local, regional

Mobility Interest
service groups

Figure 3. Partner network of the Urban Mobility Labs

(own representation)

The networks of the Urban Mobility Labs comprise a variety of stakeholders. They
are still growing and developing constantly. Figure 4 gives an oversight of the current
stakeholder groups involved in the five Urban Mobility Labs in Austria.

A strong network within the research community, including small research

institutions, offers access to a comprehensive pool of knowledge as well as the
possibility to jointly elaborate new research and innovation projects. Additionally, this
stakeholder group presents the main body submitting research projects that
presumably will be embedded in the UML structures.

Economic and industrial businesses can be valuable partners within research

projects by contributing an implementation-oriented view point. In many cases those
projects do not only focus on the prototyping and implementation phase, but also the
adaption of products on the market. Another important aspect of the cooperation is
the provision of infrastructure and real-life test environments. Small businesses and
start- ups are a specific group as they are especially flexible and innovation oriented.
They are usually part of innovative projects using the UML structures.

Regarding real-life test environments the Urban Mobility Labs are trying to provide
private as well as public space covering various traffic situations. Hence, they are
closely working together with partners offering different environments covering
testing areas such as for private commute and freight mobility solutions. Private
partners providing their infrastructure to specific laboratories within the initiative are
for example the Chemistry Park in Linz through the MobiLab OÖ and the Harbour
Vienna through Thinkport VIENNA (FH OÖ Forschungs & Entwicklungs GmbH, 2018)
(Universität für Bodenkultur Wien, 2018). Further on, Mobility service providers are
important partners when it comes to test environments. They can offer possibilities
for testing within the regular service framework or with additional infrastructure. They
often work closely with cities and regional authorities. This can be an advantage, as
some solutions need special permits.

One specific asset of the Urban Mobility Labs thereby is the strong direct
collaboration with public authorities. The need for innovation can directly be passed
on to the research community responding creating new projects meeting the demand.
Solutions and approaches potentially contribute to the regional and national mobility
goals. The Urban Mobility Labs can function as a bridge between different
stakeholders. This is also important when it comes to providing test environments in
public space. The Urban Mobility Labs are developing standardised steps to simplify
the process of getting necessary approvals from the different stakeholders.

In the sense of open innovation within the living labs it is also important to integrate
user groups and interest groups to enable sustainable research and to integrate
user needs in an early stage of the innovation cycle. The laboratories aim to include
users as innovators through co-creational methods.

2 Stakeholder Engagement Strategies

The development of a stakeholder network requires different methods targeting the

specific groups mentioned in point 2. On the one hand the UMLs want to raise their
awareness addressing stakeholder groups interested in embedding their research
projects and using the services of the laboratories. On the other hand, the UMLs need

to build up a network of stakeholder which is needed to create a successful living lab
environment. Therefore, each lab started with a bundle of measures to gain
awareness in their region and overall in Austria. Till now a wide range of
communication channels were introduced to address target groups (see Table 1).
To promote their laboratories each lab established a web presence including at least
a web page showing general information about the lab, embedded projects, events,
updates, contact information and their service portfolio. Some of the UMLs send out
newsletter and set up additional profiles on social media channels. Furthermore, the
UMLs created different promotion materials, mostly used on site, at events and
exhibitions. Media presence can improve the publicity of the UML, it is especially
important at relevant events and milestones. Press conferences were held by some
UML at the early stage as an official start of the lab.

To gain awareness within the stakeholder community, the participation in events and
conferences is an important activity. The labs to not only attend but also try to take
an active part by submitting papers, present their work, offer workshops or take a
stand in an exhibition often collaboratively with other Urban Mobility Labs. As an
example, the MobiLab OÖ and other UMLs went one step further and applied for a
membership within the EnoLL community (FH OÖ Forschungs & Entwicklungs
GmbH, 2018).

Another measure to exchange knowledge and make new contacts is a study visit.
The laboratories organise excursions to meet with other living labs or other relevant
institutions that represent important contacts. The UMLs also coordinate networking
events for their (LOI) stakeholders often targeting a specific topic.

To reach a broader audience and gain new contacts, the UMLs offer a variety of
networking events. As each UML is focussing on specific mobility topics and offer
diverse services, the direction and format of the happenings differs. As most events
can be of interest to one or more groups of stakeholders the events strongly rely on
collaborative methods. Examples will be described in chapter 4.

Access to test persons/users is a specific challenge to the UML as the integration of

users is an important open innovation aspect. On the one hand the labs contact this
stakeholder group on site offering specially designed events. The Aspern.mobil LAB
focusses for example especially on a neighbourhood level concentrating strongly on
user integration (Technische Universität Wien, 2018). Therefore, it offers regular
events in the district to build up a community for co-creational method and enabling
the citizens to take an active part in the innovative development of their surroundings.
On the other hand, some UMLs have access to data bases. The UML Salzburg offers
a possibility through a data base involving key actors that have access to test persons
(Salzburger Institut für Raumordnung und Wohnen, 2018.

Table 1. Communication Channels

Urban Mobility Lab Network

Communication Description

Web Communication
• Webpage
• Newsletter
• Twitter
• Facebook
Promotional material
• Flyer
• Sticker
• Brochures
• Roll Ups
• Goodies
Press and Media
• Press conference
• Articles
• Interviews
• TV appearances
Community Network
• Participation in events and conferences
• Presentations and workshops at events and conferences
• Publications
• Memberships in living lab communities e.g. EnoLL
• Study tours
• Events for LOI stakeholder

Network Events
• Participation in public events
• Events organised by the UML
§ Informative Events
§ Workshops
§ Lectures and discussion rounds
§ Events targeting a specific topic
§ Networking dialogues
Further Channels:
• University courses
(details see 3.2)
• Personal contact
• Advisory meetings (see 2.)
• Meeting places, display rooms, etc.
• Competitions and ideation challenges
• Mobile laboratory facilities
• Open innovation platforms
• User panels
• Hackathon with students

The Urban Mobility Lab initiative is accompanied by a so-called national contact point
at the AustriaTech (an agency of the Austrian Ministry for Transport, Innovation and
Technology -bmvit) which beside supporting the implementation phase of the
laboratories and the networking among each other, also represents a contact point

for information about the initiative and further ongoing activities of the Urban Mobility
Labs. This creates a link between all five Urban Mobility Labs in Austria and upcoming
Mobility Lab initiatives. The Urban Mobility Lab network is publicly represented by a
joint website. The contact point activates and coordinates submissions of technical
and scientific papers, presentations and expo stands at several conferences each
year. Furthermore, special match making events are organized bringing the
laboratories together with Austrian research projects with a potential to be embedded
in the UMLs. A special focus is set on research projects funded by the bmvit in order
to enhance the efficiency of national innovation projects and to speed up the
implementation process.
The supporting structure does not only aim to promote the initiative, but also to
accelerate the knowledge exchange between the Mobility Labs in Austria and further
laboratories. In addition, experts are invited on a regular basis providing knowledge
required such as insights into the development of user panels and data management.

Table 2. Supporting elements within the UML initiative

Supporting elements Description

• Austrian Mobility Labs homepage

• UML promotion material

• Providing information regarding the initiative

Contact Point
• UML hub

• Network meeting of the UMLs

Networking Events • Knowledge exchange between the UMLs
• Involvement of external experts and other living labs for joint learning

• Collective participation and presentation at conferences and other events

• Joint exhibitions at conferences and other events

Study visits • Collective study visits

Match Making Events • Special events to bring together the UMLs and Austrian research projects
• Special funding pool or preferential treatment for research projects
embedded in UML structures
Funding calls
• Participation of the UMLs at kick off events

Collaborative methods

A basic service of the Urban Mobility Labs is to bring stakeholders together. Besides
the support of knowledge and contact exchange, the main goal is to facilitate new
research projects to be embedded into the laboratories. As a result, a variety of co-
creational methods, applied in workshops and further innovative formats were
developed by the UML. The following list shows examples offered by the UMLs.

• Interactive Workshops focussing on co-creation methods such as (speed)
design thinking, design games, ideation workshops and cultural probe are
offered by the UML in all stages of the innovation cycle. The further
development of the offered methods is an important part of the UMLs work;

• Two UMLs offer mobile laboratories which can be built up wherever they
are needed offering space and equipment for co-creation in the field;

• By close collaboration with public authorities and private companies, test

environments can be offered to test prototypes in a real-life setting;

• Thinkport VIENNA offers a display room and two start-up offices,

complemented by several events with relevant stakeholders in the field of
freight transport. They also have an annual Forum called “Forum Green
Logistics”. (Universität für Bodenkultur Wien, 2018);

• The dissemination of projects and results through the communication

channels of the UML and specific stakeholder events offer a possibility to
attract stakeholder to collaborate within the research project;

• Some of the laboratories organise ideation workshops and challenges. As

an example, the MOBILITY LAB in Graz started a challenge looking for
creative mobility solution for the city region. The best projects will be
accompanied through the whole development process up until
implementation. (Holding Graz, 2018);

• Several universities and technical colleges are involved in the UMLs; they
started to offer courses and lectures to students working closely together
with the laboratories. The Aspern.mobil LAB also organised a hackathon for
students. (Technische Universität Wien, 2018);

• The UML Salzburg provides an open innovation platform as a design and

information hub for the region offering organized information about (technical)
services, documents and further data. (Salzburger Institut für Raumordnung
und Wohnen, 2018);

• Thematic excursions are offered for example by Aspern.mobil LAB to show

and analyse mobility infrastructure and public space to a variety of
stakeholder groups. (Technische Universität Wien, 2018);

• Technical tools are offered to simplify the integration of test persons. One
example is the PT feedback app offered by the UML Salzburg as a planning
tool which enables users to comment on used services.

3 Prospective Development and Future Vision

The wide spectrum of the transport community poses a great challenge regarding the
stakeholder engagement with the Urban Mobility Labs. It demands a transdisciplinary
approach elaborating different formats to address stakeholders for the laboratories.
In the frame of the support process provided by AustriaTech and the bmvit the UMLs
can exchange knowledge, learn from each other and work together if considered
beneficial. While UMLs share same challenges in terms of operating and supplying
living lab support structures, each UML also has specific characteristics in terms of
stakeholders and topics addressed. Therefore, the UMLs are trying to choose
engagement strategies suitable for their objectives. Each lab is called to monitor and
evaluate their implementation works and process towards the achievement of their
goals. Additionally, there will be a joint evaluation process conducted by an
independent party in 2020.

The support process provided to the UMLs creates a link between the laboratories
enabling them not only to represent their respective lab individually but to raise
awareness collaboratively. This joint network of the UML initiative is constantly
growing. National and international interested parties are contacting the UML contact
point to meet and collaborate with the labs. The UML initiative seems to create
interest, but there are still a lot of stakeholders which need to be addressed
specifically as the application of the living lab approach in urban mobility is relatively
new. The experiences indicate that a basic understanding of the Urban Mobility Labs
needs to be created.

While on UML level the development of a general stakeholder network is in progress,

further effort will be needed to establish and enlarge a specific stakeholder network
needed for targeted support of already embedded and future research and
development projects. The integration of the UMLs as a necessary or recommended
partner in the Austrian funding calls of the programme “Mobility of the Future” is an
important support element to encourage projects to make use of UML services. One
objective is to make the laboratories more tangible by showing the collaboration
between the research projects and the UML making the initiative more attractive for
future research and stakeholder engagement.

The salient asset of the UMLs for R&D projects which is often emphasised but maybe
not fully perceived yet is the close working relationship with public authorities. The
laboratories are still setting up the details of the services which can be offered as a
result of the cooperation. Integrating public authorities as stakeholders of the UMLs
might be a complex issue to varying policy goals, hierarchies and differing
responsibilities, but can be of mutual benefit by bringing mobility solutions responding
to actual needs into cities and open up planning process for disruptive innovations.

Looking into the future it will be important to further support the UML initiative and the
knowledge and experience exchange within Austria, Europe and beyond. This allows
the laboratories to improve management and stakeholder engagement and to further
develop the living lab approach for mobility challenges

AustriaTech . (2018, 05 15). Smart Mobility Webside- Mobilty Labs.
Retrieved from
FH OÖ Forschungs & Entwicklungs GmbH. (2018, 05 15). MobiLab -
Mobilitätslabor Oberösterreich. Retrieved from https://www.mobilab-
Holding Graz. (2018, 05 15). MOBILITY LAB- Stadtregion Graz bewegt.
Retrieved from Salzburger Institut für Raumordnung
und Wohnen. (2018, 05 15). UML Salzburg. Retrieved from
Technische Universität Wien. (2018, 05 15). Aspern.mobil LAB. Retrieved from Universität für Bodenkultur Wien. (2018, 05 15). thinkport
VIENNA. Retrieved from

The Developing Process of the Digital Game Which
Supports Well-being of Small Children
Päivi Marjanen1, Janne Pirttikoski,
Vesa Valkonen and Viktoria Tiainen

1 Laurea Universiy of Applied Sciences

Category: Research in-progress

The concept of well-being, both planet and individual, is clearly visible in the
Sustainable Development Goals. In this article the focus is on the individual's well-
being, more closely on children's well-being. In this article is described the developing
process of the digital game to evaluate and support the well-being of children aged 5-
8. The aim is to develop tool whereby subjective well-being data from small children
can be collected in a digital format, with a fun, child-friendly way. The description
follows the steps from idea to first prototype. Developers of the product consist actors
of children’s services, HEIs students and game designer from the private sector from
the metropolitan area of Finland. The ecosystem which works around the game idea
gives different kinds of knowledge, skills and test bed to develop a solution. The
designing process was carried out as a Research and Development activity at Laurea
University of Applied Sciences and it is part of the Finnish national LAPE project that
is part of the government's top program.

1 Introduction

The need to understand child well-being in a holistic way has been receiving growing
attention within policy, academic research and literature. Research can be divided to
those which focus more on deficit/vulnerability approach (Ben-Arieh and Goerge 2001;
Pollard and Lee 2003; Fattore et al. 2007) and those which focus on the developmental
approach such as human capital and social skills. The third one, child rights
perspective, emphasizes rights-based approaches to children as human beings and
thereby incorporating the direct input of the child in the process of determining what
their well-being might be and how it is best measured (Casas 1997; Ben-Arieh 2007).
Indicators do not however tell how children themselves describe their own well-being.
That understanding concept of well-being needs to take place within a context of
needs, rights and want (Marjanen, Ornellas & Mäntynen 2016). Even though this need
has been recognized in well-being of children research there are still not many
indicators to measure especially child’s will.

LAPE is Finnish government's key project which main aim is to develop services for
children and families. LAPE, services for children and family’s program, is a part of the
health, social services and regional government reform. Services for children and
families will be reformed 2016–2018.The main objectives of this reform aims to
transform the welfare services into an integrated system, strengthen basic services
and shift the focus towards preventive work and early support. (Program to address
child and family services 2017.) Basic services will be strengthened, and the focus will
be shifted towards preventive work and early support and care services. The best
interests of the child and support for parenthood are primary issues in the reform.
These changes will help to support children’s, young people’s and families’ own
resources and reduce inequalities.

Laurea University of Applied Sciences works as a partner in LAPE project. Almost all
municipalities in the Helsinki metropolitan area are involved in the project. The sub-
project in metropolitan area is extensive and over 30 separate working groups have
been started to reach the program goals. Laurea’s role in this project is to develop
digital tool which professionals who work with children can use to collect subjective
well-being information from small children age 5-8. This links to sustainable
development goals of UN promoting the well-being for all at all ages.

In this article is described the research based developing process of well-being

indicators. The process has happened in regional ecosystem which consists of
expertise from higher education but also from regional experts from public and private
sector. The development of scorecard is still ongoing. The next step is game
application which developing process has already started.

2 Subjective well-being of children

Defining well-being is not simple thing to do and there is no consensus among

scientists on what the definition of well-being should contain. It can be said that well-
being is a multidimensional phenomenon. The subjective approach refers to child's
own experience and assessment of their well-being and general social conditions.
Subjective well-being is defined, in some cases, as a synonym for happiness, life
satisfaction, or quality of life. For instance, self-realization and social relationships are
seen as elements of subjective well-being. (Konu 2002, 15-16; Diener, Oishi & Lucas
2009, 187; Veenhoven 2008, 45).

In the WHO definition, health is understood so broadly that this definition can also be
considered as a definition of well-being: "Health is a state of complete physical, mental
and social well-being and not just the absence of disease or infirmity." (WHO 1946).
This WHO definition covers the physical, mental and social well-being dimension.
Holistic, philosophical research perspective has given Schues and Rehmann-Sutter
(2013). They define the child's well-being in connection with the child's physical,
mental, personal, cultural and social development, which leads to meaningful life in
other people. A more structural but also societal approach is presented by Minkkinen
(2013), outlining the well-being of the child from the physical, psychological, social and
material situation. He emphasizes the positive elements of well-being more than
negative and related subjective wellness research into the satisfaction of life and
happiness index. According to Marshford-Scott, Church and Tayler (2012), the views
of children's well-being are social and economic, psychological and mental health,
philosophical and educational aspects.

Physical-material well-being elements can be considered as part of objective well-

being but with living and health-related experiences they also represent a strong sense
of subjective well-being. Social well-being can be thought to need such relationships
that strengthen the overall well-being of the individual. Respect, empathy, and
authenticity are key words in such relationships. (Stewart-Brown 2004, 31.) Psychical
well-being is also a prerequisite for the establishment of working social relationships
(Myllyniemi & Plot 2006, 144-146).

Various studies have shown that warm and affectionate education of children leads to
psychological well-being and success of life. To ensure psychological well-being, the
child should have sufficient care and guidance at home, but also protection and
control. (Myllyniemi & Plot 2006, 144-146.) Psychological well-being supports the
achievement of social well-being. Positive social relationships and the mental health
of the individual are essential. Mental and social factors are important for the well-
being builders. (Larsen & Eid 2008, 3-6.) Social interaction is essential for the normal
mental development and balance of the child. The core issues of interaction are the

connection between the other and the second individual's presence. (Järventie 2001,
107-108). Social and psychological well-being complement each other. One can
imagine that the individual's well-being is not balanced without the fulfillment of both
of these subjective elements. Similarly, prosperity is not balanced without considering
all its components. Physical, psychological and social well-being are a very distinct
entity. Physical and material well-being are factors influencing the individual's
experiences and thus have a very significant impact on the individual's social and
mental wellbeing.

3 Sustainable Development Goals of United Nations

In September 2015 UN member countries adopted a plan called Agenda 2030. The
plan consists of 17 Sustainable Development Goals (SDG) and 169 targets that guide
the country´s decision-making in areas of critical importance for humanity and the
planet and are monitored by both global and state´s own indicators. Three dimensions
of sustainable development - economic prosperity, social justice and a good state of
the environment are visible in the SDG. Each country is committed to include these
goals to their concrete policy and actions.

The concept of well-being, both planet and individual, is clearly visible in the SDG. Not
only in the 3th goal Good health and well-being but throughout the list of goals. In this
paper we focus on the individual's well-being, more closely on children's well-being.
Finland´s last Agenda 2030 Implementation report (2017, 27) states that ensuring the
well-being of children is one of the focus areas to strengthen a non-discriminating,
equal and competent Finland. According to implementation report of Finland (2017,
50) one key goal is to strengthen the cooperation with children by engaging them in
the national and international discussion around the subject. Finland´s Agenda 2030
implementation report (2016, 13) acknowledges the fact that there are challenges
including children to the process. Question is, how to interact with children in a way
that enhances participation, is appealing and still gives reliable information of the well-
being of individuals.

When evaluating well-being of children, one important indicator in Finland is Child

barometer introduced in 2016. Barometer is part of a society's obligation to hear and
listen every member and the obligation is based on both national legislation and the
UN's Declaration of the rights of the child, which Finland ratified in 1991. (Luukkanen
2016, 7; Aira, Hämylä, Kannas, Aula & Harju-Kivinen 2014, 6-7). We wanted to take
this well-being evaluation one step further and create a different, more fun way for
children to be heard.

4 Steps in the Design of the Game: Past, Present and Future

This chapter describes the development process of the game prototype. An essential
part of the process has been the research work that has been done to find the
indicators needed for the game. It was important to find the most important indicators
for the well-being of small children. The gamification of indicators has also proved to
be challenging.

Figure 1. Steps of the design of the indicators

Step One: Analysis of Need and Existing Indicators

The development of products should always be based on genuine social need. That
is why the development of a new product should be based on a good needs analysis.
This case started as a part of international Marie Curie research project which one aim
was to understand concepts of vulnerability and well-being. According to analysis
(Marjanen, Ornellas & Mäntynen 2016) the dominant international models
demonstrate gaps when compared to theory and literature on child well-being and
development. International well-being indicators represent a primarily political
economy-based, material approach to defining and understanding child well-being.
Indicators are still predominantly focused on a child’s needs and rights, over that of
his/her will and desire.

Step Two: Research of Indicators

It was soon discovered that the problem is international and national. The dominant
international models have failed to adequately incorporate indicators which reflect on
e.g. civic and life participation. Existing models is also difficult to fit only one definition
or theoretical approach of well-being. Especially small children voice is still missing in
Finland even though country is well known about child friendly and participatory
schools and daycares. The well-being of Finnish children is examined, among others,

on Finland's poverty indexation. Other types of repeatedly collected information on the
well-being of young children is not available. In the case of older children, Finland is
studying health behavior every two years. The WHO School Examination to measure
school exams will be implemented in grades 5, 7, 9, every fourth years. There are also
some issues of well-being in Pisa. In addition to these schools, a separate
questionnaire can be submitted to both primary and secondary school pupils and to
senior staff and staff. All of these produce very important information on well-being of
Finnish children. But still for example maternity clinics and daycares based their
analysis mostly on their own observations and discussions with parents.

The importance of hearing children also in child protection is well-known but methods
varied a lot. Common tools are needed to support multi-professional teamwork. It is
also important to note that the new childhood study highlights the perspective of
children's rights, emphasizing the child's agency and ability. This approach involves
an idea where a child is not seen as a passive subject but as a social actor who is
already and is not just becoming one (Qvortrup 2012). According to Elden (2012), a
new study on childhood has brought a Mixed-method approach to data collection in
research (see also Gabb 2010; Mason 2006). We ended up collecting information in
a multidimensional way, as this will enable a more complex understanding of the issue
to be explored. The so-called Mosaic approach (Clark 2005), which can be defined as
a Mixed-method, brings together both an individual and a community perspective. It
utilizes traditional observation and interview methods, incorporating participatory tools.
The approach emphasizes the child's own perception and consultation on their life,
interests and concerns. The method emphasizes inclusion, reflection, and listening to
children's own life experiences. (Clark 2005.) In the first phase, we decided to use the
draw and tell method. According to (Liamputtong & Fernandes 2015) drawing makes
it easier to say things that are difficult to say or express in other ways.

Research group, where was 12 students from master and bachelor studies, started
their work with subjective well-being indicators in autumn 2017. Research group
shared their work. For example, two teams interviewed children in spring 2018. These
interviews have been collected from four big cities from Metropolitan area of Finland.
So far 38 children have been interviewed so that we can ensure that the indicator
contains well-selected and high-quality questions. At this point we had 17 questions
and help questions for them. We noticed that when interviewing small children, no
more questions can be asked. Six children were interviewed before the actual
research part. After pilots we shaped questions. The main aim is that the
questionnaires are understandable and the children are easy to answer for them.

During this research process we noticed that it is essential that the indicators measure
positive things. Research situations should be nice and empowering for children. In
our research psychological well-being was studied by four questions, all of which

provided essential information on the child's experiential well-being. Even though we
avoided negative feelings and questions the most informative was, “when you have a
bad feeling”. It gave plenty of information about fears. The most remarkable were
loneliness and bullying. Social well-being was also examined through four questions.
The question of who listens to you, was some of the children difficult to answer. In the
context of the research results, the purpose of the questions is well-founded and all
the questions of the subject produced a lot of information and questions are essential.
We still noticed that in the future, social and physical well-being questions could be
asked with a combined question of who cares for you. The question turned out to be
informative. Questions related to physical well-being sought to explore the experience
of children about their own health. Children's experiential knowledge of their own
health status was very limited, perhaps because the issues of the subject were not
directly related to illness or health. The most important conclusions were the
importance of routines in everyday life which children emphasized in their speech.

Parents were also interviewed and their ideas and opinions have used to develop
questionnaire during spring 2018. The analysis of the results is still not ready. These
interviews, both children and families, were extremely important in process, because
families also provided valuable feedback on the development of indicators.
End of spring 2018 there was also a workshop for professionals where existing
material and develop tool were developed further. The main idea is that the whole
ecosystem which is dealing with children is involved for the developing work.
Practically this means that multi-stakeholders have engaged in open innovation and
experimentation processes in real world contexts.

Step Three: Prototype of the Game

International eHealth conference and its game hackathon gave the research group
possibility to develop further the idea. During hackathon students and graphic designer
tested some indicators to develop it into a playful form. Practical questions like, what
happens before bedtime or what is your favorite food at home/kinder garden are quite
easy to change to game format. Idea to use children’s day and its routines as a game
storyline arouse also from conference. (

Figure 2. The timeline of the game development process

The process of the game designing and prototyping continues during summer and
autumn. Game design timetable:

i) Prototyping and building 3 rapid iterations and testing those among the
children & educators;
ii) Choosing one prototype to improve it to the MVP (Minimum Viable Product);
iii) Testing the MVP among the children whom will provide feedback to the game
developers and building Pre-Alpha;
iv) Rinse & Repeat! Testing & building Alpha with the children;
v) Testing & building Beta;
vi) Soft launch in certain schools & kindergartens.

Game development into this stage should take approximately seven months. After
the Soft Launch the Game the team will start to collect data from the gaming
sessions and get feedback from children. New ideas and implementations in the
game will start to correlate with the questions. It will take almost two years that the
data starts to show accurate results. After that the new implementations will be
easier to make and better data can be collected from wider audience.

It is important that the game is developed by active gamers and game developers
with the help of the end-user, children. Psychologists, social workers, parents and
teachers are important part of the development. There is still a big risk that the game
loses its true potential if they are allowed to decide the game design and functions.
According to Zichermann the problem is if parents and teachers got involved in the
game design “kids could smell that shit from a mile away”.

Picture 1. There are three characters the player can choose to play with

There will be three characters to choose at the beginning of the game. All characters
have different characteristics and temperaments. “The three” as we call them, have
different personalities which reflect the moods of the player.

Picture 2. The players character home and the daily tasks are waiting

Ideas what data are collected are quite simple at the moment, one example is that
the character awakens from its home and from the player is asked “How well did you
sleep?” After the question there will be three emoji faces which allocate the answer:
1. Good, 2. Okay, 3. Bad.

Picture 3. Emoji faces for indicating the answers in the game

Step Four: Gamification

Previous research and game prototyping showed for research group that developing
process of the good tool for children is much more difficult to do than we thought. One
difficult thing is age of the children. Small children can’t focus for a long time. It is
particularly challenging to collect reliable data from small children. Understanding the
concept of "well-being" also requires a lot to understand even from adult literate
people. Mashford-Scott et al. al (2012) commented that children can also have as
linguistic as metacognitive abilities to communicate with an adult researcher from their
own perspective. That is why we have put lot of efforts to develop questions so that
they are easy to understand but still there is straight link to subjective well-being
concept. It is obvious that questionnaire can’t be very long.

5 Conclusion

Our developing process has had and will have strong collaboration with stakeholders.
It is extremely important that innovation and developing process has strong links to
the real-world context because this way the innovations being developed are
sustainable and support sustainable development of society. Even though
development process of the tool to get new kind of research data from children is still
in progress, first results and solutions have been found. The most important indicators
are known and they based on theory of subjective well-being. Theory has modified to
question form in child-friendly way. The last idea to develop product was to generate
it to game format. Game format for data collection gives more time to collect the data

than just interview children. It is possible to have more game sessions and the need
of an interviewer is not compulsory, so information can be gathered easier and

The 3th goal of SDG emphasises the meaning of good health and well-being. We
focused on children's well-being in this paper. According to many previous research
results (see e.g. Ben-Arieh & Goerge 2001; Ben-Arieh 2007; Minkkinen 2013)
measuring well-being requires new types of subjective indicators. Especially demands
to give voice also for one of the most vulnerable group, children, have risen from


Aira, T., Hämylä, R., Kannas, L., Aula, M. & Harju-Kivinen, R. (2014). Lasten
hyvinvoinnin tila kansallisten indikaattoreiden kuvaamana [Child welfare
status as described by national indicators]. Lapsiasiavaltuutetun toimiston
julkaisuja 2014:4. Available from
Ben-Arieh, A. & Goerge, R. (2001). Beyond the Numbers: How Do We Monitor the State
of Our Children. Children and Youth Services Review 23(2), 709-727.
Ben-Arieh A. (2007). Measuring and monitoring the well-being of young children around
the world. Paris: UNESCO.

Casas, F. (1997). Children’s Rights and Children’s Quality of Life: Conceptual and
Practical Issues. Social Indicators Research 42, 283-298.

Clark, A. (2005). Ways of seeing: using the Mosaic approach to listen to young
children’s perspectives. In Clark, A., Kjørholt and Moss, P. (Eds.) Beyond
Listening. Children’s perspectives on early childhood services. Bristol: Policy
Press, 29–49.
Diener, E., Oishi, S. & Lucas, R.E. (2009). Subjective Well-Being: The Science of
Happiness and Life Satisfaction. In S. J. Lopez & C. R. Snyder (Eds.) Oxford
handbook of positive psychology. New York: Oxford University Press, 187–
Elden, S. (2012). Inviting the messy: Drawing methods and “children’s voices”.
Childhood 20(1), 66-81.
Fattore, T, Mason, J & Watson E. (2007). Children’s conceptualisations of their well-
being. Social Indicators Research 80(1), 5-29.

Gabb, J. (2010). Researching Intimacy in Families. Basingsstoke: Palgrave Macmillan.

Government Report on the implementation of the 2030 Agenda for Sustainable
Development. Prime Minister’s Office Publications 11/2017.
Järventie, I. (2001). Eriarvoisen lapsuuden muotokuvia [Portraits of unequal
childhood]. In I. Järventie & H. Sauli (Eds.) Eriarvoinen lapsuus [Unequal
childhood]. Porvoo: WSOY, 83–124.

Konu, A. 2002. Oppilaiden hyvinvointi koulussa [Well-being of students at school].

Tampere: Tampere University Press.
Larsen, R.J. & Eid, M. 2008. Ed Diener and the Science of Subjective Well-Being. In
M. Eid & R.J. Larsen (Eds.) The Science of Subjective Well-Being. New York:
The Guilford Press, 1–13.
Liamputtong P. & Fernandes S. (2015). What makes people sick? The drawing
method and children’s conceptualization of health and illness. Australasian
Journal of early Childhood vol. 40(1), 23-32.
Marjanen, P., Ornellas, A. & Mäntynen, L. (2016). Determining Holistic Child Well-
being: Critical Reflections on Theory and Dominant Models. Child Indicators
Research, 1-15. DOI: 10.1007/s12187-016-9399-6

Mashford-Scott, A. & Church A. & Tayler, C. 2012. Seeking Children’s Perspectives
on their Wellbeing in Early Childhood Settings. International Journal of Early
Childhood vol. 44(3), 231-247.
Mason, J. (2006). Mixing methods in a qualitative driven way. Qualitative research
6(1), 9-25.
National report on the implementation of the 2030 Agenda for Sustainable
Development, Finland. Prime Minister’s Office Publications 10/2016. Available
Luukkanen, T. (2016). Lapsibarometri 2016 [Child barometer]. Lapsiasiavaltuutetun
toimiston julkaisuja. Available from
Minkkinen, J. (2013). The Structural Model of Child Well-Being. Child Indicator
Research, 6, 547-558.

Myllyniemi, R. & Tontti, J. (2006). Tuottoa sosiaaliselle alkupääomalle – lapsuuden

kasvatus ja aikuisiän tunteet [Return to social start-up capital - childhood
education and adult emotions]. In I. Järventie, M. Lähde & J. Paavonen (Eds.)
Lapsuus ja kasvuympäristöt. Tutkimuksen kuvia [Childhood and growth
environments. Research Images]. Tampere: Tampereen Yliopisto,
Sosiologian ja sosiaalipsykologian laitos, 142–161.

Pollard, E. L., and Lee, P. D. (2003). Child Well-being: A Systematic Review of the
Literature. Social Indicators Research 61(1), 59-78.

Qvortrup, J. (2012). Users and interested parties: A concluding essay on children’s

institutionalization. In A. T. Kjorholt. & J. Qvortrup (Eds.) The modern Child and
the Flexible Labour Market. Early Childhood Education and Care, 243-261.

Stewart-Brown, S. (2004). Parenting, Well-Being, Health and Disease. In A.

Buchanan & B. Hudson (Eds.) Promoting Children’s Emotional WellBeing.
New York: Oxford University Press, 28–47.
WHO Group (1997). Programme on mental health. Measuring quality of life.
Available from 15.5.2018
Veenhoven, R. 1996. The Study of Life Satisfaction. W.E. Saris., R. Veenhoven., A.C.
Scherpenzeel & B. Bunting (Eds.) A Comparative Study of Satisfaction with Life
in Europe. Eötvös University Press.

Zichermann, G. 2010: Fun is the Future: Mastering Gamification. Available from

“Daegu Living Labs” as City Innovation Platform
1Hee Dae Kim

1 Daegu Regional Innovation Agency (DRIA)

Category: Innovation Paper

The purpose of this study is to introduce the components, operational mechanism,
operational philosophy, and sub-models of Daegu Living Labs, and illustrate how
they are bringing changes to the city by creating sustainability. Daegu Living Labs
are a platform that transforms the city into superstructure and infrastructure. The
superstructure is a program that trains innovation processes called 'Stimulation -
Training - Facilitation - Diffusion', aiming to increase Smart Citizens and innovation
facilitators. The infrastructure is a process of co-creation with a citizen panel in a
specific place, such as an alley living lab, a social living lab, and a smart living lab.
The infrastructure is aimed at solving the problem in the context and involved Smart
Citizens and Facilitators trained in the superstructure. The top and bottom are
interconnected through Daegu Living Labs Governance and spread through the
Creative City Global Forum. As Daegu Living Labs expanded its voluntary citizen
panel, the city's direct democracy became more active, and the creative individual,
alleys including living labs were linked to each other in a multi-layered manner,
increasing the density of innovation in the city. Daegu Living Labs are assisting the
city to substantially achieve the UN’s SDGs agenda, such as Making resilient and
sustainable city, ensuring learning opportunities, achieving gender equality, Building
resilient infrastructure.

Keywords: Innovation Platform, Daegu Living Labs, Resilience, Citizen Science,

Social Living Lab, Alley Living Lab, Social Innovation, Sustainability Development

1 Background

Daegu is an inland city with a population of 2.4 million residents located in the south-
eastern part of the Republic of Korea (100 minutes from KTX in Seoul). Daegu has
played important role as the educational, industrial and cultural hub city of south-
eastern Korea. Historically, the region not only displayed high receptivity for
innovation but also has been the last bastion when it comes to overcoming national
hardships. However, from about 25 years ago, the key industries of the city started
to gradually decline, and cultural openness has been weakened.

Moreover, as global cities increasingly adopt the 2030 Agenda for Sustainable
Development Goals that UN issued with its goal of “Transforming our world,” Daegu
was strongly calling for a dynamic City Innovation Platform led by civilians. In 2014,
the Creative City Forum consisting of diverse innovators was launched: the City
Innovation Platform started its operation. The former public participatory projects
evolved into upgraded Living Labs by the City Innovation Platform. During the
process, the platform presented various forms of Living Labs and utilized them as
important tools for the city’s sustainability and resilience. In other words, the City
Innovation Platform and Daegu Living Labs are evolving together through their
interactions with each other.

This study aims to introduce the components, operational mechanism, operational

philosophy, and sub-models of Daegu Living Labs, and to illustrate how they are
making changes in the city by creating sustainability. Below are the descriptions of
the evolution process of the city:

Historically, Daegu has been a leader in preserving the country in times of crisis.
During the Japanese colonial period (1910-1945), it was the birthplace of the National
Compensation Movement, in which the citizens voluntarily repaid the debt and tried
to defend it. This movement was Korea's first donation cultural movement, which later
influenced foreign repayment campaigns in China and Vietnam. Its historical
significance was acknowledged globally last year, and records related to the Treasury
Compensation Movement were listed as UNESCO World Heritage Sites.

Culturally, Daegu has produced countless artists representing Korea in literature,

art, music, and theatre. Especially, there are concert halls where citizens can enjoy
all over the city. Various music performance festivals such as classical, musical,
opera, and folk, and various music education programs for citizens, enhance the
creativity of the city. Last year, it was recognized as a UNESCO music venue and
acknowledged its value of the city worldwide.

In economic and industrial terms, Daegu was at the heart of the economy as it was

the birthplace of the Samsung Group, a global company in the Republic of Korea.
However, due to the delay in responding to changes in the global industrial
environment, restructuring of local representative industries including textiles has not
been achieved properly. The subsequent failure of local conglomerates and the
failure of new entrants have reduced the flexibility of the labour market. At present,
the city industry is composed mainly of parts industry (secondary industry) which
cooperates with large manufacturing companies (Hyundai Automobile, Hyundai
Dockyard, and POSCO) in the adjacent cities with a high proportion of urban service
industries such as education, consumption and culture (tertiary industry). A total of
15,000 people from the higher education institutions, including Kyung-buk National
University, migrate every year from 5,000 to 8,000 young people leaving for other

Until the time when Kwon Young Jin, the 6th mayor directly elected by the citizens,
entered the city (2014), the central government promoted the policy with a supplier-
centred method of input (industrial complex infrastructure, and strategic industries
promotion policy). The monopoly of the central government's budgeting functions and
the low financial independence of the city made it difficult to escape interference and
control of the central government's planned economy. Despite these policy efforts,
the Gross Regional Product (GRDP) has been the lowest in the nation for 20 years,
and as a result, the industrial structure has also failed to improve.

Since 2014, the city has tried to make new changes. We admit that we can no longer
guarantee city development by industrial innovation alone, and revised the direction
and goal of urban innovation from the fundament. Still, the city selected the
upbringing industry and planned the investment, but started approaching it in a very
different way from the previous one.

As the need to combine the level and scope of innovations has been disseminated to
citizens, new initiatives have begun, involving citizens as subjects. In January 2015,
in order to make Daegu a full creative and dynamic city, the Creative City Study
Group was formed with citizens from all walks of life, including citizens, experts,
youth students, students, and artists. With this in mind, the Creative City Forum
(CEO: Prof. Kim Young Hwa) was formed and currently, about 2,000 creative citizens
are participating in the panel. Citizen panels are involved in various types of living lab
experiments that cope with industries, social and urban issues, and are also playing
a key role in urban innovation platforms.

2 The City Innovation Platform and Daegu Living Labs

In order to enhance the substantiality and resilience of the urban ecosystem, it is

necessary to combine the axis of innovation which centred on industrial innovation
and the axis of innovation cantred on social innovation. In the past, the two innovation
systems had not been carried out in combination with each other. During the
developing period of a city or a country, social innovation was focused on establishing
networks to solve urban problems. Innovation systems on the level would face the
limitations in the growth, whereas innovation systems on the scope would face the
limitations in the recyclability and resources.

Therefore, the vertical axis of innovation cascading from the National Innovation
System (NIS) to the Sectoral Innovation System (SIS), the Regional Innovation
System (RIS), and the Innovation System of the enterprise, must be compromised
with the horizontal axis of innovation that are expanding from individual, who have
the CEL DNA(Creativity-Empathy-Link), to instantly linking up to 1% of the city's
population (Druker, 1993; Franz, 2015; Malerba, 2002).

We consider City Innovation Platform as a combination of the level and scope of

innovation, and Daegu Living Labs as the substantialized form of this City Innovation
Platform. In other words, during the process of achieving the Sustainable
Development Goals of the city, the City Innovation Platform promoted the discovery
and operation of the individual Living Labs.

Figure 1. The relationship between City Innovation Platform and Daegu Living Labs]

3 The Philosophy of City Innovation Platform

The total framework of the City Innovation Platform including Living Labs is depicted
in the following graphics. The graphic is a modified version of “Open Innovation 2.O:
A new Milieu”model presented by Curley & Salmeline (2016), specifically adjusted in
the context of Daegu.

The philosophy of City Innovation Platform, which focuses on City Innovation

platform, can be summarized as follows.

Figure 2. The Philosophy of Daegu City Innovation Platform

*Open Innovation 2.O:A new Milieu (Curley & Salmelin, 2016), revised for City

Innovation Platform.

Operating Systems

The operation of a platform is enabled when it secures the communication,

methodology, and openness that allows each and every participant’ voluntary
participation without marginalizing them. Therefore, the City Innovation platform is
operated by utilizing the following five tools and systems:

Design Thinking, which involves stakeholders to define problems and create
solutions, Widersprüche Oriented Innovation System, which creates new
solutions while compromising contradiction between current resources and ideal
goals in the process of creating solutions, Socio-cracy, which does not ignore the
opinions of others and makes it to the end of the collective agreement, Citizen
Science, which is dedicated to the common goals of the city with the spontaneity and
dynamism of individual citizens, and Open Data, which organizes the recycling of
data resources generated from the whole life cycle of the Living Labs, are applied as
management principles (Buck, Villines, 2017; Curley, Salmelin,2016 ; Herr,2017;
IDEO,2011; Liedtka,Ogilvie,2011; Robertson,2015; Zurich Univ.,2015).


High sustainability to increase the recyclability of city resources to the next generation
and high resilience to changes in the external environment are the ultimate goals of
City Innovation Platform. In the process of living lab, the middle goal is to connect
innovation entities, to find a lively and positive urban language, and to attract new
human resources with talents (Landry, 2006; Lee, 2014; Hirvikoski, 2017).
All these goals are closely related to the UN’s SDGs Agenda. The City Innovation
Platform provides goals of the subunits and tools for evaluation in substantially
implementing the UN’s SDGs Agenda, such as Making resilient and sustainable city,
ensuring learning opportunities, achieving gender equality, and Building resilient


Innovation Platform uses the three internal resources to operate the urban innovation
platform. Smart Citizen, which is the goal and means of the living lab and also plays
a role as the subject, the city's alley as the basic unit of space of innovation, and a lot
of data created by space and people (Bror, 2017; Landry, 2000; Zurich Univ., 2015).
People are subject and object of the City Innovation Platform, and they are at the
centre of the utilized resources. The concept of resources is extended from citizens
as the centre, to time and space from people. Resources of time are recorded as data
and innovations are generated by managing the prototypes by versions or opening
up the accumulated data. When it comes to resources of space, alleys are utilized as
the basic unit. Since an alley is a minimum unit where the lives of city civilians
intersect, it serves as the minimum unit where innovations happen.

4 Structures of Daegu Living Labs

We will divide Daegu Living Labs into two separate structures, superstructure and
infrastructure, and describe both. The superstructure is directly connected to the City
Innovation Platform, and the infrastructure is related to three forms of operation at a
specific place and to Living Labs. We separated the process of discovering and
training human resources, which is one of the most important factors of Living Labs.
Thus, the superstructure is focused on training the facilitator and panels needed for
various living labs operated in a specific place, and on improving their problem-
solving capability. These training programs of the superstructure are operated under
the titles of Creative Study Group, Focus Group, Social Dinning, Social Fiction
School, and etc, including the program operation is implemented by the Creativity
City Forum.


The superstructure of Daegu Living Labs focuses on the actual change mechanism
that goes through the process of ‘Stimulation-Training-Facilitation-Diffusion’.
First, in order for innovation to take place, the stimulus to recognize the need for
change must be made naturally, and The Creativity City Forum operates an
education and learning program that stimulates these changes. The next step in this
program of change is to train citizens with the deepening process of the Focus
Research which creates highly qualified policies in the areas of interests. The Social
Dining is also used to build communities of interests and perform problem defining
and solution planning by sector. The next step is training facilitators to organize the
citizen panel through the Social Fiction Civic School, from making ideas of urban
future to making prototypes. Once stimuli-training-facilitation is done, the panel will
spread to various programs such as the Open Topic Forum and the Creative City
CEO Forum to operate the change mechanism (Chesbrough, 2003; Landry, 2000).
The superstructure of innovation is aimed at securing the Smart Citizen panels to
spread innovation within the city and organizing the necessary infrastructure activities
through the training of facilitators. Indicators were developed to measure the degree
of innovation of the panels and facilitators participating in the superstructure. The
VITAL index of H.S. Lee (2014) and CEL index of H.D Kim (2017) are typical tools.

Figure 3. The Progress of Superstructure of Daegu Living Labs

Infrastructure of City Innovation Platform

The Smart Citizens who were trained at the superstructure are experimenting Living
Labs in various forms. Alley Living Lab, Social Living Lab, and Smart Living Labs are
the representative forms of Living Labs. Each Living Lab has completely different
direction and goals. Thus, we named them “Daegu Living Labs” to manage various
living labs; to let them share common fundamental processes and human resources
including facilitators, despite the differences in their goals and direction.
Daegu Living Labs are composed of step-by-step stages which are search,
experiment, and evaluation (see Table 1). Activities at each stage are provided in a
manual format. Each stage of the living lab is mapped to a technology development
stage, and the required Civic Tech or Urban Tech is developed in an agile manner.
As the process progresses, the generated data is designed to have a standardized
data framework for a new service or next prototype (Schuurmann, 2015; Ståhlbröst
& Marita, 2012; Almirall, Lee & Wareham, 2012).

Table 2. Activities according to Living Lab Performances

Discoverin Conceptu Prototype Before After
Development Release
g Ideas alization Development Release Release

Living Lab
A. Exploration B. Experimentation C. Evaluation
User behaviour 'Prototype 'Product and service
analysis and development' and development' and
'conceptual design' implementation demonstration
① Problem-focused ① Prototype ① Development of products
user behaviour development and services
analysis - Prototype - Prototype development
- General user development through based on prototype results
behaviour analysis collaboration
Living Lab - Core User Behaviour ② Product and service
Performance Analysis ② Prototype demonstration and feedback
experiment and user - feedback to extended users
② Conceptual design feedback
of product/service for - Prototype installation
problem-solving and feedback
- Collaborative design - Participation
with users observation,
satisfaction survey

① Design of living wrap propulsion system
- Research topic setting
Preparatory - Participating organization and promotion system
Activities - Infrastructure construction: Clinical experiment and demonstration, site
selection, related technology infrastructure
- Intellectual property rights management regulations and legal system

②End-user organization
- End-user group composition scheme
- User participation motivation plan
- User feedback collection (site visit, focus group interview, survey)
- Organization of training programs for users and technicians

Alley Living Lab: Urban Regeneration Model through

This was an early model of Daegu Living Labs, and many living lab models were
experimented in the process of reclaiming old urban alleys. A representative success
story is Kim Gwang Seok Street (a famous Korean folk song singer in the 1980s)
where young artists defined the problems in the alley with the inhabited citizens and
succeeded in regenerating the city by creating works of art using alley walls. In
addition, the handicraftsmen of the old and original city centres participated in the
renovation of the alleys and created a new festival prototype called the Tool Festival
in Bukseongro Street. There is also the Modern Alley Living Lab where residents
of the unrecognized modern alleys restored historical contents, further created a new
job as an ‘Alley Commentator’. The city is expanding user participation innovation
centred on more alleys.

Social Living Lab: Urban Problem Solving and Social Venture Diffusion

In 2017, Daegu is creating a new youth culture that uses real-life space as a
laboratory centered on trained youth facilitators, who find solutions to social problems
through co-work with various stakeholders in the city (Table 2).

We support young people strengthening the living lab's capabilities through

education, mentoring, consulting, etc. so that they can discover their own social
problems and find solutions. They are becoming 'social start-up' or 'social venture' by
discovering social business model based on social innovation idea.

Table 3. Social Living Lab Project Lists

Project Name Content of the Project

Earth Market # Planning a meal with conversation and culture for young
people eating alone
Lectures and lectures are held in the provinces through
476 Project crowdfunding method for lectures, cultural performances and
information gap in local and metropolitan areas

Flosbebe Solving psychological and economic problems through
collaboration activities of single mothers

Knunyang (CAT) Prepare a plan for creating an environment where street

cats and people live together
Nurira Project Youth activities that guide students to commute safely

UNIAS Resolving the problem of unauthorized speculation of

college trailer village waste
KWANUMDONG Improving communal spaces (parks) and creating a safe
LivingLab environment for elementary school students

BANDI Cops. Activation of use of environment improvement of parks in

neighborhood and creation of theme parks

DTC in Daegu Cultural contents development that each university can

exchange and young people can communicate

Media Dalgona Producing independent publications dealing with stories of

local youths rather than metropolitan areas

Smart City based on Smart Living Lab

Many cities in Korea, which have excellent information and communication

infrastructure, are very competitive in adopting Smart City. Unlike other cities that are
built focused on hardware, Daegu uses various ICT technologies to address various
urban problems and approaches to solve specific problems with the public, citizens,
private companies, universities, and support agencies together. To this end, we are
designing the Smart Living Lab Platform in which a large number of citizens are
able to participate. Daegu Smart City is focused on how many citizen panels
participated in the problem to create a common solution, rather than how advanced
technologies are adopted.

In 2016, the Digital Twin City is being built in cyberspace as well as building
advanced ICT infrastructures on the physical space of Suseong Alpha District
(southwest of Daegu City). On that, the Smart Living Lab will be operated by
residents who use ICT tools in areas such as energy, autonomous mobility, and urban
disaster management. Data is refined, standardized and opened to enable new
business models and products to be actively developed within the Smart Living Lab.

Figure 4. Smart Living Lab in Daegu

Connection of Superstructure and Infrastructure

In the above, we explained the structure of Daegu Living Labs and the mechanism
by which Daegu's innovative platform works. The infrastructure where various living
lab models are experimented in concrete space and superstructure where they are
nurtured and spread are connected, and mutual feedback is exchanged, while
expanding throughout the city gradually. The Living Lab Governance and the
Creative City Global Forum mediate these interconnections.

The Daegu Living Labs Governance is a network organization composed of

operators and facilitators of individual living labs. Organizations will explore the future
agenda of urban innovation and provide manual for standardization of living labs. The
standard manual consists of implantable hierarchy modules, including panel
acquisition, data management, problem definition, solution exploration,
experimentation, evaluation process, specific user tools, and training programs.
Organizations also conduct activities necessary to open up and standardize city-
owned data, as well as to identify citizen-participatory business models.

The Creative City Global Forum plays a role in spreading out the superstructure
and infrastructure of the Daegu Living Labs. The Forum is a festival venue to
showcase the performance of various Living Labs being experimented in the city and
to provide deepened workshop training for participating panels of the urban

innovation platform. Through the Creative City Global Forum in 2017, we signed the
MOU with the European Network of Living Labs (ENoLL).

Pictures from Office of Daegu Creative City Global Forum

The Roles of Daegu Living Labs for Sustainable Development Goals

Various forms of Daegu Living Labs are substantially operated in direct and indirect
connections with the 2030 Agenda for Sustainable Development Goals (SDGs) that
UN adopted for its goal of “Transforming our world.”

Specifically, Alley Living Lab, which solves problems of old city alleys with the
inhabited citizens, is related to “Making cities and human settlements inclusive, safe,
resilient and sustainable” (Goal 11). Social Living Lab, which is operated with the
purpose of urban problem solving and social venture diffusion, is faithfully performing
the agendas of “Ensuring inclusive and equitable quality education and promoting
lifelong learning opportunities for all” (Goal 4) and “Achieving gender equality and
empower all women and girls” (Goal 5), in their individual projects. Smart Living Lab,
accompanied by a large-scale input of human resources and infrastructure, is
implementing the agendas of “Ensuring access to affordable, reliable, sustainable
and modern energy for all” (Goal 7) and “Building resilient infrastructure, promote
inclusive and sustainable industrialization and foster innovation” (Goal 9)

Daegu Living Labs Governance is providing the learning opportunities of SDGs

Agenda during the training of Smart Citizens and facilitators and monitoring the
operational processes by making specific standards. Specialists from various fields
and citizen panels participate in designing the standards.

5 Conclusion

We introduced the structure and operation mechanism of Daegu Living Labs, which
are being experimented with the City Innovation Platform in Daegu. For a long time,
Daegu has been experimenting with community-based co-creation focusing on the
alleys and is currently expanding throughout the city with several living labs. Various
living labs have been tested, but we are still young and evolving.

Daegu Living Labs have produced many meaningful results by organically
operating the upper and lower structures. With the expansion of the living labs, a
large number of voluntary citizen panels have been secured, and direct democracy
has been activated, such as the Citizens Roundtable, the People's Participation
Budget System, and the urban innovation density has increased as the multi-
innovation layers of Dot (Individual) - Line (Alley) - Area (Living Lab) are
connected. Every time smart city services or new public policies was provided, rather
than the existing top-down approach, the use of living lab approach (bottom-up) was
given priority, helping start-ups and young people experiment with various paths of
social entry.

By doing so, Deagu Living Labs are supporting the city to substantially realize the
UN’s SDGs Agenda, such as Making resilient and sustainable city, ensuring lifelong
learning opportunities, achieving gender equality, Building resilient infrastructure.
Daegu Living Labs Governance established the system to manage and evaluate
SDGs Agenda to make it the first interest in fostering an innovation ecosystem
approach in Daegu.

There still are problems that remain. The problems provide us with the opportunity
and challenge for the advanced studies. They will be sophisticated standardization of
Living Labs model as City Innovation Platform and of data, A time-series analysis of
the SDGs achievement in each Living Lab, and conflict management and leadership
issue during the process of implementing the SDGs.

To do this, more and more living labs are needed in the city, and advanced networks
like ENoLL need to be activated. The solidarity among living labs enables to making
cross-validation, joint learning, and interchange of prototype data. This will lead to
more smart citizens, agency organizations, and facilitators connecting and creating a
more resilient and sustainable city.


Almirall, E., Lee, M. & Wareham, J. (2012). "Mapping Living Labs in the Landscape
of Innovation methodologies", Technology Innovation Management
Review, September 2012. pp. 12-18.
Bror, S. (2017). Innovation at every level: open innovation ecosystems in policy
perspective. Living Lab Open days 2017, Krakow
Buck,J., Villines, S. (2017). We the People: Consenting to a Deeper Democracy 2nd
Ed. Washington,DC: Sociocracy.Info.
Chesbrough, H. W. (2003). Open Innovation: The New Imperative for Creating and
Profiting from Technology, Boston: Harvard Business School Press.
Curley, M., Salmelin, B.(2016). Open Innovation 2.O: A New Paradigm. Open
Innovation 2.0 Conference, Amsterdam.
Druker,P.F.(1993). Innovation and Entrepreneurship, ISBN-13: 978-0060851132,
New York: Harper Collins.
Franz, Y. (2015). Designing social living labs in urban research. Info, 17(4), 53-66.
Malerba, F.(2002). Sectoral systems of innovation and production, Innovation
System. Vol. 3(2), 247–264.
Herr,G.(2017). Die Unlogik der Innovation: Wie Sie durch Widersprüche Leadership
meistern. Frankfurt: FATIZ Communication GmbH.
Hirvikoski,T. (2017). Living lab–based social innovation and cooperative governance
for the next generation. The Creativity City Forum, Daegu.
Hirvikoski,T., Lehto,.P., Aväri,A.(2016). Development and experimentation plaform
for social, health wellbeing services in the context of Kalasatama health
and wellbeing centre. Laurea Julkaisut:Laurea Publications.
IDEO. (2011). Human-Centered Design Toolkit: An Open-Source Toolkit To Inspire
New Solutions in the Developing. IDEO.
Landry, C. (2000). The Creative City: A Toolkit for Urban Innovators. London:
Earthscan Publication.
Landry, C. (2006). The Art of City Making. London: Routledge.
Lee,H.S. (2014). Creative Economy. Seoul: Maeil Business News Korea.
Liedtka,J., Ogilvie,T. (2011). Designing for Growth: A Design Thinking Tool Kit for
Managers. Colombia Business School.
Kim, H.D. (2017). Datanomics. Seoul: Communication Books.
Robertson,B.J.(2015). Holacracy: The New Management System for a Rapidly
Changing World. New York: Henry Holt and Company, LLC.
Schuurmann, D.(2015), Bridging the Gap between Open and User Innovation?:
Exploring the Value of Living Labs as a Means to Structure User
Contribution and Manage Distributed Innovation. Dissertation for Doctoral
Degree, Ghent University.
Ståhlbröst, A. & Holst, M.(2012). The Living Lab Methodology Handbook. A
Transnational Nordic Smart City Living Lab Pilot–SmartIES.

Zurich Univ.(2015). Standards for Citizen Science: Principles and Guidelines for
Citizen Science Projects an Universities and Other Research Institutions.
Citizen Science Workshop, Zurich.

Listening to locals?
Question-oriented approach
in Field Lab Amsterdam East
1 Sandra Bos

1 Amsterdam University of Applied Sciences

Category: Research in-progress

A question-oriented approach is a popular and, in many cases, recommended modus
operandi these days. However, it can be misleading, and unless properly understood
and put into practice it may seem a mere semantic trick.15 It should therefore be
evident whose question is the focus of attention. Taking residents’ wishes as a
starting point calls for a more precise definition of the target group. Residents come
in all sorts with many different views and needs. In the context of a fieldlab, identifying
bottom-up issues from the start is a difficult assignment. Nowadays a question-
oriented approach is widespread in dealing with all kinds of local issues. No local
proposal is accepted without a prior ‘bottom-up’ consultation of residents regarding
their needs and ideas. On paper, a question-oriented approach may appear feasible,
but in practice it proves to be anything but. Starting with the question: Who is the
requesting party? The residents only or could the researcher, the city council and the
financer also have a say in the selection of the issue and how it will be handled? This
paper describes how the question-oriented approach was applied in an area based
fieldlab developed by the Amsterdam University of Applied (AUAS) in a
neighbourhood in Amsterdam East.

15 Majoor, Stan (2016). De stad als experiment. De organisatie van stedelijke innovatie. Inaugural
address for the Lectorate, Amsterdam University of Applied Sciences; Majoor, Stan (2016). ‘Inleiding’
in: Werken in een gebied: gewoon doen in Amsterdam. Amsterdam Municipality: Project Management
Bureau. pp. 7-13; Karré, P.M., Vanhommerig, I. & Van Bueren, E., ‘De stad als lab voor sociale
verandering’, In: Bestuurskunde, Vol. 24:1, 2015

1 Question-oriented approach

Question-oriented methods seem to be a counter-reaction to a period in which

administrations and market forces more so than the end user determined the
appearance of a renovated square, the types of care available in a neighbourhood,
and the frequency of local green maintenance.16 During the last decade, the position
of the end user, often the resident, in the political governance arena has changed
considerably.17 This is visible in among others the energy and housing sector and the
health care. It has different causes. Residents are seen as a valuable source of
information and for developing more effective policies. They are expected to become
involved in and take responsibility for their environment, allowing the government to
step back in some situations. At the same time, some residents wish to have more
influence on policies, since they believe they can do it better, faster, cheaper.18
Whatever the case may be, from now on public professionals are expected to operate
‘bottom-up’. The systems world reaches out to the lifeworld. The implications for
public professionals are that they have to start operating on the basis of what
residents ask of them or need from them.

The danger of the popularity of the question-oriented strategy, in which residents are
the surveyed party, is that it may lead to a less rigorous assessment of the suitability
of the strategy in each situation. Moreover, among involved and consulted parties the
novel nature of a question-oriented approach raises high expectations which may not
always be lived up to.19 After all, not all proposed issues lend themselves well to being
dealt with locally and by local professionals and researchers.20 Also, there are
frequent collisions between, on the one hand, the power of the comprehensive body
of local knowledge and the novel solutions and, on the other, a market party or a
compartmentalized administration based outside the area.21 Finally, the expression
‘bottom-up’ implies a new method, as if earlier projects were not equally question-

16 Majoor, Stan (2016). De stad als experiment. De organisatie van stedelijke innovatie. Inaugural
address for the Lectorate, Amsterdam University of Applied Sciences
17 Wetenschappelijke Raad voor het Regeringsbeleid. (2012). Vertrouwen in burgers. Den Haag:

Amsterdam University Press

18 Ibidem
19 Smit, H. & Wassenberg, F. (2015), ‘Wie geeft vorm aan de gebouwde stad?’. In: De stad kennen, de

stad maken: De gebouwde stad. Platform 31.; Karré, Philip, Vanhommerig, Iris & Van Bueren, Ellen
(2015). ‘De stad als lab voor sociale verandering’, Bestuurskunde, 24(1), pp. 3-11.; Vliet, K. van, (2002).
‘Vraaggericht werken’. In: L. Verplanke, R. Engbersen, J.W. Duyvendak, E. Tonkens & K. van Vliet
(eds) (2002). Open deuren. Sleutelwoorden van lokaal sociaal beleid. Utrecht: NIZW/Verwey-Jonker
20 Majoor, Stan (2016). ‘Inleiding’ in: Werken in een gebied: gewoon doen in Amsterdam. Amsterdam

Municipality: Project Management Bureau

21 Majoor, Stan (2016). De stad als experiment. De organisatie van stedelijke innovatie. Inaugural

address for the Lectorate, Amsterdam University of Applied Sciences; Smit, H. & Wassenberg, F.
(2015), ‘Wie geeft vorm aan de gebouwde stad?’. In: De stad kennen, de stad maken: De gebouwde
stad. Platform 31

oriented and based on collaboration with end users.22

An interesting aspect of a question-oriented approach is that the term ‘question-

oriented’ almost always refers to ‘the’ question of ‘the’ resident: the resident as the
main end user of the solution. Question-oriented implies that those asking the
question make the first step, followed by a response which reflects that question.23
This is suggestive of one-way traffic: administrations reach out to citizens and the
citizens formulate their needs. The administration is no longer supposed to be a
supplier of services, but a mediator between various different parties.24 What is
needed is a mutual, local interplay of give and take, for the very reason that what
residents are asking for is seldom clear-cut. The definition of the question is preceded
by a process of listening, attuning and redefining what is needed. A commonly used
definition of a question-oriented approach is therefore: establishing a match between
what is available and the (subjective) needs of residents in a process of ongoing
interaction between government, institutions and residents.25

It is this interaction between parties which the fieldlab facilitates. This process
involves more than the multiple needs of residents. In the dynamics of co-creation
several parties have a question and are looking for a matching answer through an
experiment driven learning process. It is up to the fieldlab team, in the case of
Amsterdam East consisting of a fieldlab coordinator, a professor and a government
representative, to investigate the relationships between these different requesting
parties; the residents, the researchers, the public professionals and the financers.

2 A resilient neighbourhood

Since its 2007-2011 neighbourhood renovation, Amsterdam-East has been an area

with an abundance of resident initiatives, neighbourhood communities and a local
administration who provides the opportunities and means to facilitate and stimulate
these dynamics at grassroots level. Experimentation with neighbourhood budgets
and social tender is lively. In short: East is dynamic and ambitious in finding new

22 Karré, Philip, Vanhommerig, Iris & Van Bueren, Ellen (2015). ‘De stad als lab voor sociale
verandering’, Bestuurskunde, 24(1), pp. 3-11; Majoor, Stan (2016). De stad als experiment. De
organisatie van stedelijke innovatie. Inaugural address for the Lectorate, Amsterdam University of
Applied Sciences
23 Vliet, K. van, (2002). ‘Vraaggericht werken’. In: L. Verplanke, R. Engbersen, J.W. Duyvendak, E.

Tonkens & K. van Vliet (eds) (2002). Open deuren. Sleutelwoorden van lokaal sociaal beleid. Utrecht:
NIZW/Verwey-Jonker Instituut; Smit, H. & Wassenberg, F. (2015), ‘Wie geeft vorm aan de gebouwde
stad?’. In: De stad kennen, de stad maken: De gebouwde stad. Platform 31
24 Ibidem
25 Vliet, K. van, (2002). ‘Vraaggericht werken’. In: L. Verplanke, R. Engbersen, J.W. Duyvendak, E.

Tonkens & K. van Vliet (eds) (2002). Open deuren. Sleutelwoorden van lokaal sociaal beleid. Utrecht:
NIZW/Verwey-Jonker Instituut, p. 230

approaches to local democracy. The question is how East, despite current budget
cuts and administrative shifts, will be able to sustain this energy.26 Could it become
even more attuned to its residents and smarter in working with all sorts of
stakeholders? That was the main question the urban district East asked the fieldlab
to investigate. This question matched the fieldlab’s own ambition to contribute to a
resilient neighbourhood. One where residents, business owners and others would be
given the space and be relied upon to do what would make them and their
neighbourhood resilient.27 The implication of this ambition was that the definition and
approach of the issues should centre on the neighbourhood. But how can a large
knowledge institute, like the AUAS, with researchers who are partly unfamiliar with
the dynamics and characteristics of East and who depend on earmarked, project-
specific funding, set up a bottom-up strategy? And this, moreover, whilst partnering
with urban district East, which has its own agenda and urgencies. This was the first
challenge of the AUAS fieldlab team.28

3 The requesting party

In order to get a general idea of current issues in urban district East in the start-up
phase, the fieldlab team organized two public work meetings for key actors in Fieldlab
East.29 In the course of the search for potential fieldlab projects, the team questioned
a number of local administrators, the secretary of the urban district council,
neighbourhood actors and AUAS representatives. Based on this information, the
team wrote an action plan, in which it proposed certain issues and potential
stakeholders. Meanwhile, three professors had joined the fieldlab team. They were
interested in conducting research in East and their studies tied in with the broad
themes contained in Fieldlab East’s Action Plan. A situation arose in which a search
for a research area was accompanied by an offer by the professors to provide
possible methods and techniques. At the same time the local council’s call for support
for its employees who were working in the neighbourhoods grew louder. This request
occurred against a backdrop of the abolition of urban districts as administrative body,
a centralization of resources and expertise, and a greater focus on area-based

26 From 2010 onwards several municipal reorganizations and budget cuts were implemented (Een
stad, een opgave (Binnenlands Bestuur, May 23, 2013)). In 2017, urban district budgets were
reduced by 7 milllion euro; this will be 10.5 million in 2018 and 14 million in 2019 (Parool, 21
September, 2016). In 2018, as part of the most recent administrative reforms, the seven urban district
councils will be abolished
27 As also stated in Fieldlab Oost’s 110 Dagen Plan, 2014
28 The priority area Urban Management of the AUAS started in 2015 with three area based fieldlabs in

three deprived neighbourhoods of Amsterdam.

29 Unlike Fieldlab Nieuw West, Fieldlab Oost did not select an area beforehand to situate its projects.

Ultimately, Fieldlab Oost carried out its three projects in three different neighbourhoods.


Each of the three parties - neighbourhood, urban district council and professors - was
keen to use the fieldlab to fulfil its own needs. In hindsight, the fact that the
neighbourhood in the end drew the short straw in the ultimate selection of problems
and areas was to some extent to be expected in the light of the 110 Days Plan.
Despite its focus on question-oriented approach and resident-controlled local
development, the plan’s chapter ‘The Players’ barely mentions the residents as
partners. Not exactly an appealing invitation for a question (read resident)-oriented

A similar trend can be observed in the projects’ execution. In the context of one of
Fieldlab East projects, the Age Friendly City (AFC) project, the neighbourhood was
regarded as a suitable study area rather than an end user with a specific question.
What’s more, the search for the most suitable neighbourhood for the project’s
execution only started after the project had already been approved. AFC is a global
network of the World Health Organization which Amsterdam Municipality has joined.
Together with the Dutch public health service GGD, the council initiated a search for
a suitable pilot area. The research group in question, which initially focused on a
project on collaboration between formal and informal care providers, saw an
opportunity to receive additional financing by establishing a link between its original
fieldlab project and AFC. Perhaps as a result of these top-down preliminaries, no
elderly individuals, care givers or care institutions were initially consulted. The
neighbourhood was finally involved in the design and implementation phase. Several
elderly individuals are even playing a special part as they are being trained as co-
researchers and as such are interviewing their elderly fellow residents with regard to
their wishes and needs in the neighbourhood.31 The local agenda of the
neighbourhood, where the pilot is taking place, lists ‘aging’ as a current issue.32 In
other words, in this fieldlab an urban project (AFC), with assistance from a fieldlab
related research group, locates a suitable study area in urban district Amsterdam
East. A win-win situation for the Amsterdam Municipality, district Amsterdam East,
and the research group in question. But what about the elderly residents? Would an
earlier involvement of this local party have contributed to another or even better

The project Climate resilient Neighbourhood originated in the research group ‘Water

30 110 Dagen Plan Fieldlab Oost 2014; Majoor, Stan (2016). ‘Inleiding’ in: Werken in een gebied:
gewoon doen in Amsterdam. Amsterdam Municipality: Project Management Bureau
31 Veldboer, Lex. (2016), Hoe ouderenvriendelijk vinden ouderen hun buurt? Research proposal Age

Friendly City, Amsterdam University of Applied Sciences

32 Amsterdam Municipality (2016). Gebiedsplan Indische Buurt 2016. Consulted at

in and around the city’. At first, this was a low-priority theme within the local
administration.33 However, the involved professor realized the urgency and
complexity of the issue as early as 2014. During a fieldlab meeting he linked the
research group to the theme and, later, to the structural maintenance trajectory in this
neighbourhood. In this case, the ‘question’ arose out of what the research group
together with the urban district’s administrators could offer, in the form of a
collaboration at the neighbourhood level and a joint learning process focussing on
excessive water and heat stress.

The district Amsterdam East, the Space & Sustainability sector of the Municipality of
Amsterdam, the local Water Institute, housing associations and neighbourhood
organizations became interested. Once again, the role of local residents became
important mainly in the project’s design and implementation phases. In this case, an
AUAS research group went looking for support among urban stakeholders in East to
be able to start working with this theme. It also approached co-financers who might
be willing to invest in experiments aimed at exploring new approaches to urban water
management. In order to link potential project-specific co-financers to a fieldlab the
study’s added value needs to be clear from the start. Also, the project should
transcend its local character and local needs. In order to justify the investment, yields
in neighbourhood X should lend themselves to being transplanted to neighbourhood
Y. The co-financers were necessary to make this fieldlab project possible. This gave
them an important role in defining the issues and method.

A question-oriented strategy with the neighbourhood as a requesting party in the

preparatory stage of a fieldlab project turns out to be a complicated task. Actors
involved in the fieldlab, and in particular the research groups, function both as
demanding and as a supplying party. As such they fulfil a pivotal role in the project’s
first phase. The city council and the co-financers have an important stake as well.

4 Looking for balance

The examples described above show that it was not evident from the start whose
question was the focus of the fieldlab. Identifying purely bottom-up (resident) issues
from the start was difficult. There are at least four reasons for this. First, Fieldlab
East’s familiarity with local networks and neighbourhoods was initially too limited to
be able to arrive at a carefully considered selection of issues. The fieldlab did,
however, at a later point reach back to a few earlier identified local issues by means

33 Amsterdam Municipality (2016). Gebiedsplan Watergraafsmeer 2016. Consulted at

of neighbourhood agendas. Second, residents seldom speak with one voice.
Therefore, they can never be addressed as one single stakeholder. Third, the
neighbourhood is obviously not the only requesting party. Also, the district council
has its own questions and needs to which it seeks answers and solutions through the
fieldlab. Likewise, AUAS research groups are looking for areas or cases which they
can use to apply and evaluate new methods and new procedures. Finally, co-
financing – a necessary condition for starting a fieldlab project – is often decisive in
the selection of the area. In the case of Fieldlab East, question-oriented strategies
therefore equal multi-stakeholder strategies, in which the question ‘whose issues will
be addressed first?’ determines the ultimate outcome. Equally crucial is the decision
as to which stakeholders will be admitted to the interactive fieldlab trajectory and
when, and how much manoeuvring room they will receive.

In the case of Fieldlab East, it became apparent that the involved professors and
area-based public professionals had a significant role in the selection of the research
topics and study area. The residents, as ‘bottom up’ stakeholders, only played a
minor part in deciding the themes. They were more prominently present in the design
and implementation phases. A bottom up strategy in the first phase demanded
apparently more attention and local knowledge of the neighbourhood. For now, the
field lab team accepted this procedure; the justification of a question-oriented strategy
lies after all not (solely) in providing what residents as end users want, but (also) in
helping to solve a particular metropolitan problem. This requires a co-creation
process with all relevant parties.

In a fieldlab, the objective is to strike an acceptable balance for all requesting parties
in order to create a structural interaction between them which all can accept. If this
can be accomplished, a fieldlab as a community of practice is able to draw upon local
knowledge, to empower residents, to make a neighbourhood resilient and to offer
professionals a robust network in which they can make a difference. One more step,
and a ‘co-city’ emerges, characterized by co-creative administrative structures in
which multiple parties share a sense of collective ownership and take responsibility
for an area, a task, a common.34

In order to avoid sematic tricks and false expectations, a fieldlab team should make
clear from the beginning whose question is considered and in which phase. The two
examples from Amsterdam East show that the research group are the most present
and dominant partner in the start-up phase of a fieldlab. In an effort to improve the
question-oriented approach with a prominent role for the ‘locals’ from the start, I will
conclude by offering a suggestion to make this possible in the context of the AUAS

34 Ibidem

fieldlabs. Instead of substantial financial investments by AUAS, the municipality and
co-financers from the moment a research plan is approved, a professor interested in
doing research in the context of a fieldlab could apply for ‘seed money’. For a period
of six to twelve months, researchers and students visit a neighbourhood to explore a
number of local, ‘bottom up’ issues that have links with the research group, and build
a network. Following this preliminary study, the research group, on behalf of the
neighbourhood, and the urban district file a plan in which the three parties argue why
they require funding to carry out a one or two-year follow-up study of the issue. During
this stage, the fieldlab team and research group search for co-financers and convince
them that they have a relevant and widely supported case on their hands which will
benefit from more research and experimentation. In this way the question-oriented
method can become truly bottom-up.

Living Labs – an Approach for
achieving Sustainable Change
in City Logistics
Nina Nesterova1, Hans Quak1, Tariq van Rooijen1

1 Netherlands Organisation for Applied Scientific Research

Category: Research Paper

The Living lab is an approach that allows for improving the co-creation processes,
putting the end-users at the heart of innovation, realizing new business models,
looking for new roles for traditional stakeholders in the innovation process, and a
more directed innovation preparation and deployment process. This article describes
experiences of eight European cities in setting up and developing living labs in city
logistics within the CIVITAS 2020 CITYLAB project and draws upon the CITYLAB
Deliverable 3.4 “Practical guidelines for establishing and running living laboratories”.
It provides a distinction between logistics service living labs and city logistics
transition living lab. The article shares experiences on setting up and operation of the
living labs and added value from using living labs set up.

Keywords: city logistics, living labs, co-creation, sustainable transition.

1 Introduction

Environmental, societal and economic sustainability is in the heart of the transition

process that our society is undergoing today. UN and EU sustainability goals and
objectives address all emission intensive sectors, setting up clear targets that have
to be achieved in order to change current trends. Setting up transition pathways is
than responsibility of these concrete sectors, choosing the most appropriate and
effective strategies and methods.

In this framework there is an increasing interest in city logistics innovations, due to

the associated negative impacts on air quality, noise, livability, climate change and
use of urban space. Solutions to address these challenges are complex in terms of
different stakeholders and different interests involved. Inspite of the many trialed
solutions to address these challenges, no large-scale steps towards more
sustainable city logistics processes are yet set. Even if experiments are proved
financially and technically successful, scaling up, transferring and bringing significant
impact is lacking.

The living lab approach is a new for city logistics sector. It allows for improving the
co-creation processes, putting the end-users at the heart of innovation, realizing new
business models, looking for new roles for traditional stakeholders in the innovation
process, and a more directed innovation preparation and deployment process. In this
way, the speed of the transition process in urban freight increases as well as the rate
of successful city logistics innovation uptake. This paper presents the results of the
CIVITAS 2020 CITYLAB project (2015 – 2018) and draws upon CITYLAB’s
Deliverable 3.4 “Practical guidelines for establishing and running living laboratories”
(CITYLAB, 2018a). CITYLAB was stablished with the objective to develop solutions
that result in roll-out, up-scaling and further implementation of cost effective
strategies, measures and tools for emission free city logistics in urban centres by
2030. The project looked into the development of the city logistics solutions within
living laboratories in eight European cities: Amsterdam, Brussels, London, Oslo,
Paris, Rome, Rotterdam, Southampton. Research zooms in and builds up
conclusions on the experience with two types of living labs: urban transition living
labs and service/product-oriented living labs.

2 Living labs in city logistics

According to Maas et al (2017), nowadays a living lab can be referred to as “a concept

for achieving a long-term sustainable solution for a societal challenge by involving the
actual end-user. It can also be a method to achieve a certain objective by connecting

and involving the right stakeholders and follow an iterative design and learning cycle
called co-creation. Finally, a living lab can also refer to the context, related to the
organizational or geographical environment for the real-world experiments”.
Although the term ‘Living Lab’ is frequently used, referring to a wide variety of local
experimental projects of participatory nature, a shared definition is still lacking.
However, several key elements that are essential for a living lab can be determined:
experiments take place in a real-world environment; co-creation and end-user
involvement is essential for the process of innovation development; involvement of
multidisciplinary competences and multi-stakeholder participation; an iterative
learning cycle, referring to the process where innovations are developed following
“plan-do-check-act” cycles.

Depending on the initial objective of the Living Lab, these principles can be either
used to find a solution for a concrete problem via development, testing and upscaling
of an innovative product or service or to guide a transition process on the city /
neighborhood / street level. Looking into the city logistics environment, two types of
living labs can be distinguished:

• Logistics service living labs - the living lab concept is applied on the level of
the individual companies, working out logistics innovations;
• City logistics transition living labs - the living lab concept is applied on the city-
wide framework, helping out cities in making a transition to, for example, zero
emission city logistics.

If logistics service living lab follows the concepts and practices related to product or
service innovation research, city logistics transition living labs are closely connected
to the urban transition theories and their main characteristics are well illustrated by
Nevens et al (2013), Roorda et al (2014), Wittmayer et al (2014).
Table summarizes the main differences between these two types of living labs.

Table 1. Logistics service living labs and city logistics transition living labs

Logistics service living lab City logistics transition living lab

Short-term perspective targeted at specific Long-term perspectives targeted at big
solutions societal problem

Detailed oriented and narrower thematically. Shared ambitions and goals of the living lab
Often linked to a company’s objective or and linked to the city and / or country’s
business goals. objectives or goals.

Medium-term collaboration with a strong Long-term collaboration of quadruple helix

focus on the participation of the end-user in stakeholders and roadmap development and
the co-creation process co-creation of actions

Iterative cycles are focused on the Several different solutions are trialled in
development of the solution for one specific parallel, including different stakeholders, but
issue at hand. all contributing to the final living lab ambition
Clearly defined end-users: innovation End-users depend on the concrete solution
developed in this lab should meet their needs trialled

Usually driven by industry Usually driven by public authorities and

knowledge institutes
Source: CITYLAB, 2018b

Logistics service living labs are privately driven collaborations that aim to address
concrete operational problems or challenges that private stakeholders are facing.
These are, for example: improving efficiency of shipping process, addressing new
category of customers, developing and rolling-out new services. They are usually
initiated by private city logistics stakeholders with the main emphasis on improving
the services they directly control. The attention is on the process of the solution co-
creation with an end-user(s). Those are usually clearly defined from the beginning,
as developed innovation in the living lab should meet their needs. Logistics service
living labs have shorter ambition focus and are finalized once innovation roll out took

City logistics transition living labs are usually initiated by public actors or local interest
groups with the objective to guide transition process in the specific geographical area
(city, neighborhoods, street). A living lab is then an action-driven partnership where
local governmental stakeholders cooperate with industry, retail, commerce, academic
and societal partners and collaboratively develop new approaches and actions to
promote sustainable logistics. Shared ambitions and goals of the living lab are often
thematically and linked to the city and / or country’s objectives or goals, addressing,
for example such problems as: reducing emissions and noise; improving accessibility
and livability of the area; reducing congestion; improving safety.

Logistics innovations are developed within an ecosystem on the city level which can
facilitate innovation development process or act as a barrier for it. Political and policy
support for the urban freight, existence of efficient stakeholder communication and
cooperation platforms, monitoring and evaluation of urban freight solutions and the
existence of efficient knowledge transfer channels are defined as the key components
of the logistics living lab ecosystem (Nesterova et al., 2018). Existence of favorable
city logistics ecosystem provides local authorities, industry partners and knowledge
institutes with an opportunity to work more efficiently together on the local logistics
problems. This ecosystem facilitates the knowledge transfer within the living labs,
therefore new solutions can build up on the learnings from the previous trials.

Experiences in setting up and operating living labs in city logistics

During the three years of the CITYLAB project, valuable experiences were obtained
on setting up, operating, or trying to start living labs in city logistics. CITYLAB
investigated and contributed to the application of the living lab principles within both
types of living labs in city logistics:

• In Rotterdam, Paris and Southampton the project assisted city-wide transition

processes contributing to zero emission city logistics;
• In Amsterdam, Rome, Oslo, London and Brussels concrete solutions and
services were trialed and upscaled using living lab principles.
This section presents some of the experiences in working with and developing of
living labs in city logistics.

Setting up a city logistics living lab

For both types, the first step in setting up a living lab is to define clear ambition and
objectives, agreeing on the main reason of the creation of the living lab and on what
should be achieved as a living lab result. The creation of logistics service living labs
is usually initiated by a private party trying to address a specific problem. As a result,
the living lab ambition focuses on finding a solution for this problem. These ambitions
are usually short term and the time necessary to develop the innovative solution is
relatively limited. The city logistics transition living labs mostly emerge from the
existing and obstinate problems in the street, neighborhood or city. Several actors
together try to address the issues and to find common solutions. Often, the ambition
of a transition living labs is related to a long-term city development vision and is
supported by all living lab participants. Experiences from CITYLAB have shown that:

• The setting up phase of the living lab is very important: here you discover
things you might not have expected, as partnerships are relatively new and, in
this phase, you learn other participants’ values, interests and ideas related to
the jointly developed ambition and goals;

• During operation of the living lab it is necessary to regularly check whether the
ambitions and the scope of the Living Lab and the individual participants’
ambitions and interests are still aligned. Critical changes in the living lab itself
and its ecosystem that can influence the implementation process should be
monitored. For example, partners that were part of the common definition of
the living lab’s ambition and objectives can change jobs. As a result, the group
has to get used to a new person, and a new person has to gain confidence in
the process and the group;

• Take personal animosities between key figures of organisations into account.
Identify the risk of non-compliance from either of the organisations and take
mitigating actions when possible.

Once the ambition is agreed, the scope of the Living Lab within the real-world
environment have to be made clear. “Inviting the respective parties to engage in the
living lab’s real-world experiment is a promising option because public authorities,
companies, and others can be more willing to overcome established attitudes and
obstacles as long as it is ‘only’ in an experimental setting (…). The experimental
setting also encourages a critical attitude and the search for creative solutions” (Steen
& van Bueren, 2017). For the city logistics system, this implies the urban area, where
the geographical scope and location of the real-life experiments will depend on the
key goals of the (city logistics transition living) lab.

Logistics service living labs focus on solving a specific stakeholder’s problem and on
the users of the developed innovation. So the area is determined by the end-users of
innovative solution and can vary from a single building, street, neighbourhood(s) or
group of customers. It can be limited by the policy regulated zone (e.g. low emission
zone; congestion charge zone) or any other area or set of areas where this particular
innovative solution can be trailed out and has potential to be upscaled. For example,
within CITYLAB:

• Gnewt cargo and TNT were looking for a solution to increase their urban freight
transport with electric freight vehicles in inner London. The geographical area
was limited by the congestion charge area, providing financial benefits for
electric vehicles. This area became the natural geographical scope of the
CITYLAB living lab implementation.
• In Oslo the CITYLAB living lab implementation looked how to optimise the
internal logistics for shopping centres and how to reduce the vehicle waiting
time, thus decreasing surrounding congestion. Therefore, the real-life test
environment of the living lab implementation has been limited to a shopping
mall building and the neighbouring area.

For the city logistics transition living labs the city as a whole, or some particular
neighbourhood(s) or street(s) can serve as an urban test environment. The city
becomes an arena to trial out multiple experiments / innovations aiming at achieving
one overarching goal.

Creation of the core living lab team is a next step in setting up a living lab – this is a
group of people/organizations that are interested to collaborate on the development
of the living lab. To create this team, the living lab initiator has to contact potential
partners, keeping in mind: idea of quadruple helix (1); end-user involvement (2); and

“out of the box” thinking as regards additional competences necessary (3). For the
city logistics living labs considering the (1) category of stakeholders is specifically
important, while (2) and (3) should be decided case by case for the individual
implementations that might run in parallel and require different stakeholders.

City logistics issues are complex to solve as there is usually no single problem-owner
and stakeholders have various objectives and stakes. Therefore, the co-creation
process in living labs is important, as: it helps to align conflicting objectives and
identify the common ambition in the process; and it aims to increase participation of
the end-users in the development of the final product, and as a result increasing the
chances of it uptake. Within logistics service living labs co-creation means actively
engaging the end-users in the development of a new solution. This is a challenging
process: as noted by Steen & van Bueren (2017), end-users “typically do not have a
professional motive to participate in innovation processes and participate on
voluntary basis”. In London CITYLAB implementation benefited from the active
involvement of industrial and research partners as well as municipality. University of
Westminster, a research partner in the living lab process, performed ex ante and ex
post evaluations, based on which decisions about new cycles in the solution
development were taken. Involvement of the municipality in the living lab process
made it possible to efficiently find necessary space for the location of the Gnewt cargo
hub(s) within congestion charge area. This is strategically important question for
London, where there is a high space scarcity. This cooperation between government,
industry and researchers has proven highly successful in London.

In city logistics transition living labs the combination of the quadruple helix
stakeholders is a pre-requisite to guide transition. The representatives from these
stakeholder groups can form “frontrunner” groups, bringing together “visionary people
from various disciplines who are willing and able to engage in a creative process
towards a long-term conceived future for a sustainable city” (Nevens et al., 2013).
The frontrunners develop transition ambition and objectives and the roadmap
towards it in time. Together they set up sustainable, user-friendly, financially feasible
and, in the end scalable and / or transferable city logistics solutions. In Southampton,
the Southampton City council, Meachers Global Logistics, Southampton General
Hospital and the two universities (Southamton Solent University and University of
Southampton) came together with a common ambition to improve local air quality by
promoting best practices in sustainable logistics and reducing their respective
transport footprints. A Memorandum of understanding was signed stating partners
commitment for this cooperation process. The University of Southampton acts as the
neutral co-ordinator of activities. Existing business and personal relationships
between the parties, emanating from the original memorandum of understanding
have been key to continue the living lab and develop and explore new ideas: e.g.
consolidation initiatives, joint procurement and electric fleet adoption. Other

experiences from the CITYLAB projects have indicated that:

• Involve external stakeholders, users, customers as much as possible from the

very beginning in the living lab process. Living labs should create value for all
stakeholders, which makes it is easier to have stakeholders’ commitment
throughout the whole process.

• Convincing stakeholders to actively join a living lab from the beginning – where
a lot of things still have to be defined and developed - requires some
persuasiveness and vision.

• Co-creation with end-user is in the heart of the living lab approach.

Participating in the development of the living lab solution, the end-user
becomes more responsive in adopting these ideas;

• It is crucial for a city logistics living lab to be successful that all partners see a
potential benefit from participating.

Understanding the drivers, interests, culture and way of working of all parties related
to the Living Lab is important and might help with their continuous involvement in and
their commitment to the living lab for a longer period. Once the core team is created,
it is necessary to analyse what kind of external parties that could help living lab to
achieve its goals are missing.

Selecting an appropriate governance model is a next step in setting up a living lab

in city logistics. There is variety of possible governance models for living labs, the key
is to find a way that works for the living lab in each specific case: a memorandum of
understanding, informal agreement, working groups set up, a covenant, etc. The
forming of freight partnerships or frontrunner groups could be a good start for a city
logistics transition living lab. In CITYLAB:

• The living lab in Southampton is set up around an informal Memorandum of

Understanding (MoU) between the Southampton City Council and the
University of Southampton on sustainable logistics. The main objective of this
agreement is to reduce overall vehicle emissions and improving air quality
standards. It is desirable to have as many stakeholders as possible sign up to
the MoU.
• The living lab in Rome is set up around the European research project
CITYLAB. Initially there were limited opportunities of collaboration between
urban freight stakeholders and Rome CITYLAB living lab implementation
acted as an opportunity to start the work on reinforcement of cooperation

between industry, research and city authorities on the question of urban
• In Oslo there was no formal agreement between the stakeholders included in
the living lab. The working group and participating stakeholders were defined
based on business decisions. However, additional stakeholders outside of the
company were included through regular meetings and workshops.
• The Rotterdam living lab is set up around the local covenant (a local Green
Deal on Zero Emission City Logistics), which was signed by the city, research
institute TNO and front running transport companies from varying logistics

After the core living lab team is in place and its commitment to work on commonly
agreed ambition and goals within living lab set up is formalised, the operation of the
living lab can start. Initial preparatory steps will include: analysis of city logistics
ecosystem; identification of potential ideas and solutions to develop within a living lab
and development of monitoring and measuring system.

Operation, evaluation and going through the cycles in living lab

Within living labs development of innovation follows a cyclical approach, where plan-
do-evaluate – act phases are consequently used until the innovation is considered
as ready to roll out or decision is taken to stop testing process. Each cycle, within a
logistics service living lab can be continued into a new loop (when improvement of
existing solution is necessary) or interrupted because the solution is considered as
not interesting. During a cycle also, a new innovation idea can be born and be than
developed within another implementation case. In Amsterdam the CITYLAB living lab
implementation went through several cycles. The first cycles have assessed an initial
idea (developed prior to CITYLAB project) where parcels were navigated in the city
by a vessel (the floating depot) and from there distributed by clean vehicles to final
destinations; this turned out to be difficult for several reasons (e.g. moving of
premises to start from, developing the floating depot, finding landing places at the
canals, finding suitable volumes, etc.). In the following cycles, PostNL considered the
possibility to use a floating depot pushed by a hybrid-push boat from where zero
emission (ZE) vehicles (EV trucks or bikes) would deliver parcels in the ‘de Pijp’ in
Amsterdam, supplying pubs, restaurants and hotels with fresh food items. Ex ante
evaluations and end-user consultations have indicated that there is no yet a paying
customer that is ready to support this. The next cycle that was tested in the real life
and resulted in the local and national (by Post NL) upscale of the solution is partial
replacement of PostNL vans in the city centre of Amsterdam with specially designed
e-freight bikes that distribute mail and parcels from micro-hubs located in the city
centre. The overall objective: more sustainable organization of PostNL’s city logistics
operations did not change during the cycles.

Other CITYLAB experiences have shown that:

• Minor adjustments can make a large difference. In some living labs it has been
crucial to make small adjustments to the business models as the
implementation has developed over time. This reflects the willingness of
organisations to make operational changes to logistics practices in favour of
sustainability when the outcome, although positive, will inherently impact
(potentially negatively) on customer / client experience.

• Evaluation should be an on-going process, both on the process of the

implementation cases as of the city logistics living lab.

• The living lab cycles for innovations should follow the natural development
process and not be forced in fixed time frames. The living lab process is
guiding the cycles.

• One should be able to recognize the act phase and go to next cycle.

At some extent, living lab approach requires changing usual ways of working for
partners. CITYLAB have shown, that:

• Even if the collaborative mechanisms are already existing in the cities, it takes
time to influence and steer towards the principles of working and innovating in
a living lab;
• It is difficult for people to work in another way: the living lab approach requires
a certain mentality change to more open-minded and open-end development
of the solutions, whereas many professionals are used to plan projects and
results in advance.

Learning from the negative experiences contributes to the positive experience and
building up on this is a skill we need to develop in order to increase the innovation
uptake level. Not all of the solutions, developed within CITYLAB according to the
living lab principles were directly successful. The first cycles in Rome, Amsterdam,
London and Brussels of the implementations did not result in the implementation roll
out, but have produced a valuable knowledge and strong cooperation structures to
move forward for the next living lab cycles. As the result sustainable city logistics
solutions developed within the London and Amsterdam implementations have rolled
out. The Rome implementation within a second living lab cycle is extending the
implementation in terms of flows involved, sites and alternative recyclable / reusable
waste. The transferability effect is also achieved on the city level, where City of Rome
wants to use CITYLAB case as a “test-case” to show all the benefits derivable from
the adoption of a living lab approach where stakeholders collaborate, create, validate

and test innovative technologies, services, products and systems. It intends to make
use of the outcomes of the CITYLAB experience to identify the most prominent
innovative freight solutions to be included in the upcoming Urban Logistic Plan and
Sustainable Urban Mobility Plan (2018), with the local knowledge partner (UoR)
providing support. This mentality switch in approaching innovative solutions is key for
the future of the innovations in city logistics. The city logistics projects are no longer
only evaluated on the direct success from its implementation, but are seen in the
broader perspective of added value from the establishment of stakeholder
cooperation processes and contribution to the long-term goals of public authorities
and market players.

Added value from the living labs approach in city logistics

Living labs in city logistics put urban logistics on the strategically important place,
attracting attention of the policy-makers, knowledge institutes, research partners and
citizens. CITYLAB illustrated that different stakeholder groups benefit differently from
participating in living labs in city logistics (CITYLAB, 2017, 2018c).

For the city authorities, urban transition living lab gives an opportunity to reach a
bottom-up policy coherence, including in the policy and decision-making process the
needs and aspirations of local and regional stakeholders, industrial parties, transport
operators and citizens. Overall, better knowledge and understanding of the urban
freight in the city is created. Creation of frontrunner group with representatives from
different sectors helps to gain a common perspective on the city logistics and better
understand needs and requirements of transport stakeholders and logistics operators
and barriers they are facing, thus, developing more efficient and targeted policies.
Joint collaboration in the development of the solutions supports short and long-term
policy planning and, through evaluation, provides a feedback on the effectiveness of
the policy measures. Municipalities “could support living labs by selecting zones
where its efforts are aimed at creating room in the public regulations allowing bottom
up initiatives and innovations” (Steen & van Bueren, 2017). Policy-makers bring in
decision-making power into the living lab process as well as possibilities to create
conditions facilitating development of innovations. They also can influence increasing
of investments in the innovations for city logistics.

For transport operators, logistics providers, retailers and other private stakeholders
in city logistics processes, being part of the frontrunner group provides an opportunity
to influence, at certain extent, the policy making process of municipalities. Next to it,
it gives an efficient access to the most recent domain knowledge and evaluation of
its own activity efficiency, providing an input for the improvement of the business

cases and potential for the innovation uptake. It also gives opportunity to better
understand other market players and their interests. Overall, if certain transitions are
to take place, being a part of the frontrunner group, helps companies not to be forced
into the changes, but to co-create the shape of these changes together with other
parties. Living labs facilitate a cooperative mentality switch in the supply chains:
potentially competitive businesses are no more seen as competitors but as partners
working together to achieve a common goal.

Both, city and industrial partners benefit from the added value brought by the
research partners in the living lab process. Along with innovative ideas they bring in,
research partners also provide a neutral opinion on the relevance, efficiency and
sustainability of the tested solutions. Research partners can do a background
literature and best practice research; undertake scoping and feasibility studies for the
industry partners for minimal cost as part of managed student projects. They can act
as secure data manager on behalf of the partners, undertaking ex ante and ex post
analysis and providing longer term evaluation of any implemented measures. A
research partner is also well positioned to be a neutral coordinator of the living lab in
city logistics, specifically for the city logistics transition living labs. Being involved in
the living lab, provides researches with cost efficient access to the first hand data and
with an opportunity to validate their ideas with market players. New research ideas
are generated via the stakeholder discussions and by going through the innovation

Stakeholder collaboration developed within living labs provides an opportunity to

build relationships and establish joint initiatives in city logistics that otherwise would
not have taken place.

3 Conclusion

City logistics is dynamically developing sector which gets increasing attention from
public authorities due to its negative impacts on the livability of the cities and quality
of life of the citizens. Living Lab concept offers a way to address complex multi-
stakeholder topics: it changes the emphasis from the solution as an isolated object
to the process of integration with its environment. Finding a solution becomes a
process involving many stakeholders with different objectives and interests within a
dynamic environment. It allows the creation of experimentation environments that are
sufficiently connected with real world stakeholders and their business models, to
allow near-simultaneous development and deployment. This requires a medium or
long-term approach, adaptive and pro-active planning and steering.

City logistics is the system and process by which goods are collected, transported,
and distributed within urban environments. It is a sector where solutions often ask for
a multi-stakeholder approach, bringing together different, sometimes not aligned to
each other interests. Due to the high dynamics in city logistics, it is unsure in advance
what type of solution will best fit with problems faced. The creation of living labs in
city logistics provides a new way to develop and address different trends and
challenges in urban freight. It supports an action driven cooperation forms fostering
innovation deployment and improving communication and cooperation between
stakeholders. Development of the shared vision, aligning individual interests to
common goals and active involvement of the end-users as well as other
competencies in the co-creation process helps to develop innovative solutions that
are more user-friendly, more financially sustainable and adapted/tested within a real-
world environment.


CITYLAB, 2015. Practical guidelines for establishing and running a city logistics
living laboratory (No. 3.1). CITYLAB. Retrieved from http://www.citylab-
CITYLAB, 2017. CITYLAB: lessons and experiences with living laboratories (No.
D3.3a-e). CITYLAB. Retrieved from
CITYLAB, 2018b. Tools for achieving CO2-free logistics in cities by 2030 (No. 6.4).
CITYLAB. Draft version.
CITYLAB, 2018a. Practical guidelines for establishing and running living
laboratories. (No 3.4.). CITYLAB. Retrieved from http://www.citylab-
CITYLAB. 2018c. Report on living-lab transferability activities (No. D6.1). CITYLAB.
Retrieved from
Maas et al, 2017. Maas, T., van den Broek, J., & Deuten, J. Living labs in
Nederland - van open testfaciliteit tot levend lab. Den haag: Rathenau
Neef et al., 2017. Neef, M. R., Verweij, S., Gugerell, K., & Moen, P. L. Wegwijs in
living labs in infrastructuur en ruimtelijke planning: Een theoretische en
empirische verkenning. Groningen: Rijksuniversiteit Groningen.
Nesterova et al., 2017. Nesterova, N., Quak, H., Rooijen, T., Cherrett, T., & Mcleod,
F., 2017. City Logistics Living Labs – an ecosystem for efficient city logistics
innovation uptake.
Steen, K., & van Bueren, E., 2017. Urban Living Labs - A living lab way of working.
Amsterdam: Amsterdam Institute for Advanced Metropolitan Solutions.
Wittmayer et al (2014), Wittmayer, J., Roorda, Ch., Steenbergen, F. van, Governing
Urban Sustainability Transitions – Inspiring examples, October 2014
Nevens et al (2013), Nevens, F., Frantzeskaki, N., Loorbach, D., Gorissen, L.,
Urban Transition Labs: co-creating transformative action for sustainable
cities, Journal of Cleaner Production, 50, 111-122.
Roorda et al (2014) Chris Roorda, C., Wittmayer, J., Henneman, P, Steenbergen,
F. van, Frantzeskaki, N., Loorbach, D., Transition management in the urban
context: guidance manual. DRIFT, Erasmus University Rotterdam,
Rotterdam, 2014.

Exploring the market and sustainability offers
of Living Labs in Germany
Justus von Geibler*1, Julius Piwowar1, Annika Greven1

*Corresponding author
1Wuppertal Institute for Climate Environment and Energy

Category: Research Paper

For more than 10 years Living Lab are emerging as new social participatory structure
engaging societal stakeholders (such as public administrations, companies, civil
society and research organizations) in collaborative environments. This paper
presents the results of a mapping and assessment of living labs in Germany being a
part of the national innovation system. The research objective of this paper is to
explore the German living lab market and its offerings, which can support the
innovation process of start-ups and SMEs; in a particular for the development of
sustainability innovation. Beside a geographical mapping the following characteristics
of living labs have been assessed: (a) degree of institutionalization; (b) sector; (c) key
actor, and (d) services offered. Nine different services have been identified and
assessed based on literature review and evaluation of living lab websites. The
services cover customer-oriented and user-integrating services as well as
sustainability assessments. The results of the mapping were validated and discussed
with research- and practitioners experts in three workshops. We conclude that living
labs can be seen as an emerging element within in the national innovation system
capable of driving sustainability innovations based on the services offered.

Keywords: Living lab, mapping, services, open innovation, user-centred design,


1 Introduction

Research and innovation are key to realize a digital economy and society, which aim
for growth, competitiveness, wealth and sustainability. At the same time innovation
actors face the challenge to productively use the increasing dynamic and complexity
of societal change within the innovation process. For example, globalization and
digitalization increasingly connect activities in various domains such as mobility,
living, retail, health and work to form the field of Smart Living. Therefore,
corresponding innovation processes are no longer industry-specific, but rather
overarching, integrated and lifestyle-specific. New social practice and competitive
advantage can be co-created based on participatory and open-innovation
approaches, which actively engage technology developers, designers, producers,
service providers (public or Private), as well as users and citizens. Examples of these
co-creative processes can be found in living labs which are emerging as new social
participatory innovation infrastructure (e.g. on Smart Home, Ambient Assisted Living,
Smart Data or e-mobility) for more than 10 years (Leminen et al. 2017).

Since then, the living lab approach has gained recognition among innovation
practitioners (Salmelin and Curley 2015; Curley 2016; Ley et al. 2015, Schuurman
2015). The concept of “living-labs-as-a-service” has been discussed as support for
innovation processes in companies, big and small (Ogonowski et al. 2015, Bódi et al.
2015; Ståhlbröst 2013). Ståhlbröst (2013) defined the concept as “the offering of such
services such as designing the idea-generation processes, planning or carrying out
real-world tests of innovations, and pre-market launch assessments”. Refering to
Schuurman’s (2015) model of living labs, Schuurman and Herregodts (2017), define
‘Living-Labs-as-a-Service’ as follows: “Living Lab organizations that have developed
a specific project process or methodology aimed at entrepreneurs to assist them in
their innovation process. These entrepreneurs, sometimes referred to as utilizers of
the Living Lab, engage in a customer-client relationship with the Living Lab to get in
touch with (end-)users to help shape their innovations.” Based on such services living
labs can offer companies a structured open innovation processes, for example with
the aim to avoid risk of insufficient market acceptance or unexpected user behaviour
(Buhl et al. 2017).

Living labs address real-world challenges in early stages of the innovation process
due to methods (and services) for user integration. They thereby open perspectives
for the improvement of sustainability innovation and transformative research
(Schäpke et al. 2018, Keyson et al. 2017; Liedtke et al. 2012, 2015). For sustainable
societal transition this is crucial, since the effectiveness of sustainability innovations
is limited by at least three set of challenges: First, many innovations with high
sustainability potential fail because of a insufficient market acceptance (Schwartz et
al. 2014) e.g. innovation pay too little attention to the desires, needs and practical

purposes of users (Franz et al., 2006, Brynjarsdóttir et al., 2012). Second, innovation
do not meet the initial expectations of their sustainability impact because of
unexpected patterns in the actual usage e.g. resources liberated by energy efficiency
savings are used for further consumption (so-called rebound effect) (e.g. Madlener
and Alcott 2011). Hence, rebound effects and failed technical adaptation to the real
life and work environments of users and applicants provoke inefficiency and
ineffectiveness, so that physically possible potentials cannot be implemented and
realized (Meijers et al. 2014, Buhl et al. 2017). Third, actual behaviour often deviates
from ‘green’ attitudes inducing the “knowledge-to-action-gap” or “attitude-behaviour
gap” (e.g. Terlau and Hirsch 2015).

In order to address these challenges a sustainability-oriented living lab can expand

the conventional innovation process towards openness and a human-centred as well
as practice-oriented innovation approach involving real world experiments and rapid
prototyping within early innovation phases (e.g. Ullman 1997). This provides
opportunities to identify more appropriate solutions and decrease financial costs
(dilemma of product innovation, see Figure 1). Furthermore, several sustainability
studies (e.g. Bódi et al. 2015) demonstrate the large resource efficiency potential of
user-centred innovations, especially for small and medium sized enterprises.

Figure 1. Dilemma of Product Innovation (left) and opportunities based on prototyping (right)
(adopted from Ullman 1997)

Despite the potential living lab for market and sustainability innovation the living lab
landscape in Germany has been described as heterogeneous field with soft contours,
not very visible in the international field (Geibler et al. 2014). As a consequence, this
paper addresses the question how living labs in Germany and their services can be
identified and mapped in order to better understand the German living lab landscape.
Based on a 3-step approach (section 2), we identify and geographically map living
labs in Germany (section 3). Furthermore, we explore living lab services based on a
literature review and an analysis of the identified websites. Research experts and
practitioners have validated the results in workshops, email survey and interviews.

Conclusions (section 4) will be drawn regarding the opportunities to strengthen the
living lab approach and sustainability potential within the national innovation system
of Germany.

2 Methodology for Mapping Living Lab and their Services

This section provides the working concept of living labs and the methodological steps
of the mapping and service evaluation the living labs and their services in Germany.

The Sustainable Living Lab concept and levels of analysis

Originally developed at the Massachusetts Institute of Technology (MIT) (Pierson and

Lievens 2005) living labs gained increasing attention with regard to their potential to
support sustainability innovation (Keyson et al. 2016; Ley et al. 2015; Liedtke et al.
2015). According to Meurer et al. (2015) there are three main constituting
characteristics of sustainable living labs:

i) To strive for a real-world environment: a context-related setting

(households, city districts, organizations, public housing etc.) facilitates
exploration on usability, user experience and user context. This implies long-
term investigations during usage phase to provide feedback possibilities and
learning loops. There are three levels of real-world environment: (a) the virtual
simulation of the real world incorporates a low level of real world integration
(e.g. in the test bed of Bosch, Germany); (b) the physical simulation of the real
world (e.g. at the inHaus Centre in Duisburg, Germany) and (c) the real-world
stakeholder integration (e.g. at Praxlabs Siegen, Germany). Living labs differ
from pure (d) „laboratories of reality“, because innovations are not immediately
implemented in the real world on a large scale, but they are tested and
developed in a small-scale real-world laboratory beforehand. Thus, they
reduce liability risks and problems of maintaining the service during the large-
scale implementation and contribute to the overall confidence in innovation

Living lab focus in the project

Virtual simulation Physical Experiment in Implementation

of real worlds simulation of real the real world in the real world
Test enviroment worlds Designing & Implementation
Innovations- testing together together with
laboratories with stakeholders stakeholders

Increasing representation of the real world

Figure 2. Representation of the real world in living labs (based on Meurer et al. 2015)

ii) The participative involvement of users/citizens is a decisive success factor

to implement a living lab structure. This indicates to involve users at an early
and steady stage within creative and co-creative processes including open
idea generation, experimental concept development, prototyped
implementation, and an evaluation in the field of practice.

iii) Sustainable development defines a lifecycle-based perspective to individual

social and environmental-compatibility focusing on ecological and resource
friendly approaches with sustainability criteria which aim to contribute to
production and consumption patterns that can be applied on the global and
long-term scale and are inter- and intergenerationally viable. This
sustainability perspective aims to enable the development of sustainability
innovation e.g. by supporting innovators with sustainability thinking and
assessments along the entire innovation process.

Schuurman (2015) specifies three levels of a living lab, as outlined below.

i) The micro or user activity level, where the various assets and capabilities of
the living lab organization manifest themselves as separate activities where
users and/or stakeholders are involved.
ii) The meso or project level, where living lab activities take place following
mostly an innovation management process with organization-specific
methodology in order to foster innovation.
iii) The macro or organizational level, where the living lab is a set of actors and
stakeholders organized to enable and foster innovation, typically in a certain
domain or area, often also with a territorial link or focus. These organizations
tend to be Public-Private-People partnerships (Leminen 2013).

By analysing not only single living labs but a number of living labs being a part of a
national innovation system (Nelson 1993), the mapping described in this paper

expands Schuurman’s three levels with an additional level of living labs:

i) the meta level or innovation system level, where a number of living labs
and their interconnections are a part of the research and innovation system
within in a region or nation. An example of the meta level are living lab
networks such as the European Network of Living Labs (ENoLL) which has
been established as international federation of benchmarked living labs in
Europe and worldwide.

Research steps and methods

The objective of the paper is to explore the market of living labs in Germany and its
offerings to Start-ups and SMEs to support the development of innovation; in
particular sustainability innovation. The research question is: What are active living
labs in Germany and what are their service offerings for start-ups and SMEs? Based
on the research question there are three research steps: 1: Classifying and mapping
of living labs and other organisations that offer living lab services, Step 2: classifying
and mapping of services; and step 3 validation and discussion with experts.

Step 1: Classifying and mapping of Living Labs and other Organisations that offer
Living Lab Services in Germany
The mapping of active living labs in Germany based on the broad working definition
of living labs (see section 2), three key references and a Google search (Table 1).
The scope of the mapping covers organizations, which are not named as a “living
lab”, but involve parts of key living lab activities and can contribute to the development
of (sustainability) innovation. This allowed covering e.g. Start-up Accelerators,
Company Builder and design agencies. For the following these are all named as living

Table 1. Sources for mapping Living Labs in Germany

Source Description

Geibler et al. 2014 Living Lab directory identifying 76 living labs in Germany and
neighbouring countries.

Meurer at al. 2015 The working paper includes the definition of living labs in a
Green Economy.

European Network of International federation of benchmarked living labs in Europe
Living Labs (ENoLL) and worldwide, founded in November 2006. It includes over 150
2016 active living labs members worldwide (409 historically
recognised over 11 years) in different domains (3 German living

Google search Search with keywords of the living lab working definition

The assessment of living labs in Germany focusses on the macro and meso levels of
living labs defined by Schuurman (2015). Table 2 provides an overview of the criteria
used. At the macro or organizational level, the mapping distinguishes three main
aspects: institutionalization, sector and key actors. Based on the project background
the mapping emphasised on three sectors: living, retail and mobility. Accordingly, the
mapping not fully covers all living labs of the sectors industry, health and nutrition. At
the meso or project level, the mapping refers to the services offered by the living lab.
The micro or user activity level is not explicitly addressed in the mapping.

Table 2. Classification Criteria of Living Labs

Macro level Meso level

criteria criteria

Institutionalization (2) Sector (4) Key Actor (4) Services (9)

Institution/organization Living Private sector User-oriented

services (4)
Project-based e.g. in Retail Research/University
research project User-integrating
Mobility Civil society Services (4)

Industry Politics Sustainability (1)

Health and

Step 2: Classifying and Mapping of Services

In order to assess the market offerings of living labs nine service segments were
identified based on literature (especially Ogonowski et al., 2015, Schuurman 2015,
Bódi et al. 2015; Ståhlbröst, A., 2013; Geibler et al. 2014, Liedtke et al. 2015) and the
exploration of existing market offers. The segments are distinguished in three types
of services, depending on how their intensity of user-engagement and sustainability-
orientation. Customer-oriented services include limited-user involvement (limited

user openness), user-integrating services as well as sustainability assessment
services such as life-cycle-assessments (see Table3).

Table 3. Service typology for Living Labs (own illustration)

Service type Service segment Description

Support to collect user feedbacks for
prototypes in showrooms.
Support to analyse with quantitative and
User studies qualitative research methods target user
Customer- groups.
services Support to generate and develop business
Business model
ideas (models), e.g. within a workshop
setting (business model ideation).
Stakeholder networking Support to connect and engage with
and brokerage relevant stakeholder.
Support to co-design innovative product
Co-design concepts with relevant stakeholder and
potential user (e.g. participatory design).
Support to co-design prototypes with
relevant stakeholder and potential user.
integrating Support to co-design, test and evaluate the
UX testing and
usability and user experience of products
services evaluation
and services.
Support to co-design innovative product
concepts, which consider intended user
Motivational design
behaviour and the motivation of users (e.g.
by gamification principles).
Sustainability Support to analyse sustainability potentials
assessment and of innovative product and service concepts
services evaluation based on life-cycle assessments.

Customer-oriented services focus on the customer and include little user-

involvement e.g. through (1.) user studies and (2.) showrooms. Customer-oriented
services include also (3.) business model development (e.g. Osterwalder and Pigneur
2009; Gassmann et al., 2013) and (4.) stakeholder networking and brokerage services
to engage multiple stakeholders from the innovation ecosystem and to drive
collaboration and knowledge exchange (e.g. Chesbrough 2003).

User integrating services include (5.) co-design, (6.) co-prototype and technology
development (Meurer et al. 2015, Curley 2016), and (7.) user experience testing and
evaluating (e.g. Hassenzahl 2010). These services utilize feedback mechanisms

within the complete innovation process and at an early stage through co-creation
methods and the lab infrastructure. The (potential) user is guided to become a co-
designer and tester of new applications or processes. In addition, these services
include in-depth user experience investigations to support (8.) behavioural
considerations (creativity at work, working motivation, sustainability at work). This
involves intense direct interaction with people, to understand their mind set, and derive
solutions reflecting their needs and desires as well as promoting behavioural change
(e.g. Lockton et al. 2010; Chou 2015).
Furthermore, services cover (9.) sustainability assessment services (e.g. Hansen
et al. 2009; Ness et al. 2007; Geibler et al. 2016). They support the analysis of
sustainability potentials of innovative product concepts taking a life-cycle perspective.
For the mapping of services and to assess the services offered, the living lab websites
were screened and evaluated with a two-level scoring: Services are communicated
and mentioned at the website (score 1) or services are not communicated at the
website (score 0).

Step 3: Validation and discussion with experts

The results of the living lab mapping were discussed with research and practitioners’
experts to validate the findings. The expert selection criteria were: (former) leader
and/or strategist from active living labs in Germany and living lab researcher form
research institutes. Furthermore, the focus was on the sectors sectors retail, living and
mobility. In three workshops experts were asked to feedback the mapping and service
evaluation. The results have also been validated and discussed based on an email
survey and in interviews with the living lab operators.

3 Results

Mapping Living Labs in Germany

In total 99 active German living lab were identified, which divide into project-based
labs (44) and institutionalized labs (55). They are active in different sectors: living
(48), mobility (40), retail (31), industry (24), and health and nutrition (13) (see Figure

Figure 1 Mapping and characterisation of living labs in Germany

Living Lab Services in Germany

Out of the 99 living labs identified, 72 living labs have service offers in the area of
customer-oriented and user-integrating services or sustainability assessments. 27
living labs provide no services or do not provide related service information on their
homepage. Table 4 presents the result of the analysis for the degree of
institutionalisation and service offers of living labs in Germany (see Annex for detailed

Table 4. Living Labs in Germany: Degree of institutionalisation and service offers

Degree of Customer-oriented User integrating

sation Services* Service*

Business model

networking and

and technology


User studies

UX Testing/
Co -Design



44 55 34 49 37 60 23 34 46 8 32

driven 13 33 15 23 21 31 21 14 23 5 8

driven 33 25 26 26 17 31 22 22 31 4 13

Driven by
other actors 8 5 7 5 5 10 7 6 4 0 9

sector 14 17 10 13 11 19 11 7 12 4 6

sector 20 28 24 25 21 36 22 21 29 6 18

sector 17 23 12 18 17 26 15 20 24 5 8

sector 10 14 10 12 8 11 10 11 4 12 3

Health and
4 9 7 5 5 9 7 8 3 9 3

Note: * Double counting possible

The analysis highlights that the identified living labs driven by Research/university
include higher numbers of project-based Labs whereas Private-driven living labs,
show a higher level of institutionalization (including the new sub type of accelerator
labs; see discussion with experts, section 2). It is notable that number of identified

living labs driven by societal actors is low because the focus of the study has not
been this type of living labs. The few societal living labs in the analysis are exemplary
to support the definition and principle classification of key actors.
When looking of the overall results of the service evaluation it demonstrates that
customer-oriented services are offered more often compared to user-integrating
services. Most services relate to field of stakeholder networking and brokerage
followed by user studies and UX testing and evaluation. Few services were found in
the field of motivational design, co-design and sustainability assessment services.
When comparing the different sectors, it is apparent that living labs with a focus on
the sector “living” offers most of the sustainability services.

Discussion with experts

The discussion of the living lab mapping at the three workshops can be summarized
by the following aspects:
• The presented results of the mapping and service evaluation received a
positive feedback: The geographical mapping is an attractive way to present
the existence of the larger number living labs. The identified living labs in the
sectors living, retail and mobility cover the current landscape of Germany, just
a few living labs were complemented by the workshop participants, such as
„Obi Düsseldorf“, „Saturn Ingolstadt“, „Inspiration Store“ (Bremen) „weShop“
in Munich, „Innovation Store“ (Bonn). Living labs from other sectors e.g.
industry, health and nutrition were brainstormed with experts but do not cover
a complete list of living labs.

• The living lab mapping could include start-ups oriented lab structures because
they are innovative and embrace experimentation. In consequence the
mapping was extended to the type “accelerator living labs”, which focus on
start-ups and innovation (comment from the workshop on mobility).

• In order to further specify the results, the living labs mapping could address
more intensively on pilot projects (comment from the workshop on mobility).
• Private driven living labs can oppose the open innovation approach services
in order to protect innovation (comment from the workshop on mobility).

• Retail innovation (e.g. technology, marketing, POS Gestalt) and related

consumer behaviour is part of testing and observation projects in real markets
but does not involve active user feedback on particular innovation needs
(comment from the workshop on retail).

The email survey of living lab operators resulted in minor correction of assessments,

e.g. some service offerings have been deleted or added.

In discussion with living lab operators the need for action for innovation policy has
been identified to support the innovation potential of the living lab infrastructures in
Germany (Geibler and Erdmann 2017). This includes the following:
§ Support for the research and innovation system through the strategic
positioning of living labs. For this, a support programme with the following
action points is necessary: a) The improved networking of living labs and other
key stakeholders, b) Harmonisation and professionalisation with respect to the
methods used and the range of services offered, e.g. for start-ups and SMEs;
c) The establishment and development of consumer and household panels

§ The creation of new potential for innovation through sustainability and a user-
oriented focus in innovation policy. Particularly the following measures with
respect to innovation policy where called for: a) Integration of the UN
Sustainable Development Goals (SDGs) and user-centred innovation
approaches (e.g. through living labs and laboratories of reality) as leading
criteria in innovation funding and innovative public procurement; b) expansion
of the streamlined allocation of small levels of funding (“Fast Track to
Experimentation”) for start-ups and SMEs to support creative stages of
development and experimentation; c) significant enhancements of action
programmes for socio-technical or non-technical innovations.

§ The establishment of integrated data and knowledge platforms for knowledge

transfer on Smart Living and Smart Cities, with the following emphasis: a) The
gathering and transfer of knowledge and experience with respect to user and
stakeholder integration in open innovation processes; b) the preparation of
knowledge and a data base for more effective innovation processes regarding
future-oriented technologies (e.g. artificial intelligence, the bioeconomy,
Industry 4.0, the Smart Home, an ageing society); c) development of
educational and informational material on Smart Living.

4 Conclusion

The mapping of German living labs and their services provides an unique overview
of the research and development infrastructures in Germany with regard to user-
driven and sustainability-oriented innovation potential. Nine living lab services have
been identified, including customer-oriented and user-integrating services, as well as
sustainability assessments. Based on such services Living labs can offer a structured
open innovation processes, for example with the aim to avoid risk of insufficient

market acceptance, unexpected user behaviour, or undesired environmental effects.
The analysis highlighted that a high share of living labs in Germany offer services.
Customer-oriented services are offered more often compared to user-integrating
services. Services often offered relate to fields of stakeholder networking and
brokerage and user studies. Few service offers were found in the fields of
motivational design, co-design and sustainability assessment services. Sustainability
assessment services offerings are limited and could be improved, e.g. by the stronger
reference to methodologies or quality standards.

The results highlight a number of other opportunities to strengthen the living lab
approach for sustainability. Firstly, the geographical mapping updates earlier
mapping exercises and – in combination with the assessment of living lab services –
offers an attractive way to present the opportunities of living labs for actors related to
innovation development processes, including small and medium sized companies
which do not have the capacity for setting up and maintaining own research and
innovation infrastructures. The mapping provides data for a more interactive
presentation of results in an online tool. Additionally, the living lab inventory can
support living labs to collaborate and to develop a customized mix of infrastructure
and service offers, to respond to concrete requests and market needs. In order to
make this happen and specify the business case, we suggest deepening the analysis
by more detailed discussion with living lab managers on their services, an explorative
analysis on existing business models of living labs, and a deeper exploration of
market demand for living lab services from start-ups and innovative small and
medium sized companies. The mapping of living lab services could also be expanded
to other countries. The mapping presented in this paper can support a dialogue on
the role of living labs in an open system of research and innovation, and to develop
the related national and international networking.


We are grateful to the experts interviewed and involved for offering their time and
expertise for the research. We thank the INNOLAB project team members for their
constructive feedback on and contributions to the research. We acknowledge the
funding received for the underlying research from the German Ministry of Research
and Education for the INNOLAB project (Grant No. 01UT1418A).“


Bódi, Z., Garatea, J., García Robles, A. and & Schuurman, D. (Eds.) (2015). Living
Lab Services for Business Support and Internationalisation. ENoLL
Brynjarsdottir, H., Håkansson, M., Pierce, J., Baumer, E., DiSalvo, C. & Sengers, P.
(2012). Sustainably Unpersuaded: How Persuasion Narrows Our Vision of
Sustainability. In Proc. 2012 ACM Annual Conf. Human Factors in
Computing Systems. ACM, pp. 947–956.
Buhl, J., von Geibler, J., Echternacht, L., & Linder, M. (2017). Rebound effects in
Living Labs: Opportunities for monitoring and mitigating re-spending and time
use effects in user integrated innovation design. Journal of Cleaner
Production, 151, 592-602.
Chesbrough, H.W. (2003). The era of open innovation. MIT Sloan Management
Review, 44, 3, 35–41.
Chou, Y.-K. (2015). Actionable gamification: Beyond points, badges, and
leaderboards. Octalysis Media, 509.
Curley, M. (2016). Twelve principles for open innovation 2.0. In: Nature 533, 314–
316 (19 May 2016) doi:10.1038/533314a,
principles-for-open-innovation-2-0-1.19911. (accessed on 09.07.2018).
European Network of Living Labs (ENoLL) (2016). Living labs. Online at (accessed on 09.07.2018).
Franz, O., Wissner, M., Büllingen, F., Gries, C.I., Cremer, C., et al. (2006).
Potenziale der Informations- und Kommunikations-Technologien zur
Optimierung der Energieversorgung und des Energieverbrauchs (eEnergy).
Stud. Für Bundesminist. für Wirtsch. Technol. BMWi Bad Honnef Wik-
Consult Fraunhofer ISI Fraunhofer ISE.
Gassmann, O.; Frankenberger, K. & Csik, M. (2013). Geschäftsmodelle entwickeln:
55 innovative Konzepte mit dem St. Galler Business Model Navigator.
Munich: Carl Hanser Verlag.
Geibler, v. J. and Erdmann, L. (2017). Innovation Infrastructures 4.0. A Position
Paper to Support Germany‘s Innovation Infrastructure with Respect to
Networking, Development and Sustainability. Online at https://www.innolab-
_Innovationsstructures4.0.pdf (accessed on 09.07.2018).
Geibler, v. J., Cordaro, F., Kennedy, K., Lettenmeier, M., Roche, B. (2016).
Integrating resource efficiency in business strategies: a mixed-method
approach for environmental life cycle assessment in the single- serve. In:
Journal of Cleaner Production 115, 62-74.
Geibler, v. J., Erdmann, L., Liedtke, C., Rohn, H., Stabe, M., Berner, S., Leismann,
K., Schnalzer, K., Kennedy, K., (2014). Exploring the potential of a German

Living Lab research infrastructure for the development of low resource
products and services. Resources 575-598.Hansen, E. G., Grosse-Dunker,
F., & Reichwald, R. (2009). Sustainability Innovation Cube — a Framework
To Evaluate Sustainability-Oriented Innovations. International Journal of
Innovation Management, 13(4), 683–713.
Hassenzahl, M. (2010). Experience Design: Technology for All the Right Reasons.
Morgan and Claypool Publishers.
Keyson, D. V., Guerra-Santin, O., & Lockton, D. (Eds.). (2016). Living Labs: Design
and Assessment of Sustainable Living. Springer.
Leminen, S., Niitamo, V. P., & Westerlund, M. (2017). A Brief History of Living Labs:
From Scattered Initiatives to Global Movement. In OpenLivingLab Days.
Research Day Conference proceedings 2017. 42-58.
Leminen, S. (2013). Coordination and participation in living lab networks.
Technology Innovation Management Review, 3(11).
Ley, B., Ogonowski, C., Mu, M., Hess, J., Race, et al. (2015). At home with users: a
comparative view of Living Labs. Interacting with Computers, 27(1), 21-35.
Liedtke, C.; Welfens, M. J.; Rohn, H.; Nordmann, J. (2012). LIVING LAB: User‐
driven innovation for sustainability, International Journal of Sustainability in
Higher Education, Vol. 13 Iss: 2, pp.106 – 118.Liedtke, C., Baedecker, C.,
Hasselkuß, M., Rohn, H., Grinewitschus, V., (2015). User-integrated
innovation in Sustainable LivingLabs: an experimental infrastructure for
researching and developing sustainable product service systems. Journal of
Cleaner Production, 97, 106–116.
Lockton, D., Harrison, D., Stanton, N.A., (2010). The design with intent method: a
design tool for influencing user behaviour. Applied Ergonomics 41, 382–392.
Madlener, R. & Alcott, B. 2011. Herausforderungen für eine technisch-ökonomische
Entkoppelung von Naturverbrauch und Wirtschaftswachstum unter
besonderer Berücksichtigung der Systematisierung von Rebound-Effekten
und Problemverschiebungen [Challenges of a Technical-Economic
Decoupling of Natural Resource Consumption and Economic Growth with a
Particular Focus on Systematizing Rebound Effects and Problem-Shifts].
Aachen: Rheinisch-Westfälische Technische Hochschule.

Meijers, M. H. C., Noordewier, M. K., & Avramova, Y. R. (2014). I just recycled. Can
I use the car now? When people continue or discontinue behaving
sustainably after an initial sustainable act. In H. C. M. van Trijp (Ed.),
Encouraging sustainable behavior: Psychology and the environment (pp. 71–
80). New York, NY, US: Psychology Press.
Meurer, J. / Erdmann, L. / Geibler, J.v. / Echternacht, L. (2015): Arbeitsdefinition
und Kategorisierung von Living Labs. Arbeitspapier im Arbeitspaket 1 (AP
1.1c) des INNOLAB Projekts. Universität Siegen Wirtschaftsinformatik und
Neue Medien, Siegen.

Nelson, R. R. (Ed.). (1993). National innovation systems: a comparative analysis.
Oxford University Press on Demand.
Ness, B., Urbel-Piirsalu, E., Anderberg, S., & Olsson, L. (2007). Categorising tools
for sustainability assessment. Ecological economics, 60(3), 498-508.
Ogonowski, C., Jakobi, T., Stevens, G., & Meurer, J. (2015). Living Lab As A
Service: Das Living Lab als Dienstleistungsbaukasten zur Nutzer-zentrierten
Entwicklung und Evaluation innovativer Smart Home Lösungen. In Mensch &
Computer Workshopband (pp. 701-711).
Osterwalder, A. and Pigneur, Y. (2009). Business Model Generation,. Self
Published (ISBN 978-2-8399-0580-0).
Pierson, J.; Lievens, B. (2005). Configuring Living Labs for a “Thick” Under-standing
of Innovation. Ethnogr. Prax. Ind. Conf. Proc. 1, 114–127.
Salmelin, B. and Curley, M. (2016). Open Innovation 2.0: A new paradigm. In: Open
innovation 2.0. Yearbook 2016. European Commission.
Schäpke, N., Stelzer, F., Caniglia, G., Bergmann, M., Wanner, M., Singer-
Brodowski, M., ... & Lang, D. J. (2018). Jointly experimenting for
transformation? Shaping real-world laboratories by comparing them. GAIA-
Ecological Perspectives for Science and Society, 27(1), 85-96.
Schuurman, D. (2015). Bridging the gap between Open and User Innovation?:
exploring the value of Living Labs as a means to structure user contribution
and manage distributed innovation (Doctoral dissertation, Ghent University).
Schuurman, D., & Herregodts, A. L. (2017). Open Innovation with Entrepreneurial
Users: Evidence from Living Lab projects. In ISPIM Innovation Symposium.
The International Society for Professional Innovation Management (ISPIM).
Schwartz, T.; Stevens, G.; Jakobi, T.; Denef, S.; Ramirez, L.; Wulf, V.; Randall, D.
(2014). What People Do with Consumption Feedback: A Long-Term Living
Lab Study of a Home Energy Management System. In: Interacting with
Computers. Oxford University Press.
Ståhlbröst, A. (2013). A living lab as a service: creating value for micro-enterprises
through collaboration and innovation. Technology Innovation Management
Review, 3(11).Terlau, W., and Hirsch, D. (2015). Sustainable Consumption
and the Attitude-Behaviour-Gap Phenomenon - Causes and Measurements
towards a Sustainable Development, 6(3), 159–174.
Ullman, D. G. (1997). The Mechanical Design Process. 2nd ed. New York, USA:

Annex: Living Labs in Germany and their services

Table 1 Living Labs in Germany and their services

Customer oriented User integrating

services* services*

Stakeholder networking

UX Testing & evaluation

Motivational design
Business model

and brokerage

assessment *
User studies

No. Living Lab City

1 COWERK Berlin 0 0 0 0 0 0 0 0 0
Fraunhofer IAO Esslingen am
0 0 0 0 0 0 0 0 0
2 Living Lab eFleet Neckar
3 Fraunhofer IGD Darmstadt 0 0 0 0 0 0 0 0 0
4 Fraunhofer ISST Dortmund 0 0 0 0 0 0 0 0 0
5 Future City Lab Stuttgart 0 0 0 0 0 0 0 0 0
6 GetMobil Kassel 0 0 0 0 0 0 0 0 0
7 Green Travel Lüneburg 0 0 0 0 0 0 0 0 0
8 EECC InnovationLabs Neuss 1 1 1 1 1 1 1 1 0
9 ILoNa Duisburg 0 0 0 0 0 0 0 0 0
10 IMKoN Berlin 0 0 0 0 0 0 0 0 0
11 Living Lab AIM Oldenburg 0 0 0 0 0 0 0 0 0
12 USEEDS GmbH Berlin 0 1 1 1 1 1 1 1 0
13 Neighborhood Lab Berlin 0 0 0 0 0 0 0 0 0
14 Userlutions Berlin 0 1 1 1 1 1 1 1 0
15 OrgLab, Duisburg/Essen Essen 0 0 0 0 0 0 0 0 0
Peter L. Reichertz Institut für
Hannover 0 0 0 0 0 0 0 0 0
16 Medizinische Informatik
17 YOUSE GmbH Berlin 0 1 1 1 1 1 1 1 0
Reallabor Nordschwarzwald
Freiburg 0 0 0 0 0 0 0 0 0
18 (ReNo)
19 Reallabor Schorndorf Schorndorf 0 0 0 0 0 0 0 0 0
Josephs - Die Service
Nürnberg 1 0 1 1 1 1 0 1 0
20 Manufaktur
21 Retail Lab der THI Ingolstadt 1 1 1 1 0 0 0 1 0
22 IFH Cologne Cologne 0 1 1 1 0 0 0 1 1
The Virtual Dimension
Fellbach 0 0 0 0 0 0 0 0 0
23 Center
24 Trafo 3.0 Freiburg 0 0 0 0 0 0 0 0 0
25 Living Lab Ludwigsburg Ludwigsburg 0 1 1 1 0 0 0 1 1
26 Ökodorf Sieben Linden Beetzendorf 1 0 0 1 1 1 0 0 1
Frankfurt am
WohnMobil 0 0 0 0 0 0 0 0 0
27 Main
28 WTW Wuppertal 0 0 0 0 0 0 0 0 0
Bremen Ambient Assisted
Bremen 1 0 0 0 1 0 0 1 0
29 Living Lab (BAALL)
Innovative Retail Laboratory
St. Wendel 1 1 0 1 0 0 0 1 0
30 (IRL)
31 GS1 Knowledge Center Cologne 1 0 0 1 1 0 0 0 0
32 EHI Retail Institute e.V. Cologne 0 0 0 1 0 0 0 0 0
33 GermanRetailLab e.V. Munich 0 0 0 1 0 0 0 0 0
Frankfurt am
Virtueller Supermarkt 0 0 0 1 0 0 0 0 0
34 Main
Obi Düsseldorf 0 0 0 0 0 0 0 0 0
35 n
36 Retail Innovation Lab SAP St.Ingbert 0 0 0 0 0 0 0 0 0
Berliner Agentur für
Elektromobilität eMO; Berlin 1 0 0 1 0 0 0 0 0
37 Schaufenster
38 SAP Future Factory Initiative Dresden 0 0 0 0 0 0 0 0 0

Customer oriented User integrating
services* services*

Stakeholder networking

UX Testing & evaluation

Motivational design
Business model

and brokerage

assessment *
User studies

No. Living Lab City

39 Logwert Heilbronn 0 0 0 0 0 0 0 0 0
Living Lab - Wohnen und
Osnabrück 0 0 1 1 0 0 0 0 0
40 Pflege
41 Porsche Digital Lab Berlin 0 0 0 0 0 0 0 0 0
Urban Living Lab/Fraunhofer
Stuttgart 0 0 0 0 0 0 0 0 0
42 IAO
43 VW Digital Lab Berlin 0 0 0 0 0 0 0 0 0
44 Fahrsimulator Oldenburg 0 0 0 0 0 0 0 1 0
45 Klimaquartier Arrenberg Wuppertal 0 1 0 0 0 0 0 0 0
46 Metro Accelerator Düsseldorf 0 1 1 1 0 0 0 0 0
47 Reallabor SPACESHARING Stuttgart 0 1 0 1 0 0 0 0 1
48 FZI House of Living Labs Karlsruhe 0 1 0 1 0 0 0 0 0
49 OFFIS e.V. Oldenburg 1 0 0 1 0 0 0 0 0
50 e-mobility Stuttgart Stuttgart 0 1 0 1 0 0 0 0 1
e-Wohnen der Zukunft—
Berlin 1 0 0 1 0 0 0 1 0
51 Projekt 4
52 Urban Office Heidelberg 1 0 0 1 1 0 0 0 0
53 weShop Munich 1 1 0 1 0 0 0 0 0
eMIR - eMaritime Integrated
Oldenburg 0 1 0 0 1 1 0 1 0
54 Reference Platform
55 Fraunhofer Fokus Berlin 0 0 0 1 0 1 0 1 0
56 Haus am Feld Wildau 0 0 0 1 1 1 0 0 1
57 Logistics Living Lab Leipzig 1 0 0 1 1 0 0 1 0
58 Science Box Bottrop 1 1 0 0 1 1 0 0 0
59 Fraunhofer inHaus-Zentrum Duisburg 1 1 0 1 0 1 0 1 0
60 SRG Berlin Berlin 0 1 0 1 0 0 0 1 0
SmartCity Living Lab (SCLL)
Kaiserslautern 0 0 0 1 1 0 0 1 1
61 (DFKI)
SmartHome der Universität
Neubiberg 1 1 0 0 0 0 0 1 1
62 Bundeswehr Munich
Kaiserslautern 1 0 0 1 0 0 0 1 0
63 SmartFactory KL e.V.
64 Tiny Houses Consulting UG Schleching 0 0 1 1 0 0 0 0 1
65 Tobit.Labs Visitor Center Ahaus 1 0 0 1 0 0 0 1 0
66 Energieeffizienz-haus-Plus* Berlin 1 0 0 1 0 1 0 1 1
67 Universal Home Dortmund 0 0 1 1 0 0 0 0 1
Furtwangen 0 1 0 1 1 0 0 1 0
68 Furtwangen)
69 SESA Lab Oldenburg 1 0 0 1 1 1 0 0 0
SILAB/Würzburger Institut 0 1 0 0 1 1 0 1 0
für Verkehrswissenschaften, Veitshöchheim
70 Veitshöchheim
WATTx Accelerator
Berlin 0 1 1 1 0 0 0 1 0
71 (Viessmann)
72 BMW Start-up Gagarage Garching 0 1 1 1 0 1 0 1 0
73 Connected Living Berlin 1 0 1 1 0 1 0 1 0
Lemgo Digital / Smart
Lemgo 1 1 0 0 1 1 0 1 0
74 Factory OWL
75 Eon agile accelerator Berlin Berlin 0 1 1 1 0 1 0 1 0
Eon agile accelerator
Düsseldorf 0 1 1 1 0 1 0 1 0
76 Düsseldorf
77 Open Mobility Forum Berlin 1 1 1 1 1 1 1 1 1
(Bundeverband deutscher Bad Honnef 1 1 0 1 0 0 0 1 1
78 Fertigbau e.V.)
TIPI4.0 - Test- und
Oldenburg 1 1 1 0 0 1 0 1 0
79 Integrationsplattform

Customer oriented User integrating
services* services*

Stakeholder networking

UX Testing & evaluation

Motivational design
Business model

and brokerage

assessment *
User studies

No. Living Lab City

TUM Living Lab Connected
Garching 0 1 1 1 0 1 0 1 0
80 Mobility
Uni Ulm/ Sustainable
transformation of the textile Ulm 1 1 1 0 1 0 0 0 1
81 industry in Dietenheim
82 Future Living Berlin 1 1 1 1 0 0 0 1 0
83 Innogy innovation hub Essen 0 1 1 1 0 1 0 1 0
eHealth Future Care Lab,
Aachen 1 1 1 0 1 1 0 1 0
84 RWTH Aachen
Smart Energy Living Lab @
Munich 1 1 0 0 1 1 0 1 0
85 fortiss
86 Innovation Store Pulheim 1 1 0 1 1 0 0 1 1
87 LiLA Walldorf Walldorf 0 1 1 1 1 1 0 1 0
Selbsthilfeprojekt Dietzenbach-
1 1 0 1 1 1 0 0 1
88 „Arminiusstrasse“ Steinberg
e:Lab – Bürgerlabor für
Energieinnovationen Dortmund 0 1 1 1 1 1 0 1 1
89 (Fraunhofer)
90 SmartHome Paderborn e.V. Paderborn 1 0 1 1 0 0 0 1 1
Ambient Assisted Living
Environment AAL Kaiserslautern 1 1 0 1 1 1 0 1 0
91 (Fraunhofer IESE )
Frankfurt am
D.lab, Deutsche Bahn 0 1 1 1 1 1 0 1 0
92 Main
E|Home-Center -
Nürnberg 1 1 1 1 0 1 0 1 1
Technologiezentrum für
93 privates Wohnen
94 INNOLAB Wuppertal 0 1 1 1 1 1 0 1 1
EnSign - Reallabor für einen
klimaneutralen Stuttgart 0 1 1 1 1 0 1 0 1
95 Innenstadtcampus
Fraunhofer IAO Office
Stuttgart 1 1 1 1 0 0 1 1 1
96 Innovation Center
Smart City Living Lab
Oldenburg 1 0 1 1 1 1 0 1 1
97 Fliegerhorst Oldenburg
98 Praxlabs Siegen 1 1 1 1 1 1 1 1 0
99 Saturn Ingolstadt 0 0 0 0 0 0 0 0 0

* Services are communicated and mentioned at the website (1); services are not communicated at the
website (0

Schools4energy: a living laboratory for energy
awareness in schools
Carmelina Cosmi*1, Filomena Pietrapertosa1,
Giuliano Sarricchio2, Michele Giordano2, Monica Proto1,
Marco Tancredi2, Monica Salvia1

*Corresponding author
1National Council of Research of Italy,
Institute of Methodologies for Environmental Analysis CNR
2Lucanian Energy Company

Category: Research in-progress

Schools4energy is a laboratory of sustainable experimentation for the implementation
of good practices that aims to promote conscious and virtuous energy behaviours
and energy efficiency in the schools. The activities, developed in the framework of
the INTERREG MED PrioritEE project, involve the students, the teachers and the
staff personnel of Primary and Lower Secondary Schools of the Municipality of
Potenza. The Schools4energy laboratory is made up of three integrated modules
(School race, Artists for energy, Energy at stake) based on different methodologies
(analytical methods, co-creation and gamification) in order to increase the interest
and the involvement of students and enhance their preferences and talents. The
proposed activities pursue multiple objectives: to increase students’ knowledge and
skills on energy, to raise awareness on energy consumption, to encourage energy
consumption reduction, to promote behavioural changes.

Keywords: Multi-stakeholder engagement, Actors Motivation for Sustainable

Actions, Public Schools, Energy Efficiency, Behavioural changes, Awareness raising

1 Introduction

Living Labs are becoming increasingly popular to encourage an active engagement

of citizens through experimentation and learning as well as to support a participative
and inclusive decision-making process promoting by this way a faster transition
towards a low carbon society.

According to the JPI Urban Europe, that firstly introduced the term Urban Living Lab
– ULL they can be defined as “a forum for innovation, applied to the development of
new products, systems, services, and processes, employing working methods to
integrate people into the entire development process as users and co-creators, to
explore, examine, experiment, test and evaluate new ideas, scenarios, processes,
systems, concepts and creative solutions in complex and real contexts” (JPI Urban
Europe, 2013).

The European Network of Living Labs (ENoLL) defines Living Labs as “user-centred,
open innovation ecosystems based on systematic user co-creation approach,
integrating research and innovation processes in real life communities and settings.
LLs are both practice-driven organisations that facilitate and foster open,
collaborative innovation, as well as real-life environments or arenas where both open
innovation and user innovation processes can be studied and subject to experiments
and where new solutions are developed. LLs operate as intermediaries among
citizens, research organisations, companied, cities and regions for joint value co-
creation, rapid prototyping or validation to scale up innovation and businesses. LLs
have common elements but multiple different implementations.”

Living Labs actually constitute a powerful form of governance that promotes

cooperation through active participation, experimentation and learning representing
a valuable tool to support a structured cooperation among local authorities, citizens
and scientific institutions.

The European Network of Living Labs (ENoLL), the international federation of

benchmarked Living Labs in Europe and worldwide, provides many examples of
Living Labs on different themes among which more than twenty examples
corresponding to the key words “Energy”, “Efficiency” and “Renewable”.
Starting from the valuable background of the ENoLL and the experience gained by
the authors in Interreg IVC RENERGY project (Dvarioniene et al, 2015), in the
framework of the INTERREG MED PrioritEE project “Prioritise energy efficiency
measures in public buildings: a decision support tool for regional and local public
authorities” (PrioritEE, 2018), the Local Living Labs (LLLs) were adopted to boost the
effectiveness of policy and measures and to facilitate the achievement of higher
energy efficiency targets through an active engagement of stakeholders. The

objective of the PrioritEE Local Living Labs is twofold: on one hand to support an
improvement in the energy performance of public buildings by monitoring their energy
consumption and highlighting technical weaknesses, on the other hand to promote
energy awareness in the whole community through experiential activities and the
active involvement of public buildings users.

Three rounds of Local Living Labs (LLLs) were thus planned in each of the five partner
regions (Karlovac County - Croatia, Municipality of Potenza - Italy, Region of Western
Macedonia - Greece, Lezíria do Tejo Intermunicipal Community - Portugal, and
Aragón region - Spain) according to the specific activities foreseen in the relate pilot
case studies. In this framework, LLLs are very useful to gather information on the
needs and expectations of public building users as well as to promote energy
efficiency and awareness measures in the public buildings of pilot areas involving
decision-makers and addressing energy managers and building users as main
stakeholder for the implementation of energy saving measures. Customised activities
have been organised in each community in close cooperation with the Local
Authorities in order to capitalise their main outcomes in the local planning framework.
An overview of the PrioritEE Living Labs is reported in Table 1.

Table 1. Overview of the PrioritEE Local Living Labs in the pilot areas

Target Area Title and Aim Stakeholders Role

- Mayor
- Municipal Staff
Schools4energy: - SEL (central - Policy makers
- National
energy purchasing - Technical experts
Research Council Potenza experiential activities
body) - Day-to day users
- Lucanian Energy Municipality for awareness raising
- Energy managers - Executors of
Company (IT) and energy saving in -Headmasters experiential
- Municipality of
schools. - Teachers & activities

-Regional Success Stories and

Development - National and Local
Region of Good Practises on - Policy Makers
Agency of West Stakeholders
Western energy efficiency: - Technical experts
Macedonia - Representatives of
- Centre for Macedonia implementation in the the technical - Executors of Good
Renewable (EL) Municipal Public departments of the Practices
Energy Sources Buildings. municipalities
and Saving

- Mayors - Policy Makers

- North-West - Municipal Staff
Croatia Regional City of - Technical experts
Success Stories in - Other public
Karlovac (HR) - Executors of Good
Energy Agency City of Karlovac: stakeholders
- City of Karlovac - Civil society Practices
implementation of

selected good
raising measures
within the community.

Success Stories
- Diputación adapted to rural
- Policy makers
Provincial de Teruel areas: - Technical experts
Teruel Province (ES) implementation of - Executors of Good
- University of customised solutions Practices
Zaragoza to improve EE in
Municipal Public

Training on the
implementation of
good practices and
awareness raising
measures: hands-on
-Leziria do Tejo
session on the energy
Community Leziria do Tejo management
- Policy makers
Intermunicipal software recently - Mayors - Technical experts
-Faculty of Community installed on 46 - Energy managers - Executors of Good
Science and (PT) municipal public - Municipal Staff Practices
Technology - buildings and
NOVA University implementation of
of Lisbon
selected good
practices for the
behavioural changes
in the target

2 The LLL of the Potenza pilot

This paper focuses on the Italian pilot case study (Municipality of Potenza) aimed at
boosting energy efficiency in public schools run by the municipality. In fact, as well
known, schools’ energy bills represent a main source of expenditure in the economic
budgets of municipalities. Their huge energy consumption is due to outdated and
inefficient buildings and waste of energy caused both by a generally bad energy
management (in particular, concerning heating) and the unaware users' behaviour.
The students of Primary and Lower Secondary schools were identified as the
recipient of the Living Lab activities. This was due to the primary necessity to reduce
the schools’ energy bills by triggering the attention of day-to-day users on energy
awareness and to exploit the potential of young generations that can transfer the
energy saving concepts on the whole community through family and friends.

Innovative and challenging educational activities based either on the use of different
methodologies and tools were designed to trigger the attention of building users on
energy themes while promoting an “energy saving” culture and a more efficient use
of energy resources. In particular, cloud-based business analytics services will be
utilised for energy consumption monitoring (Benchmonitor® and Power BI -, whereas co-creation (Rowley et. al, 2007) and
gamification ( will allow to enhance the creative and playful
components of the Living Lab. The long experience carried out by the Province of
Treviso in the framework of the projects Interreg MED EduFootprint
(, provided inspiration for
organising a set of awareness raising activities engaging Local Authorities, teachers
and students.

In the Potenza pilot Living Lab, schools will compete to achieve the highest savings
either in term of energy uses and energy bill. Awareness measures will be adopted
to support the decrease of consumption, assessing the role of consumers’ behaviour
in energy savings. In parallel, interdisciplinary thematic activities will be developed to
promote an improved knowledge on energy subjects through creative thinking and
group dynamics. It is important to underline how this experience, by exploiting the
interest of young people and their influence on the whole community, energy
efficiency can be pursued while ensuring sustainable behaviours.

The Schools4energy project and its educational relevance has been designed in the
framework of the Interreg Med PrioritEE project (PrioritEE, 2018) by CNR-IMAA and
the Lucanian Energy Company, in collaboration with the Municipality of Potenza. It
has been thought as a laboratory of sustainable experimentation in which users and
energy experts cooperate to implement a user-driven innovation process through the
Living Lab tools. The activities aim at promoting conscious and virtuous energy
behaviours through the engagement on a voluntary basis of students and teachers
of the schools owned by the Municipality of Potenza (Kindergarten, Primary and
Lower Secondary schools). The aim is to foster an improved knowledge of energy
issues and the energy bill, driving the attention of all the buildings’ users on how the
behavioural changes can influence consumption. The energy topics will be inserted
by the teachers in the curriculum of the school year 2018-2019 and, according to the
outcomes of this experience, the method shall become an integral part of the
education method in the schools operationally involved in the implementation phase.
The experience is aimed at consolidating an “energy culture” in the context of
education through a practical involvement of all the users of public schools and, in
particular, pupils of different ages. Different methodologies and activities were
integrated to foster a larger participation and encourage pupils, teachers and staff
personnel to adopt "responsible" behaviours. Schools4energy is thus designed as an

experiential laboratory for the implementation of a systematic educational program
that aims at:
• Improving the understanding of the value of energy and behavioural changes
• Supporting the creation of "energy natives" within the young generations
• Providing intrinsic motivations to improve energy uses by an awarding
mechanism (such as having a recognition or prize);
• Improving/creating a sense of community (the users are protagonists / group
• Making evident energy savings obtained from behavioural changes and
quantifying them in energy and economic units;
• Supporting the creation of routine habits to be transferred in the family context
and in the whole community.

From an educational point of view, Schools4energy contributes to four fundamental


• To promote an energy efficiency culture by highlighting the effects of the

reduction of consumption deriving from behaviour, through experimental
measurement and creative thinking
• To value economically the savings in the energy bill by adopting virtuous
• To promote environmental protection by reducing the pollutant emissions and
energy waste through a conscious use of energy resources
• To improve the scientific knowledge and skills providing the basic elements to
understand in detail energy consumption and energy bill by direct
measurement and comparative analysis.

To valorise the interdisciplinary context as well as to exploit different attitudes and

creativeness, three different groups of activities were designed (Figure 1):

Figure 1. The Schools4energy modules

The initiative will be presented to the school board prior of the beginning of the school
year, in order to allow the integration of the foreseen activities in the educational
curricula. In this regard, educational guidelines on “self-regulation” of consumption
will be published to document the relevance of the initiative and to promote the
dissemination of good practices in other schools.
A dedicated webpage to Schools4energy will be created on the institutional website
of the Lucanian Energy Company to enable a direct interaction with the community
and to provide information on the ongoing activities.

3 The Energy Team

As successfully carried out in the framework of the Green schools competition

( of the Province of Treviso (Italy) and the
EURONET 50/50 MAX project (EURONET 50/50 MAX), each participating school of
the Potenza Municipality will set up an Energy Team with coordination and control
functions, as shown in Figure 2:




Figure 2: Energy Team components

School staff personnel with different roles and selected students will compose the
school energy team. The external energy experts will train the school energy teams
on the implementation of energy awareness activities. The school energy team will
propose a set of energy saving measures (“Decalogue of good practices”) to be
implemented in order to evaluate the influence of behaviours on energy consumption.
Selected students of the Lower Secondary schools will act as facilitators to motivate
their peers.

4 Energy Benchmarking: School Race

The School Race module, designed by SEL, represents the central element of the
Potenza pilot Living Lab, supporting through a competition mechanism an increased
knowledge on energy consumption, improved skills in science and technology and
encouraging virtuous behaviour.

To motivate students and increase their interest, the participating schools are
associated to the Formula 1 stables and the various buildings complexes of each
institute will represent the pilots.

The match between the participating schools and Formula 1 stables will be carried
out during an opening ceremony that will be held at the beginning of the 2018/2019
school year.

Two web-based tools are utilised to monitor energy consumption, share the results
and show the energy trends:

• SELbench (Benchmonitor®), an innovative software implemented by the

Lucanian Energy Company for monthly management of electricity and gas
consumption, as well as Energy Benchmarking, available free of charge for all
the affiliated public administrations35, including schools,
• Power BI a business analytics service provided by Microsoft that allows for
interactive visualizations with self-service business intelligence capabilities

To reduce their consumption and improve their position in the ranking of “Formula 1
stables”, each school will apply its “Decalogue of good practices” in a 7-month period
(from October 1-st, 2018 to April 30-th, 2019).

Schools will be classified according to their features (age of construction and volume,
equipment, number of users, etc.) and average energy consumption in energy levels
(“BenchClass”). Their monthly consumption (natural gas for space heating and
electricity) will be monitored on a 12-month period (May 1-st, 2018- April 30-th, 2019)
and compared with a baseline, according to the reporting template shown in Table 2.
Key Performance Indicators will be used to normalise the data in order to allow the
comparison among schools with different features.

Table 4. Energy consumption monitoring template.

School Year 2018/2019



Energy Level (“BenchClass”)

Month i (i= May 2018 to April 2019)

Electricity consumption Natural gas consumption

Point of Delivery Number (POD) Redelivery Point Number (PDR)

Total Electricity (kWh) – baseline Month i Total gas (m3) – baseline – baseline
Month i

Total Electricity (kWh) measured - Month i Total gas (m3) measured - Month i

35 SelBench is the Lucanian version of the software called Benchmonitor®, a system for tracking,
visualizing and managing energy bill and energy consumptions, marketed in Italy through the portal

Energy saving percentage Energy saving percentage

(Variation between actual monthly (Variation between actual monthly

consumption with the baseline) consumption with the baseline)

An energy benchmarking procedure is applied to define the baseline consumption

and to evaluate the energy savings. For each school, the energy baseline is
represented by the average monthly energy consumption of the last three school
years (2015/2016, 2016/2017, 2017/2018). Energy savings are evaluated by
comparing the actual monthly consumption with the baseline.

A different involvement of pupils in the School race was devised according to their
age and educational purposes, as reported in the following.

Level 1: Kindergarten and Primary School

The pupils of each class, coordinated by their teachers and assisted by the staff
personnel will put into practice the “Decalogue of good practices” to decrease
consumption. The observance of the Decalogue rules will be monitored weekly and
the class will receive accordingly a green or red sticker corresponding to a positive
or negative score (e.g. green sticker: +10 points; red sticker: -5 points). A monthly
energy report will show the “efficiency” of each class and make evident their
progresses. The energy reports will contribute to the overall school ranking.

Level 2: Lower Secondary School

The Lower Secondary School students will be involved actively in consumption

monitoring and reporting. The students of each participating class will become part
of the Energy Team and will be trained together with the teachers and staff personnel
in the use of the ICT tools for energy benchmarking.

Monthly updated information on the ranking of “F1 stables and pilots” will be
published on a dedicated page of the Lucanian Energy Company website to keep
alive the interest on the race. A “School Race poster” will be displayed in each
participating school to explain the initiative and the rules of the game. A QR code will
provide the users with the possibility to deliver up-to-date information on
Schools4energy initiative and schools ranking (Figure 4).

School ranking
School Race
(School Year 2018/2019)

See results online

Poster with QR CODE and web
page link

Figure 3. Data management and visualisation

A reward system based on the total savings obtained and the equivalent money
earned by each participating school (a percentage of the energy bill savings) is
defined in agreement with the Municipality of Potenza to award the school that will
achieve the overall highest energy savings in the school year 2018-2019. The
absolute ranking of the schools will be presented in the final awarding ceremony in
May 2019.

5 Co-creation: Artists for Energy

Co-creation was identified as a powerful methodology to integrate energy issues in

the educational curricula, promoting an interdisciplinary cooperation of teachers,
valorising creativeness and involving a larger number of pupils.

A set of activities has been proposed to the school board for inspiration among which
drawing and literary thematic competitions, creation of storytelling, posters on energy
themes and energy saving measures, energy laboratories (construction of small
energy toys – e.g. solar cars, solar boats, etc.), "energizing" musical performances;
theatrical performances and sport competitions, addressed to children of different

The teachers will define the thematic activities to be implemented during the
2018/2019 school year according to the different educational curricula and sections
(Kindergarten, Primary, Lower Secondary) in order to maximize their educational

potential and encourage children to think about energy in a creative way.

A cooperation with local associations and NGOs (e.g. Legambiente) is envisaged to

take advantage from their expertise and involve the community as much as possible.
The artefacts made by the pupils by age group and category will be voted through
the School4energy webpage on the Lucanian Energy Company website and the best
ones will be rewarded.

6 Gamification: Energy at a stake

The role of gamification in supporting behavioural changes and motivation through

evoking gameful experiences is widely studied and applied in many disciplines (e.g.
Hamari J. et al., 2014; Kazhamiakin R. et. al 2016). This concept was exploited also
in in the context of many European funded projects that developed free downloadable
thematic videogames, most of which can be used by teachers to drive the attention
of students.

As alternative energy awareness activities, video games can encourage players to

consume less energy, support behavioural changes and awareness raising through
an alternative educational tool based on the concept of serious games.
In the Potenza pilot Living Lab framework, videogames were identified as alternative
way of learning. The teachers will examine the proposed examples in order to plan
their possible integration in the educational activities.
Table 3 reports the selected videogames concerning the energy themes.

Table 3. Thematic videogames on energy and environment

Project Game/website Objective

EnerCities Create and expand virtual cities capable
EnerCities IEE
Project to manage pollution, energy supply, renewable energy, etc.

Energy Chickens
Impact HUB To decrease electric energy consumption
Berkeley of the building inhabitants
/ To minimize the environmental impact of

TRIBE H2020 project cities by reducing carbon emissions and building energy consumption

7 Calendar of activities

The Potenza pilot Living Lab activities have been developed since December 2017
according to the timeline reported in (Figure 4)

Figure 4. Timeline of the Potenza pilot Living Lab and School4energy activities

The public events will be jointly organised by the PrioritEE scientific partners (SEL
and CNR-IMAA), the Municipality of Potenza and the schools to boost their
communication power.

At the beginning of the 2018/2019 school year, an opening ceremony will officially
launch the initiative. During the period October 2018 – May 2019, the schools will be
also involved in thematic public events on energy awareness raising to reinforce
motivation. In particular, it is highly recommended the participation in "M'illumino di
Meno", a national symbolic initiative launched in 2005 by the radio broadcast
Caterpillar of Rai Radio 2 and sponsored since 2015 by the Italian Ministry of
Education, which is organized annually around the 16 February, date of entry into
force of the Kyoto Protocol (Wikipedia, 2018).

The awarding ceremony will conclude the activities, emphasising the positive
outcomes of the initiative. In particular, an exhibition of the creative works will be
organized and the winners of each category will be awarded. The final ranking of the
school race and the total savings earned by each school will be presented, and the
winning school will be announced. In order to raise motivation and involvement, as
well as to enhance the value of learning and participation, all the pupils will receive a
certificate of participation acknowledging their commitment as “energy saving expert”.
A selected group of pupils will be also invited to participate in the PrioritEE Final
Conference to present their experience emphasising its relevance in an international
context and enabling networking initiatives.

8 Acknowledgements

This research was carried out in the framework of the project PrioritEE “Prioritise
energy efficiency (EE) measures in public buildings: a decision support tool for
regional and local public authorities” (Project Number: 1MED15_2.1_M2_205,
Duration: 01/02/2017-31/07/2019). PrioritEE was funded under The Interreg MED
Programme 2014-2020; Priority Axis: 2. Fostering low-carbon strategies and energy
efficiency in specific MED territories: cities, islands and remote areas. We would like
to thank the Municipality of Potenza (Italy) for its active role and continuous support
as Associated Partner.


Benchmonitor® (2018). Retrieved from

Dvarioniene, J., Gurauskiene, I., Gevicious, G., Trummer, D.R., Selada, C., Marques
I. and Cosmi C. (2015, 03). Stakeholders involvement for energy conscious
communities: the Energy Labs experience in 10 European communities.
Renewable Energy, pp. 512-518.
Gamification (2018). Retrieved from
Green schools competition. Province of Treviso (Italy). Retrieved from
Hamari, J., Koivisto, J., Sarsa, H. (2014, 01) Does Gamification Work? — A Literature
Review of Empirical Studies on Gamification. System Sciences (HICSS), pp.
EURONET 50/50 MAX (2018) Retrieved from
Interreg MED PrioritEE (2018). Retrieved from
Interreg MED EduFootprint (2018). Retrieved from https://edufootprint.interreg-
Interreg CENTRAL EUROPE TOGETHER (2018). Retrieved from
Kazhamiakin, R., Marconi, A., Martinelli, A., Pistore, M., Valetto, G. (2016, 09). A
gamification framework for the long-term engagement of smart citizens. IEEE
International Smart Cities Conference (ISC2), pp. 1-7.
Power BI (2018). Retrieved from
Rowley, J., Kupiec‐Teahan, B., Leeming, E. (2007, 04) Customer community and co‐
creation: a case study. Marketing Intelligence & Planning, pp. 136-146,
Società Energetica Lucana (Lucanian Energy Company) (2018): Available at:
The European Network of Living Labs (ENoLL). Retrieved from
Urban Europe: Creating attractive, sustainable and Economically Viable Urban
Areas. Joint Call for Proposal 2013. JPI Urban Europe 2013
Voytenko, Y., Mc Cormick, K., Evans, J., Schliwa, G. (2016, 06) Urban living labs for
sustainability and low carbon cities in Europe: towards a research agenda.
Journal of Cleaner Production, pp. 45-54.
Wikipedia (2018) Retrieved from

The Circle of Mediators: Towards a governance
model for tackling sustainability challenges in a city
Anne Äyväri1 Annukka Jyrämä2
Tuija Hirvikoski1

1 Laurea University of Applied Sciences

2 Aalto University

Category: Research Paper

Cities can take an active role in creating enablers, such as open innovation platforms,
living labs to enhance innovations that can contribute to solving sustainability
challenges. In this conceptual paper, we introduce a new governance model, the
Circle of Mediators, to facilitate the birth and activities of multi-stakeholder teams to
innovate and create solutions. We aim to contribute to discussions on multi-
stakeholder governance, sustainability is seen as the context that forces cities to
engage in such activities due to the complex nature of global challenges and need
for (interdisciplinary or even) transdisciplinary and multi-stakeholder competencies to
solve these challenges. The Circle of Mediators orchestrates the establishment of
open innovation ecosystems and their innovation co-creation, experimentation and
adoption activities, thus making cities and communities inclusive, resilient and
sustainable, in accordance with the UN SDG 11 (United Nations 2015). The focus is
on the new governance model looked at from mediating and living lab perspectives
and experiments with two Finnish cities and sponsored by the EU Cohesion Six City
Strategy project. The paper’s contribution builds on the development of the
conceptualisation of the new Circle of Mediators model and the identification of
research and an action path for both researchers, managers as well as policymakers.

Keywords: mediator, circle of mediators, sustainable cities, cities as living labs,

innovation ecosystems, governance

1 Introduction

In today’s world, it has been acknowledged that cities are the key players in solving
global sustainability challenges, in building the future sustainable development
(economic, environmental and social) of the European Union and its citizens. The
European Union is one of the most urbanised areas in the world. Today, more than
70 per cent of Europe’s citizens live in an urban area. The UN projects that by 2050
this percentage will reach 80 per cent
( However, it is worth noting that
solving sustainability challenges cannot be done by cities alone. These require novel
methods of thinking through multi-stakeholder activities. In addition, the new ways of
tackling these challenges require a novel mindset, abandoning the focus of one’s own
(organisational) success and looking for success in larger settings—within the context
of ecosystems.

Ideally, cities create the conditions for sustainable and intelligent living when smart
solutions are jointly developed as well as distributed resource and environmentally
efficiently (Curley & Salmelin 2018). In this way, cities can be seen as facilitators or
enablers of sustainability. They provide a good basis for creating multi-stakeholder
activities and making it possible for competitive ecosystems to emerge. Cities can
also be seen as open innovation platforms (e.g. Ojasalo 2015) in line with living lab
perspectives, or as cultivated innovation ecosystems. The city as an open living lab
is based on value cycles, building value from closed loops, where resources are used
intensively and with less risk (Curley & Salmelin 2018). With innovation ecosystems,
we refer to the context, where the interaction between actors is geared towards
finding a solution, product or service towards markets to benefit society. In this
context, the drivers for ecosystem activity are economic, social and ecological or
political challenges. The interaction within the ecosystem is open, the challenges can
be local, regional or global, yet often the global element even in local challenges is
inherent. Ecosystems offer and adopt solutions, and produce arguments and
knowledge for enabling the participation of citizens in innovation activity. However,
less attention has been paid to the governance of these challenging multi-stakeholder
practices in the city context.

In this conceptual paper, we aim to provide a novel governance model, the Circle of
Mediators, to tackle sustainability challenges in cities. Namely, how to govern multi-
stakeholder activities in ecosystems. We adopt perspectives from living lab literature,
namely the actor roles, and elaborate on them with insights from mediator literature.
We aim to contribute to discussions on multi-stakeholder governance, sustainability
is seen as the context that forces cities to engage in such activities due to the complex
nature of global challenges and need for (interdisciplinary or even) transdisciplinary
and multi-stakeholder competencies to solve these challenges. Cities operating as

living labs, aiming to solve sustainability challenges, also represent a competitive
factor, attracting people, companies and investors.

In addition, we focus on the role of mediators seen from both mediator and holistic
innovation ecosystem or living lab approaches, to differentiate our approach from, for
example, the discussion on innovation intermediaries. The discussion on innovation
intermediaries has been lively since the 1990s (Howells 2006, Bakici et al. 2013;
Agogué et al. 2017). Intermediaries are regarded as necessary actors to make
interactions and the matching of partners possible in innovation ecosystems (Katzy
et al. 2013). An innovation intermediary is an external organization or an individual
acting as a mediator and offering intermediation services between two or more parties
(see discussion on definitions e.g. in Sieg et al. 2010, Bakici et al. 2013, Katzy et al.
2013, Agogué et al. 2017).

In innovation intermediary literature, mediators are generally perceived as

organizations sharing knowledge in a somewhat linear manner, from one
organization to another. We, however, wish to emphasize the intertwined nature of
knowledge sharing, especially in the context of innovations. We see mediating as an
interconnected activity among various types of actors with a multitude of roles.
Our proposal for a new governance structure (the Circle of Mediators) for cities as
living labs differs from the concept of an innovation intermediary mainly in two
respects: first, the Circle of Mediators is not only composed of external actors and,
secondly, its role as a mediator covers all the stages in innovation processes, and
between all the actors. Furthermore, we wish to emphasize that although the
mainstream of literature on innovation intermediaries discusses mediating within the
private sector, the Circle of Mediators is proposed as a new multi-stakeholder
governance structure for cities.

Sustainability challenges and cities

In the United Nations Sustainability Development Goals (United Nations 2015),

achieving sustainable cities and communities is one of the main goals (number 11)
stating: “Make cities and human settlements inclusive, safe, resilient and
sustainable”. The global population is increasingly concentrated in urban areas,
creating the need to solve sustainability challenges linked to, for example, safety,
wellbeing or transport and housing. In this paper, sustainability is understood
holistically, through economic, social and environmental dimensions (e.g. However, we will not go
into sustainability discussions as such; however, here sustainability is the key context
and driver for multi-stakeholder activities.

Saviano et al. (2017) elaborate on the connection between sustainability and service
research. They identify the following connection points: Multi-stakeholder
engagement and co-creation logic, systems thinking mindset and ecosystems view,
education of T-shaped managers, multi-, inter- and transdisciplinary knowledge. In
this paper, we wish to highlight the importance of the multi-stakeholder view,
acknowledging the need for multi-, inter- and transdisciplinary knowledge. We argue
that in order to enable and achieve multi-stakeholder activity with multiple knowledge
creation perspectives when finding solutions for sustainability challenges, we need to
have also multiple mediators. Next, we shall open up our view on cities as living labs,
as innovation platforms.

The city as a living lab / the city as an innovation platform

As the living lab is a widely used and dynamic approach, there seems to be no
consistent definition for a living lab as a concept (see e.g. Schuurman et al. 2012;
Leminen 2013 for reviews of the definitions). However, scholars have agreed on the
main characteristics of living labs: 1) users as co-creators in innovation processes,
2) experimentation in a real-life context, and 3) multi-stakeholder involvement (Äyväri
and Jyrämä 2017). The European Network of Living Labs (2018) provides a holistic
definition of living labs: “Living Labs are open innovation ecosystems based on a
systematic user co-creation approach that integrates research and innovation
activities in communities, placing citizens at the centre of innovation.” It has been
proposed that living labs are also intermediaries focusing on the mediation between
users, public or private organisations, with two main functions: (1) closing the pre-
commercial gap by manifesting initial demand for products and services, and (2)
orchestrating the actions of disparate actors in order to gain critical mass for the
creation of a product or service (Almirall and Wareham 2011, 88, 100).

In living lab projects, different professional identities and organizational cultures are
at play, being multi-stakeholder activities and bridging different expertise and
perspectives by definition (see Äyväri and Jyrämä 2017, Hakkarainen and Hyysalo
2016), the need for mediating, facilitation of joint understanding, knowledge creation
and enabling joint actions and aims, managing tensions and conflicts of interest
becomes evident. However, we will argue that no single mediator, actor, can carry on
the mediating alone. As previously mentioned, cities can create conditions for
building sustainability and take the role of an enabler. For example, the City of
Helsinki and the City of Espoo have opened up their data and neighbourhoods as
open innovation platforms (see e.g.,,

When considering cities as innovation platforms, we wish to highlight that there are
many types of public sector innovations: administrative or technological process

innovations, product or service innovations, governance innovations and conceptual
innovations (see more in de Vries et al. 2016). We maintain that to reach the UN
sustainability goals, cities need to find new ways of encouraging innovative actions
to boost especially technological process innovations, governance innovations and
conceptual innovations as less attention has been paid to them according to the
systematic review by de Vries et al. (2016). The conceptual model, the Circle of
Mediators, proposed in this paper, can be categorized as a governance innovation.
Next, we shall present the main framework of the study building on mediator studies
with insight from living lab actor discussions. Then we shall propose a novel
governance model, the Circle of Mediators, and discuss its adaptation in different
stages. We will conclude by pointing out the implications for researchers, cities and

2 Theoretical framework

The academic discussions on mediators is extensive. However, as the concept has

been approached from a multitude of perspectives, the discussions and
conceptualizations of the mediator concept have been rather incoherent. The concept
of mediators has been previously used in several contexts from national culture (e.g.
Karppinen-Takada 1994, Möller and Svahn 2004), cultural fields (Bourdieu 1984) and
consumer culture (e.g. Bourdieu 1984, McCracken 1986, du Gay et al. 1997) to
management, learning, knowledge-sharing (e.g. Wenger 1998) and teams (Kauppila
et al. 2011). The key conceptualization includes that some people or organizations
act between groups, organizations, communities, fields or nations. They can
introduce, interpret or integrate elements of one practice into another, or build a new
shared practice. Jyrämä and Äyväri (2005, 2007, updated 2015, see also Kantola et
al. 2010) have presented a categorization of different understandings of the mediator
concept (see Table 1, updated from Jyrämä and Äyväri 2015).

The different ways in which mediators have been conceptualized highlight the various
perspectives on mediating; the skills and competencies needed from the mediator,
bearing in mind both the overlapping similarities and the subtle differences dependent
on the theoretical perspectives and contextual setting, such as the types of actors
mediated or the timeframe.

It is worth noting that an actor (organization, individual and team) can act in several
roles simultaneously or during different stages of joint activity, and that their actions
determine their roles (see e.g. Heikkinen et al. 2007). Nyström et al. (2014) conclude
that open innovation requires network structures in which the role is negotiable
between actors. Roles depend on the situation and needs that the network’s goals

require. Hence, rather than look at the actor or mediator roles as static, we need to
look into the context and practices between actors, seeing the tasks and roles as
dynamic. The roles are not usually based on organizational or field structures but can
be adopted, made and changed by the actors, i.e. roles can be based on role taking
or more often on role making (see Nyström et al. 2014). Thus to highlight, mediating,
being a mediator, may take multiple forms, change according to time and context,
and can either be given to or made by the mediator him/herself.

Table 1. Categorizations of mediators


Authors Bourdieu 1984 Wenger 1998, Brown and von Krogh et al. Mittilä 2006 Jyrämä and Kantola et Jyrämä and Jyrämä and Äyväri
McCracken 2000; Brown Duguid 1998 1997, 2000 Äyväri 2007 al. 2010 Äyväri 2015 2015
1986 and Duguid
du Gay et al. 1998

Related Boundary Initiator (Mittilä Catalyst as a Ethnographer Evaluator

concepts spanners 2006) creator of
Knowledge Catalyst as structures
broker making (Ståhle et al.
Inward and something 2004)
outward happen (Ståhle
mediating et al. 2004)

Main tasks To create To act in the To frame the To bring To create To support To foster To actively To assess impacts
meaning. area of interests of different people structures and the identity learning learn the from different
To mediate overlapping one and groups facilities for building understood practices of CoPs’ points of
between communities of community in together to joint action. process (from as becoming other view
differing fields practice trying terms of create To create a newcomer especially in communities To translate the
or worlds. to build ties another knowledge. dynamic or novice to a the context of practice to impact on one CoP
To mediate between the community’s To create structures, professional) characterize be able to to the language of
between two perspective. spaces and e.g. networks. d by create joint another CoP
national communities. occasions for numerous activity at the To assess long-
cultures. To introduce joint actions. sub-fields or intersection term impacts
elements of the To make “mini-
practices of one something worlds”.
CoP into happen.
another CoP.

Similarly, in living lab discussions the tasks and roles of actors engaged in joint activity have
been defined and categorized (Nyström et al. 2014) from the role theory perspective (e.g.
Biddle, 1986; Biddle & Thomas, 1966; Broderick, 1999; Linton, 1936, all in Nyström et al.
2014). Their analysis of 26 living labs points out several roles that confirm roles found in
previous studies, such as webber, gatekeeper, advocate, or identify new roles such as
coordinator, builder, integrator (Nyström et al. 2014). The role of a gatekeeper is somewhat
similar to the role of a promoter proposed by Gemünden et al. (2007): the gatekeeper makes
decisions and influences others by possessing significant resources (Nyström et al. 2014).
Moreover, some roles are mainly connected to a particular stage of living lab activity, such as
the webber who initiates relationships by deciding who to contact, or the advocate who
distributes positive information about the jointly developed innovation (Nyström et al. 2014).
All in all, the identified roles overlap with the roles identified in Table 1.

As a role in the living lab context, mediating roles might change during different stages of the
shared practice (see also Jyrämä and Äyväri 2015). When starting joint activity, explorer or
ethnographer roles are needed to create knowledge and understanding of the joint challenge,
and different field and actors (Jyrämä and Äyväri 2015). An activist enables shared practice
at the beginning, but later mediating might transfer to other stakeholders. In addition, when
scaling the innovation/solution, an activist role is needed, and acknowledge the role of
gatekeeper hindering the entry of new required stakeholders or new resources. During joint
practice the role of a translator becomes important, especially if there are multitude of different
backgrounds and diversified understandings (see Nykänen et al. 2012, Järvensivu et al.
2011). However, one needs to bear in mind that assigning one mediating role to one stage
is over simplistic; each phase includes several mediator roles and tasks adopted by one or
many stakeholders.

Challenges in mediating

Previous studies (e.g. Nykänen and Jyrämä 2013, Kauppila et al. 2011) have identified
several dangers in the context of mediating to understand and manage multi-stakeholder
activities, especially in the so-called wicked challenge setting, like sustainability issues (e.g.
Michaels 2009, Agogué et al. 2017). First, perceived mediating as a task for one actor, where
each stakeholder then creates a strong relationship with the mediator, but not with each other,
creating a concentrated mediating structure (for further discussion, see also the network
approach). Next, to illustrate the challenge a mini-case is presented.

An illustrative mini-case

In a development project to support local elderly customers to get more personalized help,
i.e. tailored health care services were analysed using a Social Network Analysis (SNA). The
development project was city-led, aiming to create a service network of public, private and
third sector actors. The development has had several other goals, namely piloting self-
budgeting in a public setting, establishing care managers and building a customer-centric

service provider network for elderly care. The city was adopting the role of a mediator, aiming
to enable collaboration in-between city, private companies and the third sector in the context
of a distinct area and target group. However, the SNA showed that being a strong mediator
(city) resulted in a strong relationship with the city but few in-between other
actors/stakeholders. In Figure 1, the blue squares represent city actors, green the private
sector / business, and yellow the third sector, i.e. non-profit organizations.

Figure 1. The different sectors as groups and connections between these

groups (Nykänen and Jyrämä 2013)

Secondly, a challenge identified in several studies is the danger of power games. The
mediator might become a gatekeeper, for example, when the innovator or idea generator
builds strong attachment to his/her innovation/idea and lacks the skills and competencies to
build the network/team/project, not allowing the mediator to take charge of the process.
Third, as discussed earlier, a mediator needs different skills and competencies when in
different roles and phases, in a different context. Thus, seeing mediating as a task or role for
one actor creates enormous pressures and needs towards one actor alone. Negative
feedback and vicious circles are often created as a mediator fails to fulfil all the needs targeted
to him/her from various stakeholder groups. This danger becomes even more prominent when
talking about issues such as sustainability, which has multiple, often contradictory,

Fourth, often once a mediator has enabled smooth collaboration and all stakeholders have
developed strong relationships, there is a danger of becoming a closed loop – as the actors
do not wish to open up to newcomers or leave their comfort zone.

Fifth, the inability to disseminate, scale up and pass on the new knowledge, related to the

previous danger, the participating actors might feel confident, feel they have learned and feel
that they have adopted new knowledge themselves, but then lack the skills and motivation to
disseminate the new knowledge and understanding, to scale up the project’s outcomes.
However, we need to emphasize that many of the challenges identified relate to the difficulty
to adopt new multi-actor ways, that the mindset remains in the way of seeking individual
(organizational) success rather than success seen as jointly created and jointly benefitted –
looking at the activity truly at the ecosystem level.

3 The Circle of Mediators as a governance structure

Based on the theoretical discussion above, and on the challenges of mediating identified in
previous studies, we have constructed a new governance structure, the Circle of Mediators,
for a city to orchestrate and support multi-stakeholder innovation processes and the adoption
of innovation in the context of sustainable cities. The need for a new kind of governance model
was identified in the context of consultancy work executed by two of the authors for the two
biggest cities in Finland, namely the cities of Helsinki and Espoo. The commissions were
supported by the EU Cohesion Six City Strategy. In both cities the governance structure is
being piloted on the level of city units: the social and health care unit of Helsinki (Hirvikoski et
al. 2016) and the education unit of Espoo (Sutinen et al. 2016). The model contributed to the
City of Espoo being awarded the most Intelligent Community in the World 2018.

We will introduce the concept in three stages. We maintain that in the first stage, when the
new governance structure is implemented by the city, the Circle of Mediators will work within
a specific sector of the city, such as housing and environment. However, it is essential that
the members of the Circle of Mediators represent actors both from different sectors in the city
organization and outside the city organization. For example, it has been proposed that in the
Circle of Mediators orchestrating the living lab activities in schools there should be multiple
mediators. There should be one person who has strong competencies both in teaching and
development, two to three persons in managerial positions representing different levels of
education in the city, an expert in learning technologies, a representative of an accelerator for
start-ups creating learning solutions, a parent, and a research institution (Sutinen et al. 2016).

Figure 2. The Circle of Mediators in the implementation stage of the new governance structure (blue is
actors representing the city, dark blue is the key person, the others are external actors)

During the implementation stage of the new governance model, the members of the Circle of
Mediators have to concentrate on tasks related to the roles of cultural intermediaries, brokers
and translators (see Table 1), first and foremost among themselves. It is very important that
a shared vision and understanding of the role of the Circle of Mediators is reached before
starting any major activities. Furthermore, a thorough discussion on how each member of the
Circle of Mediators will act as a broker, a webber, an activist or an advocate in his or her own
networks is necessary. In short, gaining a shared meaning of mediating tasks is the main aim
during the first stage. Building trust within the Circle of Mediators is pivotal within innovation
ecosystems aiming for disruptive innovation based on shared resources and shared risk
taking. Figure 3 presents the actor groups of the Circle of Mediators in Social and Health Care
Unit of the City of Helsinki.

Figure 3. The Circle of Mediators in Social and Health Care Unit of the City of Helsinki (modified from
Hirvikoski et al. 2016)

In the second stage of the implementation of the new governance model, the members of the
Circle of Mediators support the innovation process in the context of a certain development
project, or orchestrate the activities within several small projects aimed at finding solutions on
the same challenge, e.g. so-called Agile Pilots Programme (Smart Kalasatama 2016). On an
abstract level, the Circle of Mediators can be described as a hub or a strategic net within
networks, see Figure 4.

Figure 4. The Circle of Mediators in action in the context of a city sector.

The Circle of Mediators orchestrates the innovation process from the very beginning, i.e.
discussions on the development needs (e.g. schools) which will be transformed into
challenges to be informed and opened up to all the potential co-creation partners. If
necessary, the members of the Circle of Mediators will act as explorers to familiarize
themselves with the challenge and the context where the joint development project will take

Every member of the Circle of Mediators is responsible for communicating the challenge in
his or her own networks. In this task, the mediator role of a translator is pivotal: each mediator
will translate the challenge and the characteristics of the joint development process into the
language of his or her own actor group or network. To be able to successfully play the role of
a translator, the mediator must have good knowledge of the substance of the challenge ‒
acquired when playing the role of an explorer or ethnographer. In addition, each mediator
plays the role of an activist: he or she encourages actors in his or her network to express their
interest to be involved in a living lab activity by sending proposals corresponding the

In addition to the key person, members in managerial positions are crucial in their role of a
promoter (Gemünden et al. 2007). Managers have hierarchical power to provide resources
for their staff to engage in living lab activities—for example, to hire substitutes and otherwise
help staff members to overcome practical obstacles. However, we wish to point out that, in
order to succeed in their role of a promoter, they must have the competencies of translators
being able to frame the interests of one community in terms of another community’s
perspective (e.g. why it is beneficial for teachers and students to start collaborating with start-
ups to develop new learning solutions). Without successful promoting and translating, it will
be hard to gather the development needs and challenges in workplaces around the city.
When the managers are not promoting living lab activities, they can be considered

gatekeepers, that is, hindering the active involvement of their staff.

The mediating tasks typical for the activist are needed when all the actors interested in
developing a solution for the challenge have been identified and selected. Activists create
spaces and occasions for joint actions to continue the innovation process. The Circle of
Mediators plays the roles of an activist and a translator at the beginning of the multi-
stakeholder innovation process; once the process has proceeded, e.g. co-creation workshops
are organized or functional prototypes are tested, the role of the Circle of Mediators changes
into the role of the supporter in identity building and cultivator of care. These roles and tasks
have gained less attention in the extant literature on management of innovation processes or
on orchestration of innovation ecosystems. All the actors in living lab activities are developers
and co-creators – identities that might be new and rewarding to many. Hence, the mediator
role of a supporter in identity building is very important to ensure that people with a forward-
looking mentality find pleasure in discovering new ideas and sharing positive ideas will create
a domino effect.

We argue that in addition to the managerial activities related to innovation processes and
resources, mediation among stakeholders and activities is needed to create trust and shared
meanings enabling sheared learning, a shared vision and shared value creation among the
multiple actors needed in living labs (see Figure 5). These aspects pinpoint the relevance of
the mediator roles of a cultural intermediary, a translator and a cultivator of care.

Phase I Phase II Phase III Phase IV Phase V

Shared meaning, shared vision, shared learning and shared value co-creation
Challenge Prototyping and Post-
Ideation Validation Pre-launch Launch
definition experimentation launch

Figure 5. Mediating in different stages of an innovation process

During and after the joint development project the competencies of the impact assessor are
valuable. For the Circle of Mediators, it is not enough to be able to evaluate the outputs and
outcomes of each development project from the city’s perspective; it is important to assess

the impacts from different actor groups’ points of view and, if necessary, to translate the
impacts on one actor group to the language of another actor group. As the orchestrator of the
living lab activities of a certain sector of the city, the Circle of Mediators pays special attention
to assessing long-term impacts on sustainability.

In the third stage, a city-level Circle of Mediators will be established. The key person in each
sector-specific Circle of Mediators will be a member of the city-level Circle of Mediators. The
city-level Circle of Mediators can be seen as an invisible hand or orchestrator on the city level.
The main task of the city-level Circle of Mediators is to create synergy among different projects
in different sectors of the city, and within different networks or clusters of actors in the city and
in the region. The members of the city-level Circle of Mediators are brokers between different
sectors or units of the city. They are also valuable advocates and messengers (Nyström et al.
2014) promoting the idea of a multi-actor development for finding new solutions for
sustainability challenges, and promoting the innovations co-created in ecosystems. The tasks
of an impact assessor are also high on the agenda of the city-level Circle of Mediators.

Finally, the all the Circles of Mediators of the city work for opening up the city as a living lab
for international actors. In the longer run, the city-level Circle of Mediators and the respective
sector-specific Circles of Mediators should be active in the network of transnational living labs.
In addition, the Circle of Mediators’ active participation in building consortiums to apply money
from the EU is needed to foster international collaboration.

4 Discussion

This paper concludes by arguing that the transformation of local and global value creation
structures requires new kinds of value cycles and inclusive ecosystems consisting of
companies, other operators and citizens (Mission-oriented research & innovation in the
European Union 2018). By investing EUR 500 million for ‘European innovation ecosystems’
through Horizon Europe, the EU countries aim to improve the Return of Investment in R&D&I
by boosting sustainable growth, jobs creation, inclusion and wellbeing through optimized
interaction among multidisciplinary stakeholder groups (European Commission). As the EC
Urban Agenda (2016) states: “EU, national, regional and local policies should set the
necessary framework in which citizens, NGOs, businesses and Urban Authorities, with the
contribution of knowledge institutions, can tackle their most pressing challenges.” With this
paper, we want to emphasize that sustainable ecosystems demand better orchestration in
and between individual RDI projects and stakeholder’s engagement activities.

Overall, our results demonstrate a strong need to transfer the locus of living lab research
towards collective innovation governance models, such as the Circle of Mediators. It would
support the ongoing transition from Smart Cities as linear innovation testbeds towards
interactive and iterative multi-stakeholder innovation ecosystems, Cities as Living Labs. The
city as a living lab has potential to engage multiple stakeholders and actors at every phase of

innovation co-creation and adoption; however, keystone players are needed to act as
orchestrators and to determine the “heart rate” of the ecosystem.

The key task of ecosystem orchestrator is to achieve commitment and the right balance
between control and enabling or inspiring (Curley & Salmelin 2018). It is our assumption that
this balance is rather based on collaborative mediation than innovation intermediary
organizations. This paper introduces the Circle of Mediators as a new model for innovation
ecosystem orchestration to support co-creation and experimentation within multiple and
transdisciplinary actors aiming for sustainable solutions.

Broadly translated, our analysis indicates that, in addition to cities, all the other living lab
stakeholders are natural beneficiaries of the Circle of Mediators. The circle orchestrates the
establishment of open innovation ecosystems and their innovation co-creation,
experimentation and adoption activities, thus making cities and communities inclusive,
resilient and sustainable, in accordance with the UN SDG 11 (United Nations 2015). The
Circle of Mediators help cities, their communities and stakeholders to become more
sustainable by organizing them as value cycles (Haque 2011). In value cycles, part of
stakeholders’ resources (e.g. knowledge, experiences, connections, people or facilities and
raw materials) are shared and used intensively without depleting them. In the value cycles,
stakeholders, such as local authorities, firms, NGOs, research institutions and universities,
exchange or recycle part of their resources and share costs and risks related to novel

The role of the Circle of Mediator is to design innovation operations so that each cycle pays
back and counterbalances the capital and costs of innovation and production. Moreover, the
collective governance approach increases trust-capital, making collaboration easier among
stakeholders, and therefore making the city as a living lab more attractive for people, for-profit
and non-governmental organizations, investors and researchers. Citizens are seen as
innovation partners. People with a forward-thinking mentality find pleasure in the discovery of
new ideas and will understand their city’s role as an innovation enabler and contribute by
sharing positive ideas and thus creating a domino effect. The aim of the collective governance
model is to release the underlying potential of innovation ecosystems, namely new business
opportunities and meaningful job creation.

Future research should further develop and confirm these initial findings by investigating, for
example, innovation governance models in the winning cities of the EC Innovation Capital
Award or the Global Intelligent Community Award. Comparative research on the winners’ and
their immediate hinterlands’ governance models would be important to strengthen territorial
inclusion. In addition, action research should continue to provide feedback for more detailed
Urban Agenda (2016) policy recommendations to balance between regulations and
deregulation making the dynamic innovation ecosystems more successful. Funding for
research to constantly evaluate the impact of cities as living labs and their governance models
is pivotal. Further research would be a natural starting point to develop tangible governance

tools, business models for multi-stakeholder ecosystems, and learning material to help
individuals, organizations and regions to adapt and navigate in the ongoing social,
technological and business model transitions.


Agogué, M., Berthet, E., Fredberg, T., LeMasson, P., Segrestin, B, Stoezel, M., Wiener, M.
and Yström, A. (2017). Explicating the Role of Innovation Intermediaries in the
“unknown”: A Contingency Approach. Journal of Strategy and Management, 10(1), 19‒
Almirall, E. & Wareham. J. (2011). Living Labs: Arbiters of mid- and ground-level Innovation.
Technology Analyses & Strategic Management, 23(1), 87‒102.
Bakici, T., Almirall, S. and Wareham, J. (2013). The Role of public Open Innovation
Intermediaries in local Government and public Sector. Technology Analyses &
Strategic Management, 25(3), 311‒327.
Bourdieu, P. (1984). Distinction. A Social Critique of the Judgement of Taste. London and
New York: Routledge & Keagan Paul. (Original 1979).
Curley, M. and Salmelin, B. (2018). Open Innovation 2.0. The New Mode of Digital Innovation
for Prosperity and Sustainability. Switzerland: Springer International.
du Gay, P., Hall, S., Janes, L., Mackay, H. and Negus, K. (1997). Doing Cultural Studies. The
Story of the Sony Walkman. London: Sage Publications.
EC Urban Agenda (2016). The Urban Agenda for the EU. European Commission
Espoo Innovation Garden (2018).
European Network of Living Labs (2018). What are Living Labs?
Accessed 4.7.2018
Fichter, K. (2008). Innovation Communities: The Role of Networks of Promoters in Open
Innovation. R & D Management, 39(4), 357‒371.
Gemünden, H.G., Salomo, S. & Hölzle, K. (2007). Role Models for radical Innovations in
Times of Open Innovation. Creativity and Innovation Management, 16(4), 408‒420.
Hakkarainen, L. and Hyysalo, S. (2016). The Evolution of Intermediary Activities: Broadening
the Concept of Facilitation in Living Labs. Technology Management Innovation
Review, 6(1), 45–58.
Heikkinen, M. T., Mainela, T., Still, J., & Tähtinen, J. (2007). Roles for managing in mobile
service development nets. Industrial Marketing Management, 36(7), 909–925.
Helsinki New Horizons (2018).
Hirvikoski, T., Lehto, P., and Äyväri, A. (2016). Development and experimentation platform
for social, health and wellbeing services in the context of Kalasatama health and
wellbeing centre. Laurea Julkaisut/Laurea Publications 68. ISSN 2242-5225, ISBN
978-951-799-441-5. Vantaa: Laurea-ammattikorkeakoulu.
Howells, J. (2006). Intermediation and the Role of Intermediaries in Innovation. Research
Policy, 35(5), 715‒728.
Jyrämä, A. and Äyväri, A. (2005). Can the Knowledge-Creation process be managed? A Case
Study of an Artist Training Project. International Journal of Arts Management 7(2), 4-
Jyrämä, A. and Äyväri, A. (2007). Fostering learning - The role of mediators. Knowledge
Management Research & Practice, 5(2): 117‒125.

Jyrämä, A. and Äyväri, A. (2015). Art encountering society; identifying the skills. 13th
International Conference on Arts and Cultural management, June 28-July 2, 2015; The
Institute of Public Management and Territorial Governance of Aix-Marseille University
and KEDGE Business School, Aix-en-Provence and Marseille, France.
Järvensivu, T., Nykänen, K. and Rajala, R. (2011). Verkostojen kehittämisen keskeiset
elementit ja verkostoitumisprosessi, in Niiniö, H., & Toikko, A., (eds) Muutosvoimaa
vanhustyön osaamiseen – hankkeen loppujulkaisu. Vantaa, Finland: Laurea University
of Applied Sciences.
Kantola, T., Lassila, S., Mäntylä, H., Äyväri, A., Kalliokoski, S., Ritalahti, J. and Soisalon-
Soininen, T. 2010. ”Shared Learning Spaces as Enablers in Regional Development
and Learning.” in Ekman, M., Gustavsen, B., Asheim, B. and Pålshaugen, Ø. (eds)
Learning Regional Innovation. Scandinavian Models. Basingstoke, Hampshire, UK:
Palgrave Macmillan, 206‒225.
Karppinen-Takada, M. (1994). Vertical and Horizontal Learning in Internationalization: Market
Penetration Patterns of Finnish Companies Operating in Japan. Helsinki: Helsinki
School of Economics.
Katzy, B., Turgut, E., Holzmann, T. and Sailer, K. (2013). Innovation Intermediaries: A
Process view on Open Innovation Coordination. Technology Analysis & Strategic
Management, 25(3), 295‒309.
Kauppila, O-P., Rajala, R. and Jyrämä, A. (2011). Knowledge sharing through virtual Teams
cross Borders and Boundaries, Management Learning, 42(4), 395‒418.
Leminen, S. (2013), “Coordination and participation in living lab networks”, Technology
Innovation Management Review, 3(11), 5‒14.
McCracken G (1986) Culture and Consumption: A theoretical account of the structure and
movement of the cultural meaning of consumer goods. Journal of Consumer Research
13, 71‒83.
Michaels, S. (2009). Matching Knowledge Brokering Strategies to Environmental Policy
Problems and Settings. Environmental Science & Policy, 12(7), 994‒1011.
A Mission-Oriented Research & Innovation Policy: A RISE Perspective (2018). Brussels:
European Commission. Directorate-General for Research and Innovation. Available
Mittilä, T. (2006) Intermediary Organization in a Regional Development Network. In The
Proceedings of the 22nd IMP Conference, 7-9.9.2006 Milan, CD-ROM.
Möller, K. and Svahn, S. (2004) Crossing East-West Boundaries. Knowledge Sharing in
Intercultural Business Networks. Industrial Marketing Management, 3, 219‒228.
Nykänen, K., Jyrämä A., Mangs, A. and Rajala, A. (2012).Facilitating innovation scalability through
network management. 28th IMP Conference, Rome, Italy, 13th-15th September, 2012.
Industrial Marketing and Purchasing Group (IMP).
Nykänen, K. and Jyrämä, A. (2013). Functioning network structures: The role of collaboration
process and their management. In The 29th IMP 2013 conference proceeding.
Nyström, A-G., Leminen, S., Westerlund, M., and Kortelainen, M. (2014). Actor Roles and

Role Patterns influencing Innovation in Living Labs. Industrial Marketing Management,
43, 483‒495.
Ojasalo, J. (2015). Open service innovation platform in a smart city. In R. P. Dameri & L.
Beltrametti (eds.) Proceedings of the 10th European Conference on Innovation and
Entrepreneurship. 17‒18 September 2015. Italy, Genova: University of Genova.
Saviano, M., Barile, S., Spohrer, J. C. and Caputo, F. (2017). A Service Research Contribution
to the Global Challenge of Sustainability, Journal of Service Theory and Practice,
27(5), 951‒976.
Schuurman, D., Lievens, M., DeMarez, L. and Ballon, P. (2012), “Towards optimal User
Involvement in Innovation Processes: A Panel-centered Living lab -approach”,
Proceedings of Technology Management for Emerging Technologies, Vancouver, 29
June-2 August, 2046‒2054.
Sieg, J. H., Wallin, M. W. and von Krogh, G. (2010). Managerial Challenges in Open
Innovation: A Study of Innovation Intermediation in the chemical Industry. R & D
Management, 40(3), 281‒291.
Smart Kalasatama. (2016). Kalasatama Agile Piloting Programme.
Sustainability, Wikipedia,
Sutinen, P., Erkkilä, K., Wollstén, P., Hagman, K. H., Hirvikoski, T. and Äyväri, A. (2016).
KYKY Living Lab Handbook for Co-creation by Schools and Companies. Espoo,
Finland: The City of Espoo.
United Nations (2015) Sustainable development Goals Sustainable cities,
Wenger, E. (1998). Communities of practice. Learning, meaning, and identity. USA:
Cambridge University Press.
de Vries, H., Bekkers, V., and Tummers, L. 2016. Innovation in the Public Sector: A
Systematic Review and Future Research Agenda. Public Administration, 94(1), 146‒
Äyväri, A. and Jyrämä, A. 2017. Rethinking value proposition tools for living labs. Journal of
Service Theory and Practice, 27(5), 1024‒1039.

The management of information and knowledge flows in
Urban Living Labs: a constructivist approach
Carolina Campalans1

1 Universitat Oberta de Catalunya

Category: Research in-progress

Though knowledge management has consolidated as a field of study since a long time, the
phenomenon has been approached in the context of formal organizations, mainly. In open
innovation ecosystems like Urban Living Labs, the creation and dissemination of knowledge
is a key issue, that nevertheless should not be approached with such restrictive conceptual
approaches as the traditional dichotomy of tacit/explicit knowledge or quantitative designs
exclusively focused on technological aspects. In this paper we introduce a work-in-progress
of a constructivist, interpretive multiple case study that aims to build an in-depth,
comprehensive explanation on the management of information and knowledge flows in the
context of Urban Living Labs.

Keywords: Urban Living Labs; Open innovation; Knowledge management; Knowledge flows;
Innovation clusters; Innovation networks

1 Introduction

How do Urban Living Labs (ULLs) manage the creation and diffusion of knowledge? This is
the central question posed by the research that we are currently developing. This paper
presents a general overview of the state of progress of our study, as well as a summary of
the state of the art, the research objectives and an abridged presentation of the theoretical-
methodological framework.

Open innovation ecosystems -like Living Labs, Urban Labs, Fab Labs, or Maker Spaces,
among others- proliferate in cities, especially in urban areas with a concentration of population
and problems of coexistence and/or sustainability: ‘Growing urbanization puts pressure on
both social and ecological systems’ (Baccarne, Logghe, Schuurman, & Marez, 2016),
boosting citizens and institutions to search for creative solutions, based on cooperation
between various actors with different skills, competences and resources.

Living Labs can be conceived as ecosystems of third parties (Almirall, Lee, & Majchrzak,
2014), aiming to implement open innovation (Kerry & Danson, 2016; Leminen, Westerlund, &
Nyström, 2012; Leydesdorff & Ivanova, 2016) at an urban scale. Social actors involved in
knowledge creation are thus ready to ‘meet and interact to catalise creativity, trigger invention,
and accelerate innovation across scientific and technological disciplines, public and private
sectors (…), in a top-down, policy-driven as well as bottom-up, entrepreneurship-empowered
fashion’ (Carayannis & Campbell, 2011, p. 330).

The structuring of these novel urban phenomena, such as the ULLs, is conditioned by
multiplicity: there are multiple stakeholders (Alex, 2011), and multiple managerial levels and
informational nodes (Carayannis & Formica, 2008b), too. Implied social agents are also
multiple and diverse: university, government, industry and civil society converge in the
innovation ecosystem, contributing their respective competences and skills to develop ‘co-
specialized knowledge assets’ (Carayannis & Formica, 2008a, p. xvii). In other words,
innovation ecosystems are network-centric (Zahra & Nambisan, 2011). Innovative products
and processes are developed and tested in urban space, but are replicable and scalable,
even to supra-local levels (Leminen, Rajahonka, & Westerlund, 2017). This latter statement
points to some further features of innovation ecosystems: they are solution-oriented
(Edwards-Schachter, Matti, & Alcántara, 2012) and citizen-centric (Leminen et al., 2017),
assigning cities a role as a ‘driver’ for innovative solutions (Cohen, Almirall, & Chesbrough,
2017) to urban living issues.

The multi-stakeholder partnership involved in the articulation of an innovation ecosystem is

the structural characteristic that has driven scholars and managers to adopt the metaphor of
a Triple Helix (university-industry-public administration) (Etzkowitz, 2003; Leminen et al.,
2012; Leydesdorff & Zawdie, 2010) or Quadruple Helix (Triple Helix, plus civil society)
(Carayannis & Campbell, 2009). ULLs represent one of the shapes this Helix can adopt.

Our research focuses on the management of knowledge flows in the frame of ULLs,
considered as outstanding expressions of the Triple Helix or Quadruple Helix model of open

The defining feature of ULLs is the methodology they apply to achieve innovative solutions
for real urban living problems (Dell’Era & Landoni, 2014; Scozzi, Bellantuono, & Pontrandolfo,
2017). This methodology, apart from being solution- and user-centred, stands out for using
the city as a laboratory (Cohen et al., 2017) to interactively test products, services or
processes in real life (Franz, 2015; Hirvonen-Kantola, Ahokangas, Iivari, Heikkilä, & Hentilä,

The role that information and knowledge flows play in urban labs’ procedures has been
identified by more than just a few researchers (Baccarne et al., 2016; Lehmann, Frangioni, &
Dubé, 2015; Westerlund & Leminen, 2011). Nevertheless, while theoretical investigation has
clearly stated this connection, empirical studies concerning ULLs tend to postpone knowledge
flows to a rather secondary position, putting instead more attention to structures and patterns
of actors’ interaction inside these complex networks (Georges et al., 2015; Juujärvi & Lund,
2015), or governance and decision-making (Gatta, Marcucci, & Le Pira, 2017; Voytenko,
McCormick, Evans, & Schliwa, 2016).

Another key aspect of ULLs is their intensive use of digital Information and Communication
Technologies (ICT) (Baccarne, Schuurman, Mechant, & De Marez, 2014). Digital ICT are, in
fact, imprinted in ULLs’ mission, management, and operation (Lehmann et al., 2015).
Therefore, it would seem a hardly justifiable omission not to consider ICT in an empirical
approach to the information and knowledge phenomena that take place within ULLs.
Nevertheless, material resources and formal information systems have traditionally received
a disproportionate attention in comparison with the social or cultural aspects of the processes
of knowledge creation and dissemination in organizations. This impels us to consider the
benefits that the application of a qualitative, interpretive and constructivist perspective could
bring to the field of knowledge management (Hislop, 2009).

This research project responds to a concern about the management of information and
knowledge flows in the context of the complex partnership networks that characterize
ULLs. We aim to offer both conceptual and methodological contributions to the field
of study. In terms of theoretical and conceptual development, the research aspires to
contribute not only to the topic of open innovation, but to the empirical studies of
information and knowledge flows, in that it widens the problem to encompass complex
networks (i.e., networks that are multi-dimensional, multi-level, and multi-faceted). On
the other hand, the methodological contribution would rest in an empirical, inductive,
in-depth explanation of the way knowledge is managed inside ULLs.

Two main questions arise from this problem. First, how are information and knowledge flows
managed in the framework of an open innovation ecosystem? And second, how does the

complexity of multi-partner networks impact on the management of information and

We can consider that there are three factors involved in the problem that motivates this
research: 1) The characteristics of the networks of actors and stakeholders that are the pillar
of open innovation platforms like ULLs. 2) Information and knowledge flows that contribute to
shape those networks, and the correspondent relation between these two phenomena. 3) The
management of those flows, which connects with the strategic dimension of open innovation.
The three factors previously enumerated give rise to three investigative objectives,

The first objective consists in describing and categorising the multi-stakeholders network of
an urban living lab. As an initial conceptual assumption, we consider that at least two
dimensions affect the complexity of these structures. On the one hand, we can identify a
territorial or spatial dimension (Canals, Boisot, & MacMillan, 2008), in the sense that open
innovation platforms, including ULLs, tend to form innovation nodes (Rickne, Laestadius, &
Etzkowitz, 2012), which would presumably have an impact on the production and circulation
of information and knowledge. On the other hand, a substantive dimension could be
categorized, meaning the affinity of topics or objectives pursued by the different innovation
platforms that lead them to constitute virtual networks, even when there is no geographical
proximity (Salvador, Mariotti, & Conicella, 2013). We consider that the issue of proximity,
whether geographic or thematic, can be better described and analysed with the help of geo-
localisation and data visualisation procedures, such as mapping techniques, for example.
Therefore, our methodological design includes some quantitative techniques for the analysis
and visualisation of networks and nodes.

The second objective of the research is to explain the information and knowledge flows that
take place in ULLs. This goal involves studying how both phenomena (information and
knowledge) are interrelated. At this point we see the need to address some recurring
concepts, such as tacit knowledge and explicit knowledge (Nonaka & Takeuchi, 1995). To do
so, we renounce the dichotomous perspectives that have usually characterized the field of
knowledge management and we lean, instead, toward interpretive (Hislop, 2009) and non-
incremental approaches (M. Boisot & Canals, 2004).

Finally, the third objective proposes to explain the decision-making processes involved in the
production and diffusion of knowledge in ULLs. This interest brings us to the problem of the
governance of innovation and knowledge (Bahlmann & Huysman, 2008; Bocquet & Mothe,
2010; Bulkeley et al., 2016), which involves strategic considerations about the management
of material resources (including technology, for example) and symbolic resources (like human
capital) and their impact on knowledge management.

2 State of the Art

Knowledge management has consolidated as a discipline with multiple and interdisciplinary

intellectual sources (Davenport & Prusak, 1998). Nevertheless, the study of knowledge flows
and the strategies, resources and procedures applied to manage them has been more
frequently approached in relation to formal and stable organizations. Unlike more traditional
organizations or corporations, ULLs can present different shapes depending on their
objectives or functions. Unlike more traditional organizations or corporations, ULLs can
present different shapes depending on their objectives or functions. They can rest on
temporary networks or stable organizations, with a structured org chart; they can consist in
an isolated project or a long-lasting strategy or policy. Regardless, they constitute multi-
dimensional, multi-partner and dynamic networks, according to most authors (Baccarne et al.,
2016; Leminen, 2015).

In their classic work, Nonaka and Takeuchi (1996) described the connection between
information flows and knowledge and, besides, highlighted the relevance of knowledge
management for organizations. In the field of organizational studies, knowledge management
has consolidated as a discipline for quite a long time (Davenport & Prusak, 1998). The
academic interest towards knowledge flows has gone beyond organizations boundaries,
reaching the level of societal context, especially since the paradigm of knowledge and
information society (Castells, 1996) has succeeded and established. Similarly, information
and knowledge flows have proved their impact on cities and urban living. This is an
underpinning idea at the bottom of i-City theory (Castells, 1989).

The link between knowledge and innovation results outstanding and primordial in
organizational contexts (Hislop, 2009; Swan, Newell, Scarbrough, & Hislop, 1999). At
innovation urban spaces, however, this link has been established in conceptual terms, mainly
(Hirvonen-Kantola et al., 2015; Westerlund & Leminen, 2011). Nonetheless, although a
minority, the empirical studies on ULLs also provide important observations on how
information and knowledge flows are handled in these urban innovation ecosystems
(Schuurman, Mahr, De Marez, & Ballon, 2015; Voytenko et al., 2016).

The topic has served as a conceptual basis for some case studies, such as the work of Scholl
and Kemp (Scholl & Kemp, 2016) or Gascó (Gascó, 2017). In these two cases, there is an
interest in describing the connection between learning processes and urban governance
policies. Gatta, Marcucci and Le Pira (Gatta et al., 2017) also position themselves at the
political or strategic level of decision-making. Their study reveals a pragmatic interest,
aspiring to modelling a strategy for the integration of new knowledge produced in ULLs within
the framework of the most general urban policies.

From a different perspective, more familiar to personal interactions, Ståhlbröst and Holst
(2017) have studied the learning processes and the adoption of digital innovation, applying
the same case study approach that characterizes the majority of research in ULLs. Also Sharp

and Salter (2017) realize the convenience of approaching the perspective of the users or
participants of a Living Lab to understand knowledge processes in these innovation
ecosystems. At this same level -the users- Edwards-Shachter, Matti and Alcántara (2012)
have studied the effects that the age gap may have on the adoption of business opportunities
between different types of participants of a Living Lab. Some research has focused on
identifying the factors that can affect the degree of involvement of the participants in an
innovation project (Buhr, Federley, & Karlsson, 2016; Sharp & Salter, 2017; Voytenko et al.,

In this sense, we could say that we have two levels in which the creation of "social value"
(Baccarne et al., 2014) can be potentially produced from the generation of new knowledge:
on the one hand, the level of strategic decisions that affect the collective of the citizens of a
determined ecosystem of innovation (that is to say, the level of the policies) and on the other
the level of the users and their own learnings. Two levels that should be considered in the
definition of the objectives of our own research, as well as in the theoretical-methodological
design and in the selection of data collection and analysis procedures.

If we are seeking more specific tools -with greater descriptive potential- to analyse the
information and knowledge flows among the partners of ULLs, we can see that there the
available analytical models are not many.

Leydesdorff et al. (2014) apply a ‘Triple Helix indicator’ to assess information exchange
among partners in innovation ecosystems. The production of organizational knowledge has
been explained in terms of a tacit/explicit relation (Davenport & Prusak, 1998; Nonaka &
Takeuchi, 1995, 1996) or a process of codification/abstraction/diffusion (M. Boisot & Canals,
2004). These models have been applied and validated in the framework of an organization,
but innovation ecosystems like ULLs are multi-dimensional (Drechsler & Natter, 2012;
Ståhlbröst & Holst, 2017), multi-level (Lehmann et al., 2015; Leydesdorff & Zawdie, 2010) and
multi-faceted (Alavi & Leidner, 2001) networks, which suggests the convenience of reviewing
the models of information and knowledge flows.

3 Methodological Design

Most investigations on ULLs apply a conceptual approach (Baccarne, Schuurman, & De

Marez, 2015). On the other hand, empirical studies on this subject tend to utilize a qualitative
and exploratory case study design (Baccarne et al., 2014; Gascó, 2017).

Wishing to give some continuity to this line of research, along with the aim of furthering the
findings about knowledge flows inside ULLs, we choose the multiple case study as our
research method. The appropriateness of this empirical approach has been proved along
several studies (Baccarne et al., 2015; Bakici, Almirall, & Wareham, 2013; Edwards-
Schachter et al., 2012; Scholl & Kemp, 2016). Though admitting that the methodological

limitations of a case study design lie in the fact that results can hardly be extrapolated to a
larger sample (Georges et al., 2015), researchers apply this method as a way to illustrate one
or a few cases in a real-life context (Baccarne et al., 2015; Bakici et al., 2013; Yin, 2009).
As we could observed in the review of the state of the art, researchers have identified at least
two levels of the relationship that links ULLs with the subject of the management of knowledge
flows: a level linked to policies and decision-making (strategic level) and the level of
interactions between individuals or groups (level of action or interaction). The theoretical-
methodological design of our multiple case study should approach both levels in some way.
Our research proposal aims to advance an interpretive and inductive explanation (Corbin &
Strauss, 1996; Miles & Huberman, 1994) of the management of information and knowledge
flows inside ULLs. To achieve this purpose, in-depth semi-structured (Eisenhardt, 1989;
Gillham, 2001) and multidimensional (Baccarne et al., 2015) case studies will be carried out.
Our research design has been inspired, too, by the qualitative methodology of multiple case
study developed by Leminen, Rajahonka and Westerlund (2017), and founded upon Yin’s
approach (Yin, 2009).

Collected data will be analysed through a mixed method, with both qualitative and quantitative
techniques (Quivy & Van Capmenhoudt, 1997), including mapping techniques and the
application of a location intelligence software to analyse the spatial dimension of knowledge
networks and to geo-reference the presence of nodes or clusters of open innovation labs.
This quantitative information will serve as the basis for the selection of in-depth case studies.
Our research involves several phases, as is usual in case studies. Currently, the research is
in an initial stage, in which the main objective is the exploration of the ULLs landscape to build
a system of categories that will allow us to define the profiles of the cases that will be analysed
in-depth. That is to say that the result of this first phase of the investigation will consist in the
definition of the number and type of ULLs that will be included in the analysis. This stage is
currently underway and therefore we do not yet have conclusive results, but we can provide
some research advances.

4 State of progress

The first phase (exploratory) of our research rests on the basis of the theoretical framework
and the state of the art of investigations on knowledge management in innovation systems.
This theoretical background reveals the relevance of network participation (Castells, 1989;
Cohen et al., 2017) for knowledge management in ULLs.

As a starting point for the exploratory study, we have chosen the Living Labs pertaining to the
ENoLL network and located in European Union countries. At this stage, the selection is based
on a documentary review of the information published on the official website of the network
(, exclusively.

This rapid documentary review helps us to verify the relevance of the Quadruple Helix model.

Although it also suggests that the complexity of information and knowledge flows in systems
integrated by four types of partners can be quite elevated. It could even be possible that
previous models (like TH indicator (Leydesdorff et al., 2014)) that have already been tested
to the study tof the Triple helix may not be directly applicable.

5 Preliminary results and conclusions

The review of previous studies on the management of knowledge and information in

organizations reveals three central points: first, that it consists in a consolidated field; second,
that research has privileged descriptive approaches and technical aspects; and third, that the
conceptual and methodological framework must undergo some updates when the object of
study is no longer a stable organization, such as a firm, but an open innovation platform, such
as urban living labs.

Our research, which is still at an early stage, is considered as a multiple in-depth case study
of how information and knowledge flows are managed in Urban Living Labs. The project
intends to make a theoretical contribution to the field of knowledge management, by
connecting this topic with the phenomenon of open innovation. In addition, it seeks to
contribute to the methodological field, applying a constructivist and interpretive approach, with
a combination of qualitative and quantitative techniques.

After the first steps of the exploratory research, we arrive to the conclusion that the ULLs to
be included in the multiple case study (qualitative in-depth research) must meet three
fundamental requirements. These requirements result relevant in accordance to the academic
literature and the information obtained from the preliminary documentary research (review of
the web):
i) They should be part of a network or cluster of open innovation systems, since this can
foreseeably affect the inputs and outputs of information (Bahlmann & Huysman, 2008)
and therefore the management of knowledge flows.
ii) They should have active projects, executed under the methodology of Living Lab,
which is characterized by real-life tests of innovative solutions to urban living issues
(Steen & Van Bueren, 2017).
iii) They should respond to the model of Quadruple Helix, that is, present a multi-partner
structure, with participants coming from the productive sector, academic institutions,
and government institutions (Triple Helix), plus the participation of representatives of
civil society (Carayannis & Campbell, 2009).

These criteria will allow us to advance in the definition of the profiles and the number of cases
that will be studied in depth in the following phase of investigation.

If we try to make a forecast of the difficulties that we may find in the phase of in-depth study,
we can think of the probability of a high degree of complexity of the relations of exchange of

information, since each case will present at least four types of partners (Quadruple Helix). In
this way, the application of a model such as the TH Indicator (Leydesdorff et al., 2014) will
probably present some difficulties and it would be necessary to adapt it.

On the other hand, we previously pointed out the conceptual interest that a non-incrementalist
model (such as Boisot's (1999) model of codification/abstraction/diffusion of information and
knowledge) can have; although it is also necessary to recognize the difficulties to take it to
the empirical field. But there is an aspect of this model that is especially interesting when it
comes to the study of multiparners systems: we refer to the problem of coding and abstraction.
These two phenomena are basic issues of the processes of knowledge creation and transfer
in an organization. In addition, we can state as an initial theoretical assumption that coding
and abstraction will present a high complexity when it comes to a network of collaboration
between participants with highly diverse characteristics and objectives, as in the Quadruple
Helix model (Campbell & Carayannis, 2010). One aspect that we must add to the operational
difficulties of Boisot’s model. Even so, the conceptual contributions of this model make it
advisable that we take it into account as part of the theoretical framework of the research. For
example, we can consider the question of how do an ULL's partners resolve the differences
among their respective linguistic codes or ontologies or the conceptual categorizations they

The nature of the problem here presented leads us to approaching this question from the
point of view of the various involved cultures. Therefore, we think that a qualitative
methodology, supported by the constructivist perspective, is the appropriate one to address
this study.


Alavi, M., & Leidner, D. E. (2001). Review: Knowledge management and knowledge
management systems: Conceptual foundations and research issues. MIS Quarterly,
25(1), 107–136.
Alex, H. (2011). Creating sustainable cities through knowledge exchange: A case study of
knowledge transfer partnerships. International Journal of Sustainability in Higher
Education, 17(6), 796–811.
Almirall, E., Lee, M., & Majchrzak, A. (2014). Open innovation requires integrated
competition-community ecosystems: Lessons learned from civic open innovation.
Business Horizons, 57(3), 391–400.
Baccarne, B., Logghe, S., Schuurman, D., & Marez, L. De. (2016). Governing Quintuple
Helix Innovation: Urban Living Labs and Socio-Ecological Entrepreneurship.
Technology Innovation Management Review, 6(3), 22–30.
Baccarne, B., Schuurman, D., & De Marez, L. (2015). Facilitating Quintuple Helix
innovation with Urban Living Labs. In Proceedings of XXVI ISPIM Conferences (pp.
1–16). Budapest: ISPIM.
Baccarne, B., Schuurman, D., Mechant, P., & De Marez, L. (2014). The role of Urban
Living Labs in a Smart City. ISPIM Conference Proceedings, (June), 1–17.
Bahlmann, M. D., & Huysman, M. H. (2008). The emergence of a knowledge-based view
of clusters and its implications for cluster governance. Information Society, 24(5),
Bakici, T., Almirall, E., & Wareham, J. (2013). A Smart City Initiative: The Case of
Barcelona. Journal of the Knowledge Economy, 4(2), 135–148.
Bocquet, R., & Mothe, C. (2010). Knowledge governance within clusters: The case of small
firms. Knowledge Management Research and Practice.
Boisot, M., & Canals, A. (2004). Data, information and knowledge: Have we got it right?
Journal of Evolutionary Economics, 14(1), 43–67.
Boisot, M. H. (1999). Knowledge Assets: Securing Competitive Advantage in the
Information Economy. New York: Oxford University Press.
Buhr, K., Federley, M., & Karlsson, A. (2016). Urban Living Labs for Sustainability in
Suburbs in Need of Modernization and Social Uplift. Technology Innovation
Management Review, 6(1), 27–33.
Bulkeley, H., Coenen, L., Frantzeskaki, N., Hartmann, C., Kronsell, A., Mai, L., …
Voytenko Palgan, Y. (2016). Urban living labs: governing urban sustainability
transitions. Current Opinion in Environmental Sustainability, 22, 13–17.
Campbell, D. F. J., & Carayannis, E. G. (2010). Triple Helix, Quadruple Helix and
Quintuple Helix and How Do Knowledge, Innovation and the Environment Relate To
Each Other? : A Proposed Framework for a Trans-disciplinary Analysis of
Sustainable Development and Social Ecology. International Journal of Social
Ecology and Sustainable Development, 1(1), 41–69.
Canals, A., Boisot, M., & MacMillan, I. (2008). The spatial dimension of knowledge flows: a
simulation approach. Cambridge Journal of Regions, Economy and Society, 1, 175–
Carayannis, E. G., & Campbell, D. F. J. (2009). “Mode 3” and “Quadruple Helix”: toward a
21st century fractal innovation ecosystem. International Journal of Technology
Management, 46(3), 201–234.
Carayannis, E. G., & Campbell, D. F. J. (2011). Open Innovation Diplomacy and a 21st
Century Fractal Research, Education and Innovation (FREIE) Ecosystem: Building

on the Quadruple and Quintuple Helix Innovation Concepts and the &quot;Mode
3&quot; Knowledge Production System. Journal of the Knowledge Economy, 2(3),
Carayannis, E. G., & Formica, P. (2008a). Key Working Concepts Defined. In E. G.
Carayannis & P. Formica (Eds.), Knowledge Matters: Technology, Innovation and
Entrepreneurship in Innovation Networks and Knowledge Clusters (p. xvii-sviii).
New York: Palgrave Macmillan.
Carayannis, E. G., & Formica, P. (2008b). Knowledge Matters: Technology, Innovation and
Entrepreneurship in Innovation Networks and Knowledge Clusters. (E. G.
Carayannis & P. Formica, Eds.). London: Palgrave Macmillan.
Castells, M. (1989). The Informational City: Information, Technology, Economic
Restructuring and the Urban-Regional Process. Oxford, UK: Blackwell.
Castells, M. (1996). The Information Age: Economy, Society and Culture. Oxford, UK:
Cohen, B., Almirall, E., & Chesbrough, H. (2017). The City as a Lab: Open innovation
meets the collaborative economy. California Management Review, 59(1), 5–13.
Corbin, J., & Strauss, A. (1996). Basics of Qualitative Research:Techniques and
Procedures for Developing Grounded Theory. London: Sage.
Davenport, T. H., & Prusak, L. (1998). Working knowledge: how organizations manage
what they know. Boston (Mass.): Harvard Business School.
Dell’Era, C., & Landoni, P. (2014). Living lab: A methodology between user-centred design
and participatory design. Creativity and Innovation Management, 23(2).
Drechsler, W., & Natter, M. (2012). Understanding a firm’s openness decisions in
innovation. Journal of Business Research, 65(3), 438–445.
Edwards-Schachter, M. E., Matti, C. E., & Alcántara, E. (2012). Fostering Quality of Life
through Social Innovation:A Living Lab Methodology Study Case. Review of Policy
Research, 29(6), 672–692.
Eisenhardt, K. (1989). Building theories from case study research. Academy of
Management Review, 14(4), 532–550.
Etzkowitz, H. (2003). Innovation in Innovation: The Triple Helix of University-Industry-
Government Relations. Social Science Information, 42(3), 293–337.
Franz, Y. (2015). Designing social living labs in urban research. Info, 17(4), 53–66.
Gascó, M. (2017). Living labs: Implementing open innovation in the public sector.
Government Information Quarterly.
Gatta, V., Marcucci, E., & Le Pira, M. (2017). Smart urban freight planning process:
integrating desk, living lab and modelling approaches in decision-making. European
Transport Research Review, 9(3), 1–11.
Georges, A., Schuurman, D., Baccarne, B., Coorevits, L., Georges, A., Schuurman, D., …
Georges, A. (2015). User engagement in living lab field trials. Info, 17(4), 26–39.
Gillham, B. (2001). Case study research methods. New York: Continuum.
Hirvonen-Kantola, S., Ahokangas, P., Iivari, M., Heikkilä, M., & Hentilä, H.-L. (2015). Urban
Development Practices as Anticipatory Action Learning: Case Arctic Smart City
Living Laboratory. Procedia Economics and Finance, 21, 337–345.
Hislop, D. (2009). Knowledge management in organizations: a critical introduction (2nd
ed.). Oxford: Oxford University Press.
Juujärvi, S., & Lund, V. (2015). Enhancing Innovation among Actor Roles in an Urban
Living Lab. In ISPIM Conference Proceedings (pp. 1–14). Manchester: The
International Society for Professional Innovation Management (ISPIM).
Kerry, C., & Danson, M. (2016). Open Innovation, Triple Helix and Regional Innovation

Systems: Exploring CATAPULT Centres in the UK. Industry and Higher Education,
30(1), 67.
Lehmann, V., Frangioni, M., & Dubé, P. (2015). Living Lab as knowledge system : an
actual approach for managing urban service projects ? Journal of Knowledge
Management, 19(5), 1087–1107.
Leminen, S. (2015). Q & A. What are living labs? Technology Innovation Management
Review, 5(9), 29–36.
Leminen, S., Rajahonka, M., & Westerlund, M. (2017). Towards Third-Generation Living
Lab Networks in Cities. Technology Innovation Management Review, 7(11), 21–36.
Leminen, S., Westerlund, M., & Nyström, A.-G. (2012). Living Labs as Open-Innovation
Networks. Technology Innovation Management Review, 2(9), 6–11.
Leydesdorff, L., & Ivanova, I. (2016). “Open innovation” and “triple helix” models of
innovation: can synergy in innovation systems be measured? Journal of Open
Innovation: Technology, Market, and Complexity, 2(1), 1–12.
Leydesdorff, L., Park, H. W., & Lengyel, B. (2014). A routine for measuring synergy in
university-industry-government relations: Mutual information as a Triple-Helix and
Quadruple-Helix indicator. Scientometrics, 99(1), 27–35.
Leydesdorff, L., & Zawdie, G. (2010). The triple helix perspective of innovation systems.
Technology Analysis & Strategic Management, 22(7), 789–804.
Miles, M. B., & Huberman, A. M. (1994). Qualitative data analysis: an expanded
sourcebook. London: Sage Publications.
Nonaka, I., & Takeuchi, H. (1995). The Knowledge-creating company: how japanese
companies create the dynamics of innovation. New York (N.Y.) [etc.]: Oxford
Nonaka, I., & Takeuchi, H. (1996). A theory of organizational knowledge creation. IJTM,
Special Publication on Unlearning and Learning, 11(7/8), 833–845.
Quivy, R., & Van Capmenhoudt, L. (1997). Manual de recerca en ciències socials.
Barcelona : Herder.
Rickne, A., Laestadius, S., & Etzkowitz, H. (2012). Innovation governance in an open
economy: Shaping regional nodes in a globalized world. Innovation Governance in
an Open Economy: Shaping Regional Nodes in a Globalized World.
Salvador, E., Mariotti, I., & Conicella, F. (2013). Science park or innovation cluster?:
Similarities and differences in physical and virtual firms’ agglomeration phenomena.
International Journal of Entrepreneurial Behavior & Research, 19(6), 656–674.
Scholl, C., & Kemp, R. (2016). City Labs as Vehicles for Innovation in Urban Planning
Processes. Urban Planning, 1(4), 89–102.
Schuurman, D., Mahr, D., De Marez, L., & Ballon, P. (2015). A fourfold typology of living
labs: An empirical investigation amongst the ENoLL community. In 2013
International Conference on Engineering, Technology and Innovation, ICE 2013 and
IEEE International Technology Management Conference, ITMC 2013.
Scozzi, B., Bellantuono, N., & Pontrandolfo, P. (2017). Managing Open Innovation in
Urban Labs. Group Decision and Negotiation, 26(5), 857–874.
Sharp, D., & Salter, R. (2017). Direct impacts of an urban living lab from the participants’
perspective: Livewell Yarra. Sustainability (Switzerland), 9(10), 1699–1713.
Ståhlbröst, A., & Holst, M. (2017). Reflecting on Actions in Living Lab Research.
Technology Innovation Management Review, 7(2), 27–35.
Steen, K., & Van Bueren, E. (2017). The Defining Characteristics of Urban Living Labs.
Technology Innovation Management Review, 7(7), 21–33.
Swan, J., Newell, S., Scarbrough, H., & Hislop, D. (1999). Knowledge management and

innovation: networks and networking. Journal of Knowledge Management, 3(4),
Voytenko, Y., McCormick, K., Evans, J., & Schliwa, G. (2016). Urban living labs for
sustainability and low carbon cities in Europe: Towards a research agenda. Journal
of Cleaner Production, 123, 45–54.
Westerlund, M., & Leminen, S. (2011). Managing the Challenges of Becoming an Open
Innovation Company: Experiences from Living Labs. Technology Innovation
Management Review, 1(1), 9–25.
Yin, R. K. (2009). Case Study Research: Design and Methods (4th ed.). Los Angeles, CA:
Zahra, S. A., & Nambisan, S. (2011). Entrepreneurship in global innovation ecosystems.
AMS Review, 1(1), 4–17.

Applying the Living Lab Approach for the Design of Public
Spaces– A Living Lab Case Study
Sonja Pedell1, Gareth Priday1, Alen Keirnan2,
Flavia Marcello1, and Andrew Murphy1

1Swinburne University of Technology, Australia

2RMIT, Australia

Category: Research Paper

Living Labs approaches emphasise the importance of real life contexts in public spaces and
service provision as an integral design component of the physical space including
technological interactions. We report on the findings and outcomes of a Living Lab Project -
the Waiting Room of the Future Project. This research focuses on a waiting room redesign
for Access Health and Community, a partner of the Future Self and Design Living Lab both
located in Melbourne, Australia. Waiting rooms are an example of a public space where the
physical design is an integral part of the service provision. The emotions of staff and patients
have a significant impact on service perception and a direct impact on overall patient
satisfaction. Previous studies concerning healthcare waiting rooms provide little emphasis on
current and desired emotional responses of stakeholders (e.g. staff, patients, and clinicians).
This research focuses on patients’ emotions and communication as an integral component
for re-designing patient experiences. We place emphasis on how stakeholders would like to
feel when engaging with future technology, spaces and services of a local community health
care provider. We illustrate our findings concerning stakeholder experiences using goal
models and how they can be translated into an innovative design. In considering the spatial,
service, and technology layer in one study at the same time we are able to come up with a
waiting room concept that has user needs permeating through all three layers truly addressing
the complexity of designing such spaces. Lastly, we offer a variety of co-creative methods
that are suitable to enable key stakeholder to communicate their emotions that connect
strongly service and technology delivery in healthcare settings.

Keywords: Living Labs, Health technology design, spatial health care design, user-centered
design, waiting room, co-creation

1 Introduction


The literature on Living Labs emphasizes co-creation with users, a focus on real-life setting,
quadruple helix innovation, multi-method adoption and multi-stakeholder approaches (e.g.
Eskelinen et al 2015; Veeckman et al).

Coorveits and Jacobs (2017) point toward the benefit of a Living Lab approach to capture “the
uncontrollable aspects of real-life environment.” (p. 26). Building on a model by Jumisko-
Pyykko and Vainio (2012) they note different aspects of real-life contexts including: temporal,
physical, technical / informational, social and task contexts. Nilsson and Ballantyne (2014)
suggest a similar notion that the co-creation of value is embedded within a physical
environment and that this “servicescape” incorporates aspects of the physical and virtual
service environment. They also note temporal aspects of context, such periodic crowding in
spaces such as airport check-in desks. “This means that the provider firm is not always able
to influence the value-creation process in ways they would like, unless they recognize the
underlying functional, technical, symbolic and social dimensions which frame the meaning of
any particular servicescape and how people interact within that space” (Nilsson and
Ballantyne, 2014, p.377). Heinonen et al (2010) argue that customers’ “use all input, current
and remembered to form an impression of value influenced by both cognitive and emotional
perceptions” and these impressions act as “an emotionally charged market in the customers
memory…” (p .537). In contrast, negative emotions can lead to value co-destruction between
customers and service providers (Smith, 2013).

While emotions play a significant part in value co-creation and potential co-destruction they
are often overlooked in the design process. Pedell et al (2017) demonstrate how emotional
goals modelling can be incorporated effectively into a Living Lab process.

In this project, we map Jumisko-Pyykko and Vainio’s context dimensions to a waiting room
setting (Table 1) to identify value and areas. Waiting rooms display a diversity of
considerations in the core dimensions of context and emotions in service provision When
making services innovations in a waiting or similar public space it is likely that there will be
changes to these aspects especially in the physical, virtual and emotional domains.

Table 1. Context dimensions according to Jumisko-Pyykko and Vainio (2012) applied to waiting room and
extended to include emotional aspects.

Temporal Peak times, delays, priorities, limited time slots and user
desires to have appointed time
Physical Comfort, wider variety of user needs and capacities. Privacy
and security needs
Social Wide variety of age, cultures, different social expectations for
service provision
Task Variety of services and associated tasks.
Technical Combination of information collection and dissemination.
/Informational Virtual (computers boards, etc) and physical paper, noticed
boards and so on.
Emotional Anxiety, prior experience, anxiety based on current illness /
problem, care for children, parents, etc.

Within the innovation context of a living lab, the “real-life context” is changing alongside the
technology. Emotions also play a significant role for staff, patients and their supporters in
waiting rooms and health care settings in general.

Physical, Temporal and Social aspects in the waiting room

Wait time has been identified as a barrier to accessing health care (Pomey, Forest, Sanmartin,
DeCoster, Clavel, Warren, Drew, & Noseworthy 2013). The ‘wait’ also has a negative impact
on patient satisfaction as perceived waiting time increases (Yeddula, 2012). Also, the
perceived wait time is affected when patients are waiting for unknown reasons, with anxiety,
in discomfort or in an unproductive state for an unspecified time (Karaca, 2011). While Joseph
et al. (2009, p. 9) outline that “distractions reduce anxiety” there is often little thought into how
this waiting time can be utilised meaningfully. The pressure of mounting queues also
permeates through to physicians, who in turn need to deal with acute and chronic conditions
from patients within a limited timeframe. Sherwin et al. (2013) highlight that limited
consultation timeframes can lead to patients feeling rushed resulting in unanswered questions
or incomplete information. Furthermore, staff can experience low morale (Knight et al., 2005)
derived from the difficulties in managing patient wait times, queues, and complaints. Here,
healthcare services are presented with a challenge to improve patient and staff satisfaction,
presenting an opportunity to study the role of emotions within the space of the waiting room.
Opportunities for redesigning the space and implementing innovative technologies services
are also presented.

Previous research surrounding the healthcare setting waiting room focuses on patient

perceptions of waiting room environments in relation to perceived quality of care (Arneill,
2002), staff efficiency (Ulrich et al. 2008) and the influence of design to create functional
environments that communicate brand attributes (Cooke, 1983). Gaps in the literature reveal
a limited focus on the role of emotions within the space of the waiting room. There are further
gaps where research does not discuss the practice of designing waiting rooms from the
perspective of end users emotions both in relations to access to services and spatial design.
The consequences of omitting how users feel when engaging with medical practice service
can result in dissatisfied experiences that do not reflect ideal emotional user experiences.

Technology in waiting rooms

Self-service technology is widely integrated in service-oriented industries to reduce operating

costs (Castillo-Manzano & López-Valpuesta, 2013) and improve efficiency for both
consumers and providers (Gelderman et al., 2011). Health providers have started adopting
similar check-in style kiosks touted as offering “greater convenience and privacy to patients,
while liberating staff to do more meaningful work” (Fallis, 2012, p. 339).

Furthermore, displaying waiting times on digital signage can help alleviate anxiety generated
by unknown wait times (Karaca, 2011; Nemschoff, 2015). Ideally this signage should be
prominently positioned in multiple locations (Labarre, 2011) so it is easily visible for everyone
in the waiting room Some health clinics are now allowing patients to check-in for an
appointment using a smartphone application (Kennedy, 2016). Technology can also be used
to educate patients about their health while waiting (Stripling and Richardson, 2016).
Providing wireless tablets in the waiting room, they found that patients were more likely to be
satisfied with their visit when they used their wait time educationally.

2 Waiting Room of the Future

Methods for Understanding Emotions in the Waiting Room

This section reports on the methods used to gain insights about existing and preferred
emotions surrounding service, technology use, information flows and spatial arrangement in
the waiting room. A multidisciplinary team consisting of a Human-Computer Interaction (HCI)
specialist, a digital media designer, a service designer and an architect collaborated and
worked closely with Access Health and Community. Multiple methods were used to develop
an understanding from patients and staff about current problems and future desires across
the different aspects of the servicescape. This resulted in a proposed design for a waiting
room of the future that integrated all inputs. Table 2 summarises the methods used in each
stage, whether they examined current state issues or desired state and their relationship to
context and the emotional elements in table 1.

Table 2: Methods used and their relationship to context and emotional elements.

Context and Service Element

Method Temporal dimension
Informational flows
Theme 1Efficiencies and Technology
Inefficiencies - Current Spatial Issues
Issues Service barriers
Snap-it Theme 2 Ideal Patient
Service - Current issues
and desired future state
Theme 3 Towards the Technology
Future – Desired future Spatial
state Emotional
Focus Group & Rich Spatial
Desired future state Technology
Emotional goal
Desired future state Technology
(patients only) Emotional

1: 20 Model Current state Spatial

Action research through Spatial
Future State
design Information

SNAPIT* (Method 1)

A photo documentation kit named SNAPIT* (Figure 1) was distributed to Access Health and
Community staff members. SNAPIT* was used as an early exploratory tool, uncovering issues
which staff members thought needed to be addressed by using photography as a way of
capturing stories and building narrative. Theme 1 asked staff members to capture images
concerning efficiencies and inefficiencies within the work place including barriers to workflow.
Theme 2 asked staff members to document images that prevent Access Health and
Community from giving their patients an ideal service, while Theme 3 looked toward the
future; focusing on how a patient should feel when engaging with Access Health and
Community. A total of seven SNAPIT* photo-documentation kits were returned after a period
of 2-3 weeks. The participants ranged from reception staff, through to upper management.
The results, especially in relation to spatial features, were a key element for the redesign of
the space.

Findings SNAPIT* Theme 1: Efficiencies and Inefficiencies

Technology of different kinds is key to daily work routines. For various reasons, multiple
patient information technology systems do not communicate with each other which means
that staff members are required to manage appointment requests for patients requiring
multiple services. One participant noted that the ‘appointments on multiple systems don’t talk
well to each other’ leading to confusion and time taken away from servicing patient requests.

Spatial features including the corridor width from the reception to the medical suites made
access difficult for people using prams and wheelchairs. Further, reception staff often direct
patients to the bathrooms, out of view from the reception desk. Staff suggested that the
‘relocation of the reception desk to the far back wall’ will open up the space more to reduce
congestion while reducing the need for reception staff to direct and navigate patients.
Distraction was also presented as a spatial inefficiency for reception staff who ‘have to watch
the door slowly close to ensure no one else enters’. Lastly, it was noted that reception staff
need to ensure that patients do not access restricted areas: ‘Patients are easily able to walk
through the building unattended if unnoticed by reception staff”, putting them into the
secondary role of ‘security’.

Figure 1. The SNAPIT* photo-documentation kit instructions.

Figure 2. An image of a low-tech communication tool returned in the SNAPIT* data.

Service (in)efficiencies were also discussed. One staff member presented an efficient paper-
based communication system between patient, doctor, and reception staff (see Figure 2).
After each appointment, the clinician writes fees, funding source and care provider on a card
as well as details for booking the next appointment. They pass it to reception to process, thus
facilitating communication between the clinician and reception staff (i.e. clinicians are using
their time for patient care, rather than admin tasks). It was also presented that patients can
be put on multiple waiting lists to access services based on the length of the waiting list, rather
than the patients’ priority need.

Findings SNAPIT Theme 2: Ideal Patient Service

Information Provision refers to types of information shared between Access Health and
Community and their patients. It is common practice for health care waiting rooms to make
health related brochures available to their patients. The SNAPIT* activity revealed ‘brochures
and information that is scattered and poorly organized makes it difficult for patients to find
information relevant to them” (Figure 3). One staff noted that “Brochures and flyers need to
represent the diversity of patients that access services”, and “there are number of irrelevant
or out of date brochures”. Reception staff are also put under a great deal of pressure to help
patients locate services that are often difficult to find, for instance, one participant commented
on regularly being asked to find the call number cards for nursing services.

Spatial Barriers can also prevent ideal service delivery. One staff member noted that the
height of the reception desk excludes people in wheelchairs and that the spatial location of
the reception desk enables other patients who are waiting for an appointment to overhear
private patient details. It was also expressed that “there is no private room in the reception
area for a distressed or unwell patient to wait in, away from other patients” and that the only
private waiting area doubles as the Needle and Syringe Room (NSP). This is “however not
ideal for an individual experiencing trauma given the potentially confronting nature of the NSP
information and disposal bin” as stated by one clinician. Flow of people was indicated as
problematic given narrow pathways from the waiting area to GP suites (Figure 4). The narrow
space does not allow people who are using prams, walkers, or wheelchairs to use the narrow
hall simultaneously. It was reported that the door to the reception area opens straight onto a
busy road, causing concern for the safety of toddlers and pre-schoolers.

Figure 3. Information overload Figure 4. Narrow pathways

Findings SNAPIT Theme 3: Towards the Future

Towards a digital future. Staff presented an idea to include self-check in technology as a

preferred digital intervention: “The idea of this would be to reduce wait lines at reception and
to give individuals an element of privacy when communicating details”. Participants refined

the idea of the check-in kiosk, so that it could also be available for those who are unable to
use the digital check-in [iPad], or would prefer to speak with the receptionist by including the
dual option for both digital and face to face check-in. An interactive display was also
considered by participants as ideal for “keeping children entertained without having small toys
left around” (see Figure 5 and Figure 6), as well as making Wi-Fi available for all patients. Not
knowing how long a patient needs to wait and changes in scheduling can cause anxiety and
unnecessary stress for patients and (consequently) staff so digital tools can reduce stress
levels in patients and give them more control.

Towards a refined spatial future. Participants noted that one goal for Access Health and
Community should include furniture that “has been approved by Occupational Therapists for
people with different mobility needs”. This was an important guiding principle for the
designers. Furniture should also include “self-soothing items (such as stress balls) which can
help to reduce anxiety and distract people as they wait”. As activities at the reception desk
can be overheard in the waiting room, one participant suggested that the reception desk
should be moved away from the waiting room, to enhance privacy. One participant even
suggested that a varying level reception desk would give patients who might not be able to
view the staff behind the counter, an opportunity to do so comfortably. As a result, the design
and location of the reception desk was the primary focus of the re-design (Figure 15). It was
also suggested that the nearby corridor be widened better lit to allow people to see who is
behind and in-front of them. When security was discussed as a future goal, it was indicated
that installing “a swipe security door between the main reception area and clinical rooms”
would limit the number of patients who may become lost when trying to find the toilets and
also reduce theft of personal staff belongings.

Towards an ideal future service for patients. It was recommended that “a needle and syringe
disposal bin and vending machine for individuals to access sterile equipment could be fixed
to the outside of building”. All staff participants preferred a more accessible model that moves
away from the largely ‘one size-fits-all approach’ to a more tailored service. According to
participating staff a future goal for Access Health and Community should be that “Patients
can access the services in a number of ways and receive care for their priority need in a timely
way” (See Figure 8 opposed to a ‘waiting in one-line model’). Also, patients who need to see
multiple care providers should receive coordinated, and integrated care that prioritized
patients based on their needs. Lastly, one participant noted that “the service must provide
equitable access to health care for population groups who are marginalized and those who
experience poorer health outcomes”.

Figure 5. Existing childrens’ waiting space Figure 6. An example of an interactive child space

Figure 7. Dual use of needle syringe disposal room can cause anxiety in some patients
Figure 8. Vision of patients accessing services in a flexible way in the future – ducks representing patients.

Focus group (Method 2)

Participants and approach. After identifying important themes from the SNAPIT* photo
documentation kit, a focus group with five staff members was held to better understand the
context where the themes were placed. Discussions were framed around two activities: The
first asked participants to share their ideas, feelings, and emotions with specific focus on
service delivery, technology adoption and goals for the future. The second saw these ideas,
emotions and feelings represented on a large floor plan of the waiting room area at Access
Health and Community.

The focus group, with emphasis on “do”, “be” and “feel”, supported the development on the
emotional goal model (Figure 9). Goal model 1 presents a broader landscape of preferred
emotions experienced at Access Health and Community. The key stakeholders are: patients
who access the service and their supporters (e.g. family), the management team who are
responsible for co-ordinating and planning (or ‘back of house’), the reception staff who are

the crucial ‘front of house’ and then the nurses and clinicians. The function of the waiting room
is to provide a welcoming environment for all patients to access services. It is important to
note that a welcoming environment, as referenced by one participant is one that “is supportive,
encouraging, non-judgmental, and empowering. [Where] we embrace human diversity and
welcome and support all people”. The need for a welcoming environment was later a key
consideration when choosing floor materials and wall treatments. To support ideal service
delivery for patients, reception staff expressed the need to have flexible patient intake and
appointments that feel simple to manage and are responsive to change. Further, a
consolidated health record that is streamlined and up-to-date will support the role of reception
staff in delivering an ideal service to patients. Similarly, nurses can help facilitate the flexible
patient intake through communicating between appointments, reassuring the patient and
providing helpful information.

When providing a health care service to a patient, it should feel ‘caring’, while also presenting
Access Health and Community as having ‘best practice’ qualities. To provide health care
services that accommodate qualities of ‘caring’ and ‘best practice’ it is recommended that
general practitioners and specialists operate with unified and shared care plans that feel
interactive, are patient centered and individually tailored.

Figure 9. Goal Model 1 - Emotions in the Waiting Room

Findings of the SNAPIT* Photo-documentation kit showed that a consolidated, shared health
record was an integral goal towards reducing barriers. Hence, the workshop discussed the
preferred emotional goals of a consolidated health record (Goal model 2). The key feature of
a consolidated health record is to function as a ‘patient access point’ allowing patients to
access their own health history. The consolidated health record should be helpful in sourcing
the information that a patient requires, be secure, and should empower a patient while also
re-assuring them of their health history and private information. For staff, a consolidated
health record would enable all medical staff to create and provide unified and shared care
plans that can be accessed by designated health professional linked to the patient. These
care plans should feel holistic, enabling medical staff to include and access data pertaining

to the patient, while also having a patient centered quality. Similar to goal models 1 and 2, a
consolidated health record should enable reception staff to facilitate flexible patient intake,
make appointments that feel simple to manage and are responsive to change. Further, a
consolidated health record that is streamlined and up-to date will support the role of reception
staff in delivering an ideal service to patients. This information was crucial for the interaction
design students when putting together the App.

Figure 10. Goal Model 2 – Emotions and Patient Information Systems Goal Model

The waiting room paper and online survey with patients (Method 3)

A total of 70 paper and 26 digital surveys were completed by patients about (i) current and
preferred emotions and technology use in the waiting room and (ii) giving and receiving
information from Access Health and Community. The results from these surveys provided
important data for the students and researchers to explore both technological and spatial

Table 3. Key themes from paper survey: preferred emotions (left)
and technology use during check-in (right)

Patients wanted to feel welcomed and comfortable (Table 3 left). The survey revealed that a
more private waiting room would facilitate this. Staff suggested that self-check-in technology
was considered a faceless option, suggesting that many patients appreciate personal
interactions within their overall waiting room experience (Table 3 right). It was concluded that
if technology were introduced in the waiting room, it should support face-face interaction
rather than replace it.

Some participants wanted to do nothing while waiting, others preferred to be productive by

using technology such as smart phones to do work, use apps, or watch television, preferably
with free Wi-Fi. This information guided the interior design students to create different zones
such as the ‘Zen Zone’, the ‘Work Zone’ with hot desks, the ‘Family Zone’ and the central
seating zone. In its current state, participants mostly used their phone or read magazines in
place of alternative activities (Table 4 left). Participants placed strong emphasis on technology
to alert patients about changes or delays in their appointments (Table 4 right).

Table 4. Key themes from paper survey: Spending time while waiting (left) and technology use for updates

Tailoring Information for Access Health and Community patients was an important insight.
Currently, information is provided as brochures or information pamphlets that have been
described as ‘out of date’ and largely ‘one-size-fits all’ in accordance with staff comments. An

overwhelming 85% of participants expressed that they would like to have information tailored
to their health needs, including the types of services that are relevant to their needs.

When asked about what role Access Health and Community should play in supporting
patients on their health journey, three quarters of the participants reported that they would
like support in taking a more active role. This was considered in the technology design.
However, participants did not feel as strongly when compared to the provision of tailored
health information with one third partially agreeing and 20% neither agreeing nor disagreeing
on this topic.

Concerning privacy and access to sensitive patient information, four participants responded
that not all health services provided by Access Health and Community should have access to
their personal medical data. Potentially highlighting a desire for more unified health care (and
less repetitive questioning).

There was consensus amongst 80% of participants that if they knew the reason for
appointments running late they would not mind as much (Figure 11). This is consistent with
the literature (Karaca, 2011). Two participants strongly disagreed with the statement, perhaps
wanting action (text message, displayed wait times) rather than just a reason.

Fig. 11. Communication of reasons for running late (“strongly disagree” left to “strongly agree” right).
Fig. 12. Information provision on health via iPad (“strongly disagree” left to “strongly agree” right).

The use of iPads in the waiting room to provide and receive information about health and
health data received a somewhat positive result with just over 50% of participants wanting to
use technology for this reason (Figure 12). However, there was a large amount of indifferent
responses with 30.77% of participants neither agreeing nor disagreeing.

Finally, participants were asked to rank five statements in order of priority. Using ascending
mean value “Taking an active role in my own health journey” ranked as the most important
(2.4 mean, 21 votes) followed by “Receiving information tailored to my health needs” (2.65
mean, 23 votes). “Using technology to provide and receive information about my health data”
ranked lowest (3.4 mean, 25 votes). “Knowing why my scheduled appointment is running
late” yielded mixed results. It was the highest priority for 7 participants (equal with “Taking an
active role in my own health journey”), but the lowest priority for 9 participants being at third
position (3.1 mean, 26 votes).

3 Outcomes

Technology Design

Four students (Alexandra Mold, Elléna Mills, Nicole Matthew, Sarah Morris (Bachelor of
Design Honours) designed an application for mobile devices derived from the data collected
across the multiple methods into a prototype. The Access Health and Community App aims
to encourage patients to become more active in managing their personal health with the aim
of improving overall communication with Access Health and Community. The result was as a
fully functioning prototype based on the screen designs (Figure 13). In response to the data
collected, this application would enable patients to view their appointment calendar, book,
cancel and reschedule appointments. Patients are also able to input health statistics like
weight and blood pressure to enable better communication with clinical staff. The main benefit
to tracking personal health information is that patients can review their progress and feel
motivated to change/continue their healthy habits. The ‘Follow up Care’ feature allows them
to see referrals, access results and, most importantly, a written version of doctor’s instructions
as to address the concern of verbal instruction being forgotten, misunderstood or

Figure 13: Appointments, Track Your Health

and Follow Up Care as main functions of the suggested app

Spatial Redesign

Based on the findings, two action research design studios undertook the task to re-design the
waiting room space with students from the Bachelor of Interior Architecture (Honours)

Fig. 14: The entry view – final concept

In the first studio, 16 students developed individual concepts for the waiting room based on
the SNAPIT* exercise, goal modelling, survey data and the published research report. Each
student addressed the principal problems of flow and circulation (spatial layout), universal
access requirements and the need to provide a welcoming atmosphere supported through
use of materials and manipulation of light and shade. Staff then selected four schemes to be
further developed in a second studio. Students worked in teams to refine the designs based
on the outcomes of a co-design workshop with task that highlighted three key points: the
position and accessibility of the reception desk, the integration of technology and the location
of the Needle Exchange room. (see Figure 14). The reception desk designs feature different
heights for accessibility, afford complete views of the room for security and separate sitting
areas for privacy.

Students were given a set of empathy provoking exercises through mobility devices and
ageing suits in a full-scale model of the waiting room that was set up in their classroom space.
The students role-played being patient, carer and staff and reported that the emotions
triggered by this exercise were fundamental to their understanding of the different user groups
of the waiting room. The experience strongly affected their design approach. Staff and
executives agreed on the strongest scheme but also indicated key features from the others
that they wanted included.

Fig. 15: Final design – view from reception desk area (different heights)

Three students (Fiona Nowland, Chrissa Drosopoulos and Sarah Tucker) then worked in
close collaboration with the architect to refine the design to a single project that was approved
by management for execution (see Figure 15). It is anticipated that construction will
commence in 2019.

4 Conclusions

Health care is an extremely personal and private practice. Individual experiences and
expectations cause each patient to develop a unique perspective on their health, and what it
means to visit a health care practitioner. This research focused on the emotions of both
patients and designers, following a Living Lab approach in collaboration with our partner
Access Health and Community. Perceived waiting time and the feeling of being welcome are
key to improving patient experience and satisfaction and informed the new spatial design.
Improved communication between the patient and their clinic leads to a stronger bond,
forming a safe, trusting relationship which can be maintained via the suggested app solution.
Understanding these influential factors in patient empowerment is a central component
improving the individual experience in an innovative waiting room.

The description of the research of interactions within the “waiting room” did start and end at
the waiting room doors. There are physical and virtual interactions outside the waiting room
that this publication did not incorporate, but our research touched on such as missed
appointments, patient information between visits provided by the app and parking. Future
work will incorporate these aspects to further improve the experience of patients and staff
within an innovative waiting room permeating further into the lives of the patients.

This research is unique in that it adds to a limited body of literature about emotions and design
related to health care environments. Furthermore, it shows how to ‘design for’ emotions to
improve efficiency and usability across spatial, technology and service delivery problems in

health environments.

5 Acknowledgements

We would like to thank all Access Health and Community staff who have participated
throughout the co-creation process for their valuable input. In particular, we wish to thank
Christine Jones and Rae Pang for continued feedback and support throughout all parts of the
Waiting Room 2020 Project. Thank you to Harry Majewski for initiating this collaboration with
the Future Self and Design Living Lab. We also acknowledge the excellent work of the
Swinburne University of Technology Design students: Alexandra Mold, Elléna Mills, Nicole
Matthew, Sarah Morris (Bachelor of Design Honours) who have created the prototype app
based on user-centred research and Fiona Nowland, Chrissa Drosopoulos and Sarah Tucker
(Bachelor of Interior Architecture (Honours)) who created the final spatial redesign of the
waiting room.


Arneill, A. and Devlin, A, S. 2002. Perceived quality of care: the influence of the waiting room
environment. Journal of Environmental Psychology, 22(4), 345-360.
Castillo-Manzano, J. I., & López-Valpuesta, L. (2013, November). Check-in services and
passenger behaviour: Self service technologies in airport systems. Computers in
Human Behavior, 29(6), 2431-2437.
Coorevits, L., & Jacobs, A. 2017. Taking Real-Life Seriously: An Approach to Decomposing
Context Beyond “Environment” in Living Labs. Technology Innovation Management
Review, 7(1): 26-36.
Eskelinen, J., Robles, A., Lindy, L., Marsh, J., Muente-Kunigami, A., 2015. Citizen-Driven
Innovation – A guidebook for mayors and public administrators. World Bank,
Fallis, J. (2012, April 17). Touch the screen now to see a doctor. Canadian Medical
Association Journal, 184(7), 339.
Gelderman, C. J., Ghijsen, P. W., & Diemen, R. v. (2011, September), Choosing self-service
technologies or interpersonal services—The impact of situational factors and
technology-related attitudes. Journal of Retailing and Consumer Services, 18(5), 414-
Heinonen, K., Strandvik, T., Mickelsson, K.J., Edvardsson, B., Sundström, E., Andersson, P.
(2010) "A customer-dominant logic of service", Journal of Service Management, Vol.
21 Issue: 4, pp.531-548, https://doi-
Joseph, A., Keller, A., & Gulwadi, G. B.. (2009, March). Improving the Patient Experience:
Best Practices for Safety-Net Clinic Redesign. Center for Health Design. Retrieved
May 29, 2016, from http://www.
Jumisko-Pyykko and Vainio. 2012. Framing the Context of Use for Mobile HCI. In J. Lumsden
(Ed.), Social and Organizational Impacts of Emerging Mobile Devices: Evaluating Use,
2: 1–28. Hershey, PA: Information Science Reference.
Karaca, M. A., Erbil, B., & Özmen, M. (2011). Waiting in the Emergency Room: Patient and
Attendant Satisfaction and Perception. European Journal of Surgical Sciences, 2(1),
1-4. Retrieved May 24, 2016, from fu_folder/2011-
Kennedy, B. (2016, March 18). Melbourne technology gives doctor’s waiting room hi-tech
overhaul. The Herald Sun. Retrieved June 1, 2016, from
Knight A. W, Padgett J., George B. and Datoo, M R. (2005). Reduced waiting times for the
GP: two examples of “advanced access” in Australia. The Medical Journal of Australia
(MJA) 183(2)
Labarre, S. (2011, August 16). Six Ways To Improve Doctor’s Waiting Rooms. Retrieved May
21, 2016, from http://www.fastcodesign. com/1664797/six-ways-to-improve-doctors-

Nemschoff (2015). Winning Strategies for Waiting Rooms. Retrieved May 23, 2016, from
files/Nemschoff_Insight_Winning_Strategies_for_Waiting_ Rooms_2015_01_13.pdf
Nilsson, Elin ; Ballantyne, David. 2014. Reexamining the place of servicescape in marketing:
a service dominant logic perspective, Journal of Services Marketing, 19 August 2014,
Pedell, S., Keirnan, A., Priday, G., Miller, T., Mendoza, A., Lopez-Lorca, A., et al. 2017.
Methods for Supporting Older Users in Communicating Their Emotions at Different
Phases of a Living Lab Project. Technology Innovation Management Review, 7(2): 7-
Pomey, M.P., Forest, P.G., Sammartin, C., DeCoster, C., Clavel, N., Warren, E., Drew, M.
and Noseworthy, T. 2013. Towards systematic reviews to understand the determinants
of wait time management success to help decision-makers and managers better
manage wait times. Implementation Science, 8:61.
Sherwin, H. N., McKeown, M., Evans, M. F., & Bhattacharyya, O. K. (2013). The waiting room
“wait”: From annoyance to opportunity. Canadian Family Physician, 59(5), 479–481.
Smith, Anne M. European Journal of Marketing; Bradford Vol. 47, Iss. 11/12, (2013): 1889-
Stribling, J. C. and Richardson, J. E. 2016. Placing wireless tablets in clinical settings for
patient education. Journal of the Medical Library Association, v.1-89.
Ulrich, R.S., Zimring, C., Zhu, X., Dubose, J., Seo, H-B., Choi, Y-S., Quan, X., Joseph, A. A
review of the research literature on evidence-based healthcare design. Health
Environments Research and Design Journal. 1, 3 (2008), 61-125.
Veeckman, C., Schuurman, D., Leminen, S. & Westerlund, M. 2013. Linking Living Lab
Characteristics and their Outcomes: Towards a Conceptual Framework. Technology
Innovation Management Review, 3(12): 6-15.
Yeddula, V. (2012, August). Healthcare Quality: Waiting Room Issues. Industrial and
Management Systems Engineering -- Dissertations and Student Research. Retrieved
May 23, 2016 from http://

Comparison of Health and Wellbeing Living Lab Business
Models – Preliminary result based on Business Model
Canvas Evaluation
Teemu Santonen1, Mikko Julin1

1 Laurea University of Applied Sciences

Category: Research in-progress

The most popular thematic focus area among ENoLL members is the health and wellbeing,
which covers nearly a half of all the certified Living Labs. However, the studies evaluating
Living Lab (LL) business models in general and especially those focusing on health and
wellbeing Living Labs are rare. Therefore, the aim of this study is to assess industry standard
Business Model Canvas (BMC) usefulness to compare LL business models. The business
models among 12 Baltic Sea Region Health and Wellbeing LLs were described and
compared. After the recoding process, the original 241 different BMC attribute spellings were
reduced to 87 different attributes. On the average 26.7 attributes were used to describe the
LL business model. As an example, two network illustrations are presented to showcase the
different Living Lab business models. As a result of the assessment, the need to develop LL
specific business modelling tool based on fixed a set of valued attributes was recognized.

Keywords: Living lab; Health, Wellbeing, Business model, Business Model Canvas, Lean
Canvas, Service Business Model Canvas, Service Logic Business Model Canvas, Baltic Sea

1 Introduction

The number of officially certified Living Labs (later LL or LLs) have been steadily growing
since the launch of European Network Living Labs (ENoLL) over decade ago (Garcia Robles
et. al. 2016). Historically there have been nearly 400 officially recognised LLs across the
world. Currently there are 170 active LL members in ENoLL. Importantly, the most popular LL
thematic focus area among ENoLL members is health and wellbeing. This thematic area,
covers 44% of all ENoLL LLs (N=74). However, a great majority (78%, N=58) of the Health
and Wellbeing LL (later HWLL) are also operating in various other domains such as “Smart
cities & Regions”, “Culture & Creativity”, “Energy, Mobility”, “Social inclusion”, “Social
innovation”, “E-government” or “Education”.

According to ENoLL’s definition, LLs are operating in the real-life environments together with
end-users and various other relevant stakeholders while utilizing various research and
development methods. As a result, there are multiple implementations and a great variety of
locations where LLs are operating. Therefore, it is suggested that there are also multiple
business models, which can significantly differ between the LLs. However, studies focusing
on LL business models are relatively rare, and therefore the aim of this study is to identify and
compare what kind of business model’s health and wellbeing LLs are currently following.

2 Living Lab and Business Models

Tools to Evaluate and Develop Business Models

Business model (Osterwalder, 2004) and especially business model innovation (Chesbrough,
2010) as a research domain are both relatively young phenomena. The prior literature reviews
have argued that there is no overall definition for business model (Zott et al., 2011). Generally
speaking a business model is a method in which an organization builds and uses its resources
to offer their customers better value than their competitors, and make profit by doing so (Afuah
and Tucci, 2001). Business model combines potential environmental factors and
organization’s capabilities in order to define and implement a sustainable recipe for
competitive advantage.

Multiple theoretical models to evaluate and develop business models have been suggested
especially in a form of “canvas”. Canvas approach became extremely popular after the
introduction of Business Model Canvas (BMC) by Osterwalder and Pigneur (2010). The
canvas approach is also an essential element in the service design and co-creation toolbox
and a great variety of canvases for different purposes have been suggested. Theoretically
BMC is grounded on a system-level holistic view on the business logic of an economic entity
(Zolnowski and Böhmann, 2014; Zott et al., 2011). Other rivalling business model canvas
approaches include e.g. (1) Lean canvas (Maurya, 2012), (2) multiple embodiments of
“Service business model canvas” (Zolnowski and Böhmann, 2014; Daxboeck, 2013) and (3)

also “Service logic business model canvas” (Ojasalo and Ojasalo, 2015) which has similar
theoretical foundation as the Service business model canvas.

As a result, all the canvases include partially common and non-common elements (see
Appendix 1) while the viewpoints and backgrounds questions for each item varies from focal
company, partner to customers perceptive and in some cases even to customer’s

Living Lab Business Models

In general, most of the LL studies focusing on business modelling have more or less been
grounded on single or combination of only few case studies. This is typical approach when a
particular research stream is still evolving strongly. Schaffers et al. (2007) identified
preconditions and critical aspects for rural LL business model and concluded that LL business
model includes “various dimensions of partnership creation and operation across the different
Living Labs development stages”. This finding highlights the importance of understanding the
maturity of LL when defining a business model. Rits et al. (2015) explored the benefits of
integrating business model research within LL project and argued that these two
methodologies are complementary and should be utilized in conjunction. Mastelic et al. (2015)
investigated ENoLL’s new member evaluation process and what kind of selection criteria
measures are included during this process. As a result, they suggested that the following
business model elements are missing from current ENoLL evaluation criteria process: 1)
identification of the cost structure, 2) customer segments and 3) the revenue stream. To
conclude, the various business model canvases can be promising tools to empirically
evaluate the existing business models among the Health and Wellbeing LLs, which are
currently more or less uncharted. As a result, the second aim of this study is to test BMC
usefulness to compare LLs business models.

3 Research Methodology

Sample Selection

The unit of analysis of this study is a LL which is thematically focusing on the health and
wellbeing topics. The data for this study is grounded on the self-evaluation of the 16 LL which
are taking a part to the Interreg Baltic Sea Region funded ProVaHealth project. Interreg Baltic
Sea Region funding supports integrated territorial development and cooperation for a more
innovative, better accessible and sustainable Baltic Sea region. The main objective of the
ProVaHealth project is to develop sustainable business models and a transnational Living
Lab concept for the participating LLs. Country wise ProVaHealth includes 1 Living Lab from
Estonia, 1 from Latvia, 1 from Lithuania, 4 from Finland, 1 from Sweden, 5 from Denmark, 1

from Germany and 2 from Poland. The consortium members cover a large variety of health-
related fields such as: active healthy aging, homecare, telemedicine, diagnostics,
biomedicine, cardiology, oncology, acute care, health tourism, physical rehabilitation,
neurorehabilitation, osteoarthritis, public diseases and robotics. Only one ProvaHealth
consortium member (Laurea University of Applied Science) was an ENoLL member at the
time of the study and thus the only officially recognized LL within the sample selection.
Therefore, the estimation of the Living Lab establishment year was somewhat problematic,
since the LL operations have been integrated as part of the hosting organization activities.
On the basis of the responses from eight ProvaHealth consortium members they had started
their LL activities as follows: 2006 =1, 2009 = 1, 2012 = 2, 2013= 1, 2014=1, 2015=1 and
2018=1. For more details about the participating LLs and hosting organizations see Scanbalt
website (

Data Collection Process

First, the participating LLs and their hosting organisations websites were analysed in order to
find basic information about their profile and LL activities. Second, the key informants of the
LLs were contacted in face-to-face workshop during the project consortium meeting and the
data collection process was explained for them. After the consortium meeting the written
guidelines and the following four Business Model Canvases were send by email to the
consortium member contact persons: 1) Business Model Canvas by Osterwalder and Pigneur
(2010), 2) Lean canvas (Maurya, 2012), 3) Service business model canvas (Zolnowski and
Böhmann, 2014) and 4) Service logic business model canvas (Ojasalo and Ojasalo, 2015).
The selected canvases were including both common and non-common items as defined in
the Appendix 1. Using multiple canvases enabled theory triangulation (Smith, 1975:273;
Denzin 1978, p. 291) which can provide more insight into topic and as well as a better
reliability and validity for the results. However, due space limitations this study is reporting
only the data generated by the original Business Model Canvas elements, thus omitting the
non-matching elements. Furthermore, those four LL which had not filled the original Business
Model Canvas (BMC), the missing information was collected from the other canvases. Data
collection took place during March – April 2018 and in the Table 1, number of responses for
each canvas are summarized.

Table 1. Comparison of responses between four Business Model Canvas (N = 16)

Business Model Canvas type Number of responses

Business Model Canvas (Osterwalder and Pigneur 2010) 12 (75%)

Lean canvas (Maurya, 2012) 16 (100%)
Service business model canvas (Zolnowski and Böhmann, 13 (81%)

Service logic business model canvas (Ojasalo and Ojasalo, 13 (81%)

Reliability Analysis of the Canvas Data

After receiving the filled canvases, the key informants from each LL were contacted by email
to evaluate the reliability of the responses. In all only 10 responses were received for the
following questions: (1) Who filled these canvases, (2) How much time did it take to fill out the
canvases, (3) If it was difficult to fill these, what would be the reason for that, in your opinion,
and (4) Any suggestions of how to develop these canvases to help you out in developing your
work / business in the future? The reliability analysis is summarized as follows:

Who filled these canvases? Eight out of ten responding LLs were fulfilling the canvases as
a team work. The team composition varied from two persons, four persons or multiple persons
in which the number of team members were not explicitly defined. One LL also indicated that
an external validation process was taken place in which canvas results were discussed with
steering committee and main stakeholders. As a result, can be argued that the responses
should be reliable since in most cases multiple persons have been involved in the filling
process. Team approach is also typically suggested within Business Model Canvas tutorials
and guidelines. The other person who fulfilled canvases alone, indicated strong personal
involvement in their LL as well as having a business background. Thus these responses could
also considered reliable.

How much time did it take to fill out the canvases? Among the respondents, the process
of filling the canvases were ranging from 1.5 hours to about 2-2.5 days. The LL reporting only
1.5 hours workload was clearly the lowest. Three LL spent about half a day and the five
remaining LL which had indicated response time were spending more than one day. When
including also the number of persons who participated in the data collection process, the
amount of resources to give reliable responses is considered sufficient in most cases. The
allocated time is also somewhat in-line with typically suggested within Business Model
Canvas tutorials and guidelines.

If it was difficult to fill these, what would be the reason for that, in your opinion? The
responses relating how easy the canvases were to fulfil were clearly mixed. Partially this could
be explained if LL had or not had the previous canvas filling experience or a person having
business consulting experience (or similar). If the canvases had been used in the team before
or person(s) had business consulting experience, filling the canvases were somewhat straight
forward process. In contrast, without prior experience seemed to require more efforts.
However, the prior experience was not the only nominator when considering the effortless of
filling process. Few respondents also argued difficulties were derived by the fact that their LL
was at early development stage or the LL was a part of larger organization and therefore not

considered as a own business unit, which have a specific business model. As a result, these
respondents did not have a clear vision what is their business model, which naturally makes
canvas filling difficult. There are also indications that some of the canvas elements and the
questions derived from these elements were hard to interpret. Therefore the help from
business consulting (or similar) expert who would thoroughly understands the canvas models,
would valuable help in the filling process. Also Service Logic Business Model Canvas by
Ojasalo and Ojasalo (2015) was considered by one LL more as B2C rather than B2B tool.

Any suggestions of how to develop these canvases to help you out in developing your
work / business in the future? It appears, according to the few respondents, the current
canvas tools are not optimized for LL business development. Developing tailor made
“Business Model Canvas” tool for LL needs was e.g. suggested as one option or at least
having a more specific guidelines for filling process (note from authors: few reference pointers
were given in the filling guidelines, but not step-by-step instructions). Using Excel sheet as
data collection tools also gained critics by one LL. One of the LLs was also currently using
Balance Score Card approach to define their further strategy and argued that Business Model
Canvas was therefore not perfectly fitting to their needs. Using only one simple canvas tools
as a starting point was suggested by one LL as a good starting point to define business model.
Based on the above feedback, evidently there is a need to develop better and simpler tools
for LL business modelling development. However, the composition of respondents, and the
amount of the allocated resources could be considered as a sufficient for our research
purposes. Therefore, the data is a robust enough to preliminary compare the LL business

Data Analysis and Attribute Clustering Process

After receiving the four canvases from LLs, the data analysis process was conducted as

First, the “master list” of the individual attributes, included within each of the nine BMC
elements, was constructed by the second author of this study on the basis of the set of
examples given by the first author. At this stage, the master list included also all the different
spelling variation and the amount of hits each individual attribute had generated.

Second, in order to make sure that no coding errors were made, the first author of this study
evaluated the master list attributes by comparing the attribute list to the original canvas data.
Afterwards the two authors discussed about the few coding differences and agreed about the
final attribute names to be included into on the master attribute list.

Third, in-depth analysis, which was done collaboratively by the two authors, revealed that in
the master list´s great variety of wordings had been used to describe somewhat similar

attributes. Therefore, on the basis of the several iterations between the two authors, the
thematically similar attributes were combined into the single attribute and a simplified name
with additional descriptions the descriptions were given. The combined attributes to be used
on comparison are presented in Appendix 2 while in the Table 2 attribute reduction rates are

Table 2. Attribute reduction rate

Number of Number of new

BMC Combined /
original combined
element Original
attributes attributes

1. Key partners 32 8 25,00 %

2. Key activities 34 7 20,59 %
3. Key resources 29 9 31,03 %
4. Cost structure 20 8 40,00 %
5. Value proposition 42 12 28,57 %
6. Customer relationships 13 9 69,23 %
7. Channels 25 13 52,00 %
8. Customer segments 26 12 46,15 %
9. Revenue streams 20 9 45,00 %
Total 241 87 36,10 %

The raw data included 241 different attribute spellings which after recoding and combination
were reduced to 87 (i.e. 36% from original). The key activities (21 % from original size) and
key partners (25% from original size) were compressed most, whereas customer relationship
decreased the least (69% from original size). However, it must be remarked that some of the
attribute descriptions were relatively short, and therefore somewhat blurry to make fully
reliable conclusions what is the attribute actually describing.

Furthermore, in the case of few LLs, it appeared that some of the key attributes were omitted
from a certain business model canvas element, even if the same attribute was highlighted in
some other canvases. This could partially be explained by the fact that the available space
within canvas is limited and therefore only the most important attributes were presented by
the LLs. This observation supports the need for (at least partially) quantitative business model
evaluation tool which is tailor made for LL business model evaluation. Furthermore only 12
LL filled the business model canvas. Thus, only these LL´s canvases will be used for business
model comparison.

Finally, on the basis of the above classification process binary data ATTRIBUTE x LL matrix
in which individual attributes were represented as rows and LL were presented as columns
was constructed in collaboration by the two authors.

4 Results

Evaluating Business Model Description Accuracy

To evaluate the business model description precision, the total number of identified attributes
per each LL was compared (Figure 1).

Figure 1. The number of attributes distribution on each LL BMC

On the average 26.7 attributes were used to describe the LL business model, while standard
deviation was 7.1. The minimum attribute number was 18 and the maximum value was 46.
The maximum value was also identified as an outlier. This could partially be explained by the
fact that this particular LL was responsible for the data collection and leading the ProVaHealth
project’s development of sustainable business model work package. Therefore, this particular
LL could have done more thorough analysis process than the others in order to anticipate the
follow up tasks. Importantly, even this LLs attribute descriptions covered only 52.9 percent of
all attributes. Therefore, it is more than likely that also this LL could have omitted some of the
key attributes during their analysis process.
On the basis of these observations, it is argued that the business model evaluation tool for LL
should not only evaluate the existence of the given attribute, but estimate also the perceived
importance of the different attributes. This kind of valued tool could more easily distinguish
the key attributes for different living labs.

Business Model Comparison Testing

In Figure 2 the 12 LLs business models are visualized as a two-mode network dataset in
which the connections are representing LL’s linkage to individual business model canvas
attributes. Since the attribute labels becomes unreadable, they are omitted from the
illustration and only the LL labels are presented.

Figure 2. Living lab business model comparison visualization

As shown in the image, some business model differences can be observed among the 12
LLs. Since the interpretation is difficult without showing the attribute labels, as an example
the connections within KEY PARTNER attributes and LLs are presented in Figure 3.

Figure 3. KEY PARTNER attribute connections

For example, the visual observation suggests that the partnership with state level actors might
be one of the distinguishing factors between the different LLs. Only LL6, LL14 and LL15 have
indicated state level actors as their partners, whereas all the other LL are not. Therefore, in
order to evaluate what kind of clusters the LLs are forming on the basis of their partnerships,
the following convergent correlations approach was applied. The matrix correlations for LL X
KEYPARTNER matrix was calculated several times in succession in which the resulting
correlation matrix was used as a starting point in the next round. The goal of this convergent

process was to identify groups of LLs which share a correlation in some of their attributes. As
a result, LL4, LL6, LL7, LL10 and LL11 formed one cluster and the remaining LLs the other
cluster (i.e. LL12, LL13, LL14, LL15, LL16, LL2 and LL9).

The similar matrix correlation analysis was done for key partner attributes in order to reveal
which attributes could be clustered. As a result, HOSPITALS AND HEALTH SERVICE
ACTORS formed the second cluster. Notably the largest relative difference was the
second cluster all LL had relationship with those partners.

Similar analysis process could be conducted also to other business model canvas elements.
However, due space limitation and the robust analysis presented in section 4.1 these
analyses are omitted from this paper.

5 Discussion – BMC tool usefulness to identify and compare the LL

Business Models

The LL Business Model Comparison testing with LL X KEYPARTNER matrix resulted two LL
clusters and two attribute cluster, which verifies that different business models are existing
among the LL at least when it comes to key partner selection. Therefore, it is argued that the
BMC tool can be used as tool to compare the LL business models if the list of attributes within
each BMC is valid and covering the essential attributes.

However, it appeared that using the Business Model Canvas (BMC) tool to describe LL
business model was resulting high variety of attribute descriptions among the investigated
Living Labs. This is not a big surprise since the BMC tool can be regarded as a standardized
open-ended interview approach. In this kind of research approach, respondents are asked
identical questions, which helps to reduce the researcher biases and enable respondents to
describe their responses in as much detail as they desire. As a result, the coding of the open-
ended responses into unambiguous variables can be difficult task, as described in the 3.3
Data Analysis and Attribute Clustering Process section.

Even if the streamlining and clustering of attributes reduced the number of different attributes
from 241 to 87, some doubts remained relating attribute unambiguity. The number different
attributes used by an individual LL to describe their business model, varied from 18 to 46. At
the lower end of the scale this means that LL was on the average using only two attributes
per BMC element to describe their business model. On the contrary, the LL having maximum
number of attributes was identified as an outlier although it covered barely over half of the all
identified attributes. Thus, the business model canvas tool usability and usefulness as a

robust comparison tool could be questioned if the BMC is used as an open-ended tool.

As a result, it is concluded that reliable business model comparison within LL context should
include a common set of attributes in which the given attribute importance to a specific LL is
evaluated based on valued Likert scale. Even if this limits the number of options within each
BMC element, it should result unambiguous description of the LL business model if the
attribute set is including all the relevant items. Since the current set of attributes described in
Appendix 2 was only validated by the authors of this study, further refinement and validation
round within the ProvaHealth consortium members and/or other LL communities is needed.

6 Conclusions

The business models among the Living Labs, and especially in the case of health and
wellbeing thematic area, have been uncharted. By applying industry standard Business Model
Canvas (BMC) approach, the business models among 12 Baltic Sea region Health and
Wellbeing Living Labs were described and compared in order to test the usefulness of the
BMC tool. Grounded on the list of provisional BMC attributes, the BMC tool was found a
suitable tool to identify the differences between the LL business models. However, developing
a reliable business model comparison tool would require further research efforts due following

Limitations of the study. First, the ProvaHealth consortium includes only one ENoLL member
and thus is the only officially recognized LL within the sample selection. Even if LL operations
can be executed by non-ENoLL members, the follow-up sample selections should include a
larger set of ENoLL members, which have been officially verified as LLs. Second, all
ProvaHealth consortium members were operating in the health and wellbeing thematic area,
which business model can differ from the other thematic LLs. This thematic bias limits the
generalizability of the results and therefore the sample selection should include also non-
health and wellbeing LLs. Third, few LL in the sample selection had just started their LL
activities and evidently their business model was still somewhat vague. As a result, their
business model description could reflect more on the desired business model rather than
actually implemented business model. However, on the other hand, the sample selection
which includes different maturity level LLs could also increase the reliability and reveal how
the business models between experienced and newcomer LLs are differing. Finally, by the
definition a business model describes how an organization uses its resources to offer their
customers value and make profit by doing so. In this study the LL business performance was
not investigated and therefore conclusion cannot be made what kind of impact business
model have to business performance.

Directions for the future studies. First, collaboratively refine the suggested attribute list among
ProvaHealth consortium members, but rather with ENoLL community, which includes more
experienced and diverse LLs than ProvaHealth consortium. Second, once the agreement of

the attributes has been achieved, a Likert-scale based survey questionnaire should be
formulated, which evaluates the perceived importance of the given attributes within each of
the nine BMC elements. Third, after receiving the perceived importance data from each LL,
interviews with selected LL should be conducted to investigate the story behind each attribute
value selection and make adjustments to the valued scale if needed. The interview process
will help to triangulate the valued data selections and provides more in-depth understanding
how the LLs is operating. Finally, the data collection process should be scaled up with the all
ENoLL members in order to include also non-health and wellbeing LLs as well as more
experienced LLs.

References and Notes

Afuah, A., & Tucci, C. L. (2001). Internet business models and strategies (p. 358). New
York: McGraw-Hill.
Chesbrough, H. (2010). Open services innovation: Rethinking your business to grow and
compete in a new era. John Wiley & Sons.
Daxboeck, B. (2013). Value co-creation as precondition for the development of a service
business model canvas. Studia Universitatis Babes-Bolyai, Negotia, 58(4).
Denzin, N. K. (1978). Triangulation: A case for methodological evaluation and combination.
Sociological methods, 339-357.
Garcia Robles, A., Hirvikoski, T., Schuurman, D., & Stokes, L. (2016). Introducing enoll and
its living lab community. European Network of Living Labs (ENoLL).
Mastelic, J., Sahakian, M., & Bonazzi, R. (2015). How to keep a living lab alive?. info, 17(4),
Maurya, A. (2012). Running lean: iterate from plan A to a plan that works. O'Reilly Media,
Ojasalo, K., & Ojasalo, J. (2015). Adapting business model thinking to service logic: an
empirical study on developing a service design tool. THE NORDIC SCHOOL, 309.
Osterwalder, A., & Pigneur, Y. (2010). Business model generation: a handbook for
visionaries, game changers, and challengers. John Wiley & Sons.
Osterwalder, A. (2004). The business model ontology: A proposition in a design science
Rits, O., Schuurman, D., & Ballon, P. (2015). Exploring the Benefits of Integrating Business
Model Research within Living Lab Projects. Technology Innovation Management
Review, 5 (12): 19-27.
Schaffers, H., Cordoba, M. G., Hongisto, P., Kallai, T., Merz, C., & Van Rensburg, J. (2007,
June). Exploring business models for open innovation in rural living labs. In
Technology Management Conference (ICE), 2007 IEEE International (pp. 1-8). IEEE.
Smith, H. W. (1975). Strategies of social research: The methodological imagination.
Prentice Hall.
Zolnowski, A., & Böhmann, T. (2014a). Formative evaluation of business model
representations - The service business model canvas.
Zolnowski, A., Weiß, C., & Bohmann, T. (2014b). Representing Service Business Models
with the Service Business Model Canvas--The Case of a Mobile Payment Service in
the Retail Industry. In system sciences (HICSS), 2014 47th Hawaii International
Conference on (pp. 718-727). IEEE.
Zott, C., Amit, R., & Massa, L. (2011). The business model: recent developments and future
research. Journal of management, 37(4), 1019-1042.

APPENDIX 1: Business Model Canvas element comparison and added attributes

Business Service Logic

Canvas Lean Service Business Model
model Business
elements Canvas Canvas
Canvas Model Canvas

1. Key X X, but partner(s) also X, but also

partners analysed via same 7 from customer
attributes as the focal viewpoint
2. Key X X, but also from customer(s)
activities and partner(s) point of view
3. Key X Also from customer(s) and X, but also
resources partner(s) point of view from customer
4. Value X X, with Also from customer(s) and X, but also
proposition unique partner(s) point of view from customer
notatio viewpoint
5. Customer X named customer(s) and
relationships partner(s) point of view
6. Channels X Also from customer(s) and
partner(s) point of view
7. Customer X X Customer(s) analysed via
segments same 7 attributes as the
focal company
8. Cost X X Also from customer(s) and X, but also
structure partner(s) point of view from customer
9. Revenue X X Also from customer(s) and X, but also
streams partner(s) point of view from customer

Attributes beyond original Business Model Canvas: (1) Early adopters as sub customer segment, (2) Problem,
(3) Existing alternatives, (4) Key metrics, (5) Unfair advantage, (6) Solution (note: might be also considered as
activities but using a different name), (7) Mobilizing resources and partner, (8) Interaction and co-production
and (9) Customer’s world and desire for ideal value.

APPENDIX 2: Living Lab Business Model Canvas combined attributes

Key partners

1. RESEARCH (e.g. research institutions, universities, national research centres and

institutes, research councils, researchers, experts).
2. HOSPITALS AND HEALTH SERVICE PROVIDERS (e.g. public, regional district and
private hospitals, primary care units)
3. MANUFACTURERS e.g. companies/Industry partners, International mHealth/IT
industry, long term company relationship, SMEs, medical devices and equipment
providers, industry experts (groups)
4. END-USER e.g. 3rd sector organizations, NGOs, Senior associations, End-Users
5. EDUCATION e.g. educational institutions, universities, students, teachers
6. REGIONS AND MUNICIPALITIES (e.g. Regional support and administrative
departments, council, government, development, municipals, Public organizations
7. NETWORKS AND CLUSTERS (e.g. Company cluster organizations, Company
networks, international partners and networks, cluster members, ecosystem partners,
Life science cluster, accelerator, life science innovation, affiliated LLs)
8. STATE ACTORS: (e.g. State budgetary unit, Health data authority)

Key activities

1. R&D SERVICE e.g. projects, test medical equipment or orthosis, Medtech solutions,
Externally funded R&D projects, User-center workshops, Project management, Clinical
trials, Provide consultations, Guidance to develop Medtech solutions, Need and market
analysis, Customer journey
2. EDUCATION AND TRAINING (e.g. basic, in-service training, simulated learning
environments, expert lectures, , Educational training for SMEs, Consulting
stakeholders (for LL methods)
3. NETWORK MANAGEMENT (e.g. Management of stakeholder and customer
networks, Single point entry, Connecting partners, develop relations, Networking and
networking meetings, Open access to infrastructure,
4. FUNDING SUPPORT e.g. Acquire project funding
5. REGIONAL SUPPORT e.g. Regional innovation governance and support system,
Regional virtual app centre, Public-Private-Partnership, Support to political committees
6. MARKETING AND SALES e.g. Marketing and sales support, Raising awareness and
knowledge in healthcare data, writing articles, Outside events, Technology library,
7. END-USER SERVICES e.g. services for end-users, personal wellbeing data

Key resources

1. PERSONNEL e.g. personnel, staff, human resources, researchers, cross disciplinary

teams, Arena management team, Project management
2. INFRASTRUCTURE AND TECHNOLOGIES e.g. facilities, infrastructure, premises,

wellcome center tools, regional campuses, Simlab services, Virtual App Centre, Single
Point Entry, technology library, local LLs, access to health care data infrastructure, IT
infrastructure, technologies
3. EXTERNAL EXPERTS e.g. consultants, health care professionals, high level medical
personnel, steering group for innovations
4. MANUFACTURERS e.g. Equipment vendors / manufacturers
5. STUDENTS e.g.-bachelor, master level
6. DATA AND SCIENTIFIC DATABASES e.g. Databank (wellbeing from clients, internally
collected data), access to scientific publication databases
7. IPR e.g. Intellectual property, know-how, TTO specializes in building value from new
technologies and Intellectual Property Rights (IPR).
8. EXTERNAL NETWORKS e.g. memberships in (established) international network,
university networks
9. END-USER (PATIENTS) with wide range with different diagnosis

Value proposition

1. R&D SERVICES e.g. research services (developmental), to test equipment and tool in
real environment, research and development of medical robotics, research with R&D
and regional development, refinement in various product dev phases
2. R&D WITH END-USER e.g. app testing on patients before launching on IT-platform,
Everyday teamwork with patients and families, Fast access to agile piloting with users,
Organizing test groups for companies and their products, User and customer feedback,
3. POSITIVE ARGUMENTS: fast development, Cost effective LL development, high
quality research, long-term and wide-range experience, reliable partnership for
services, Custom-made assistance for collaborative projects, adjustments of test apps,
sustainable concept or solution tailored to custom needs,
5. UNIQUE INFRASTRUCTURE e.g. unique testbed for devices and healthcare data
solutions, Unique test setup for technical and user elements, Facilities and technology
available for partners, Access to novel equipment and research services, Access to
public facilities and resources
6. FUNDING e.g. access to grants (that need public partner), full or partial funding of
innovation project, Feedback on project applications
7. VALUE AND IMPACT EVALUATION e.g. Healthcare economics, Determination of
potential value of cooperation, Sparring and analytical support
8. MULTI-DISCIPLINARITY e.g. Inter-professional testing at different development
phase, ensuring multidisciplinary development, Testing according to international
protocols and validation
9. ECOSYSTEM AND PROJECT MANAGEMENT e.g. single point entry by customers to
access partners, Orchesterating innovation ecosystem, Partner identification, One

point for all questions, Project management
10. EDUCATION AND TRAINING e.g. updating professional competence, distribute latest
info about the technologies, safe simulated learning environment (medical), Seamless
integration with education and LL/SD activities
11. METHOD DEVELOPMENT e.g. new scientific discoveries to improve LL / CC / SD
methods and pedagogy
12. MARKETING SUPPORT e.g. develop visibility of medical services for medical

Customer relationships

1. LONG-TERM RELATIONS e.g. existing long-term contacts and relations, Partner for
manufacturers, Permanent and non-permanent agreements with internship providers,
institutional relationships
2. NETWORKING e.g. networking / collaborating with other innovation and/or research
3. EVENTS e.g. National and International events
4. DIRECT e.g. email, phone, face-to-face, skype
5. PROJECT BASED e.g. project and need based collaboration
6. ADVISORY e.g. business advisory with potential customers
7. INTERNAL e.g. Internal business supporting projects
8. STEERING e.g. Ministry of Educations
9. CO-CREATION e.g. co-creation and community


1. ONLINE e.g. internet, social media, Youtube

2. DIRECT e.g. direct marketing, direct contacts, personal contacts, word of mouth
3. EVENT PARTICIPATION e.g. participating conferences and events (fairs e.g.),
4. EVENT HOSTING e.g. hosting conferences and events, Annual events for selected
stakeholders, workshops, customer journey meetings
5. MEDIA PROMOTION e.g. newsletters, media, printed media, PR
6. NETWORKS e.g. national and international networks, Scanbalt promotion list, ENoLL
7. REGIONAL LEVEL CHANNEL e.g. Regional inter- and intranet
8. CO-OPERATION PROJECTS e.g. participating as a partner in projects, co-operation
partners, other innovation actors
9. SCIENTIFIC AND LAYMAN PUBLICATIONS e.g. articles, scientific articles
10. EDUCATIONAL CHANNELS e.g. degree programs and individual courses, training
courses in simulation, Internship as a part of studies
11. LOBBYING AND POLICY CHANNELS e.g. public sector policy and strategy papers
and recommendations, advisory meetings, Information meetings with hospitals and

12. NATIONAL LEVEL CHANNEL Single point of entry
13. OWNERS e.g. Owner stakeholders

Customer segments

1. HOSPITALS e.g. Hospitals (regional)

2. CARE PROVIDERS e.g. care providers, nursing homes, Housing companies (assisted
3. WELLBEING SERVICE PROVIDERS e.g. Wellbeing service providers (gyms,
wellbeing centers)
4. LAYMAN END-USERS: e.g. Individual clients, service users, end-users, Common
people (elderly), Foreign medical tourists
5. PRIVATE COMPANIES e.g. startups and SMEs, Developers and producers of medical
equipment, medical devices, medical supply stores, distributors, Medtech (with focus
on ehealth), healthcare solutions
6. RESEARCH ORGANIZATION e.g. Public(research) organizations (THL , Danish
Health data authority), scientific units
7. EDUCATIONAL INSTITUTES e.g. students, teachers, researchers, staff, schools,
universities, technical universities
8. HEALTCARE PROFESSIONALS e.g. medical professionals, healthcare professionals
9. REGIONS AND MUNICIPALITIES e.g. regional support and administrative units,
Cities, region and municipalities, policy makers, public sector actors
10. VARIOUS INDUSTRIES e.g. healthcare Biotech, Pharma, Acro biotech, chemical
industry, food industry, All healthcare related areas where competences
11. NGOs e.g. 3rd sector actors
12. NETWORK AND ECOSYSTEM PARTNERS e.g. ecosystem partners, International
partners and network

Cost structure

1. PERSONNEL e.g. personnel costs, human resources, administrative costs, internship

fees, PERSONNEL e.g. personnel cots, human resources, administrative costs,
internship fees,
2. EXTERNAL EXPERTS e.g. external consultants (incl legal)
3. INFRASTRUCTURE AND FACILITIES e.g. facilities, technical environments (rent),
equipment, amortisation of equipment, (software) licences, , Outsource expences,
depreciation of the truck, utilities costs, common costs, memberships, distribution and
hosting costs fees in networks
5. MARKETING AND SALES e.g. Marketing costs, materials and consumables,
Customer acquisition costs (webpage expences), engaging users, own share of

external funded projects, conference and event participations
6. VARIABLE COSTS e.g. arranging LL activities, reagents
7. IPR e.g. Patents and IPR protection

Revenue stream

1. GRANTS e.g. Grants from national and international partnership and projects
2. SERVICE AND PROJECT CONTRACTS e.g. Contracts and Invoicing from the
services, Royalties, consulting, membership fees, organizing test groups, annual fees,
events, certifications, workshops
3. BASIC FUNDING e.g. regional fixed grants, or basic funding, internal funding
4. SITE VISITS e.g. Visit to the facilities
5. RETAIL e.g. equipment dealer
6. RENTAL e.g. Rental living lab, truck, equipment
7. DONATIONS e.g. Individual donations
9. ROYALTIES e.g. Royalties from IP properties or elsewhere

Developing a mutualized R&D for network organizations
based on Living Lab methods: the transformation of Sion
military airport into a commercial airport
Emmanuel Fragnière1, Benjamin Nanchen1, Marine Héritier1, Randolf
Ramseyer1, Magali Dubosson1 and Joelle Mastelic1
1 Entrepreneurship and Management Institute University of Applied
Science Western Switzerland

Category: Research Paper

When faced with a radical disruption, a firm must be innovative regarding its new services in
order to survive. In the particular case of industrial firms, this goes traditionally through R&D
departments. In the field of services, it is more complicated because the production and the
consumption happens simultaneously and always involves the consumer as a co-creator of
value (Vargo & Lusch, 2004). In the context of this research project, we aim to develop a
newly conceptualized framework for adapting the R&D function to the service sector based
on Living Lab methods. The R&D function could be externalized since usually the customer
journey is based on an unordered series of touchpoints involving different service providers
with the active participation of customers. A living lab is often presented as being at the
junction of open and user innovation. We thus propose a new model where the living lab is
integrated into the formal framework of the “à la Frascatti” R&D process in order to break the
linearity of the latter by including co-creation. This model allows the living lab to be better
accepted in typical “industrial mentalities” and, due to the formalism of the R&D process, to
rely rather on systematic and rigorous academic disciplines (for exploration, experimentation
and evaluation) than on less formal practitioner approaches. The case study of the renewal
of Sion Airport will be presented in this paper to test its feasibility. There is a major disruption
in its service offer, transforming it from a military to a touristic airport. So, all kinds of services
have to be developed from scratch. For instance, the Lab organized last year the mandatory
crash plane simulation on the tarmac, required by the Federal Office of Civil Aviation (FOCA).
In this paper, we present an applied research project, which is about the creation of a
reception area more accessible, welcoming and functional for crews and their passengers of
business jets.

Keywords: Living Labs, Service Design, Airport Services, Tacit Knowledge

1 Context

"Aviation - Sion airport shacked the statistics on Saturday. 647 passengers transited by the
capital. Sion airport feels like it's growing wings. On Saturday, the infrastructure beat his
record with 263 plane movements. Last year's record was 185 movements. Of this total, 63
movements are business jets. If we add the test link with London, the Sion airport saw 647
passengers transiting during the day of Saturday." (, February 12, 2017).

Inaugurated in 1935, Sion airport is ideally located in the heart of the Swiss Alps and close by
the largest tourist resorts in Valais such as Crans-Montana, Verbier, Zermatt and Saas Fee.
Until 2017, Sion airport was essentially a military airport. Since the army decided to leave Sion
airport and to concentrate the aviation sector in another region, the airport will transform
radically its offer toward the tourism sector. It will propose more scheduled air services,
business and commercial flights. Today, Sion airport faces thus a major touristic challenge.
Indeed, tourism in Wallis needs its infrastructure in order to evolve and meet growing demand
of travelers. The current infrastructure does not respond to this evolution. The number of
passengers and flight frequencies are also no longer the same as when it was first operated
nearly a hundred years ago. The airport needs R&D activities in order to provide innovation
in its services and adapt to this disruption in its business model.

This case study research is particular in the sense that the aim of the living lab here is to
"revive" an airport originally intended for military purposes, which must be transformed as
quickly as possible into a profitable commercial airport. The living lab by its logic of co-creation
and co-production constitutes a process of innovation, which is completely adapted to this
type of challenge. The living lab is here directly embedded in the transforming ecosystem
(from a military to a commercial airport). Usually living labs are designed as a catalyst for a
region (i.e. macro level). In this case the living lab (meso level) is integrated in a formal R&D
framework to gain immediate acceptance from all the stakeholders. The purpose of the living
lab in this particular context is to create the types of services needed at a commercial airport.
All these services already exist at other airports. Consequently, the objective is therefore
through co-creation to adopt a rapid transformation of the airport. We generally tend to apply
field studies as well as quasi-experiments to enhance the design of “tacit knowledge” of airport
operations in the context of digitalization. To do so we rely on approaches coming from social
sciences such as survey research, conjoint analysis, ethnomethodology, human ethology,
experimental phenomenology, visualization techniques, social experiments … We describe
in this paper one of the activities (micro level) undertaken by the living lab and which concerns
the design of the passenger reception area.

The well-known Frascati Manual was the first attempt (its first edition was published in 1920)
by OCDE to formalize the research and development function in manufacturing. R&D is not
fundamental research, nor even applied research. It is defined as “experimental
development”. It is a formal process, which accumulates specific knowledge that can be
transformed into commercial applications. R&D “à la Frascati” has been adopted by every

industry in the world. At the time of its origin, it was essentially a matter of sovereign economic
power. Over the years, the R&D process has become standardized, based on a linear
sequence (exploration, design, development, testing and scaling-up). It is typically conceived
for make-to-stock logistics. From Chesbrough and the open innovation paradigm, the barriers
of the firms have opened to integrate external knowledge and ideas (2004). Today, most of
the innovative initiatives are intended for the service sector. A service represents an intangible
production and is typically not adapted to the strict linear R&D process. Vargo & Lusch go one
step further with its service dominant logic paradigm and affirm that every industry is a service
industry. The product is only a vehicle to transfer the value to the customer. The customer is
always a co-creator of value as no value is created if the product is not consumed
(2004).There is thus an urgent need to revisit the Frascati Manual to devise an effective R&D
function along with relevant performance metrics specific to the service sector to facilitate the
dissemination of innovations. In this paper, we develop a new kind of R&D for Sion airport
based on Living Lab methods. Indeed, Sion airport represents a reunion of multiples SMEs
and public entities in one location. It has thus to work in a collaborative manner, integrating
all co-design the stakeholders in the of new innovative services.

The main applied research question addressed in this paper is "Sion Airport: How to make the
reception area for crews and their families more accessible, welcoming and functional?" The
reception hall of Sion airport is no longer up to date. Since its creation, no renovation work
has been carried out.

The methodology explains somehow the common thread. The pilot project has been
preceded by qualitative interviews with all the stakeholders who frequent the premises of the
airport, including employees of the companies. An immersion program took also place to
observe the actual services. Then, different Service Design tools were applied in order to co-
develop the service: writing a screenplay, creating blueprints and sharing a theatrical
experience. This pilot project is finally presented in 3D, with the Kozikaza program. This
project corresponds to the discovery of the necessities indispensable to the development of
the commercial infrastructure at Sion airport. It focuses on the travel experience and therefore,
only on the airport lobby, including the information desk.

This paper is organized as follows. In Section 2, we present a literature review related to the
notion of R&D for services. In Section 3, we describe the concept of the R&D for services
based on Living Lab methods that has been implemented. In Section 4, we focus on one of
the pilot projects that we realized and that is related to the new airport lobby. In Section 5, we
conclude and provide directions for further research.

2 Literature review

The Taylor model, first instituted in industry, calls for standardizing and simplifying tasks as
well as emphasizes repetition. Service enterprises have tended to base their organizational

models on models from the manufacturing and industrial sectors that incorporate hierarchy,
task repetition, and the standardization of procedures. By extension, service companies have
relied on traditional service design models that consist of sequential multiphase approaches,
beginning with idea generation and proceeding stepwise to concept development, business
analysis, prototype testing, market testing, and commercialization (Stuart and Tax, 2004). In
turn, organizations have sought to simplify work-related issues in order to maintain continuity
and solve problems by training employees to rehearse and practice routines (March, 1991).
Those routines constitute a source of not only efficiency, but also stability and predictability
within organizations, since employees can cope with new situations by applying routines with
which they are already familiar (Starbuck, 1993).

Today, however, such models present clear limitations. Above all, the Taylor organizational
model, which originated in the industrial world, is no longer relevant in today’s service and
knowledge-based economy. In its volatile environments that seek increased heterogeneity, it
becomes increasingly difficult to develop, disseminate, and apply routines. Standardization
that once achieved success might be inappropriate at another time and therefore cause

We have long known that a firm has to be innovative in order to survive in such volatile
environments (Drucker, 1954), in which innovation a new product development are critical for
any firm’s survival and growth (Penrose, 2009). According to Teece et al. (1997), it is not only
a unique set of resources that matters but an organization’s ability to constantly adapt,
reshape, reconfigure, and innovate. To obtain unique resources (i.e., dynamic capabilities),
organizations should be based upon highly firm-specific processes and shaped by specific
asset positions (Gawer and Cusumano, 2002). To put a name to such resources, Den Hertog
et al. (2010) have defined service innovation capabilities as dynamic capabilities of
transferring and imitating.

Knowledge is a dynamic capability since “learning allows tasks to be performed more

effectively and efficiently, often as an outcome of experimentation, and permits reflections on
failure and success” (Ambrosini et al., 2009). According to Drucker (2009), knowledge has
become both the key economic resource and the dominant source of competitive advantage.
Considered an important firm resource that is unique, inimitable, and valuable (Grant, 1997;
Wernerfelt, 1984), knowledge exerts a positive influence on the probability of innovating
(Gopalakrishnan et al., 1999). A knowledge-based service refers to a service delivered by
highly trained providers that offer high-quality services designed to meet customers’ needs
(Debély et al., 2006). In knowledge-based services, intellectual capital embedded in people
and systems is critical (Oliveira et al., 2003). That evolution implies a different organizational
model that relies more on creativity and implicit knowledge, which is the essence of expertise.

In our project, we seek to demonstrate how expertise and creativity have become essential
to today’s value-added services. Service offerings need to encompass the following elements
by relying on the provider’s knowledge, not a Taylor model-inspired process:

• A clear, thorough understanding of the needs and expectations of clients;
• The ability to elaborate a diagnosis of clients’ needs from limited information;
• The outline of a specific service proposal, co-created with the stakeholders;
• The efficient use of delivery processes and existing products or product modules;
• A custom-made solution that incorporates perceived benefits, often referred to as
problem resolutions; and trust among the stakeholders, including the customers.

In each step of implementing that process, expertise should be transparent. Evidently,

however, the functional flow chart of the Taylor model was not created around such reasoning.

The underlying principle of the knowledge-based view is that the foundation of an

organization’s performance is its ability to generate, combine, recombine, and exploit
knowledge (Kogut and Zander, 1996). Knowledge, especially tacit knowledge, is scarce and
difficult to transfer and imitate (Winter, 1987). Because tacit knowledge is embedded partly in
the psyche and intuition of individuals (Brown and Duguid, 1998; Grant, 1996), it is not readily
articulated and resists codification (Baumard, 1999). Although tacit knowledge is also difficult
to identify, interpret, formalize, and transfer, all of which processes can be expensive, the high
value of tacit knowledge motivates organizations to attempt to capture it (Spiegler, 2000).
After all, the challenge of readily codifying tacit knowledge affords it the potential to create a
sustainable advantage for individuals and organizations that possess superior knowledge
bundles (Lubit, 2001). At the same time, in a problem-solving context, applying explicit
knowledge without also applying tacit knowledge can result in suboptimal solutions. Since
tacit knowledge is a latent asset unless constantly and systematically enacted, organizations
have to invest continually in updating their skills and experiences (Daft and Weick, 1984;
Orlikowski, 2002).

Bartol and Srivastava (2002) have defined knowledge sharing as an action in which members
of an organization disseminate relevant information to other people in the organization. Tacit
knowledge, held by individuals or the group, is learned by observation, imitative application,
participation in routines, and personal experience (McIver et al., 2012). If tacit knowledge is
subconsciously understood and applied, then it is usually shared via highly interactive
activities (Zack, 1999) and created during collaboration (Hardy et al., 2003; Lee and Choi,
2003). Generally, the transmission of knowledge occurs implicitly, in formal meetings and on-
the-job training, meaning that social factors, particularly trust, are crucial in the transfer of tacit
knowledge (Collins and Hitt, 2006). As a first step, direct interaction facilitates the
identification of people with the appropriate knowledge (Lee and Cole, 2003).

The continuous sharing of knowledge contributes to innovations in teams, units, and entire
organizations. To better execute innovative tasks, members of organizations always have to
search for explicit knowledge already existing in the organization or borrow from the tacit
knowledge of colleagues. That latter process is crucial to developing a network of face-to-
face interactions. Indeed, agents of R&D in the service sector are often responsible for that
function and perhaps even part of that function in different organizations (Holste and Fields,

2010; Lin, 2007). Accordingly, it is necessary to foster a willingness to share knowledge,
especially tacit knowledge, that can be acquired only via training and rehearsal. In that sense,
a knowledge-sharing approach has to be implemented throughout the organization; it needs
to be part of the organization’s culture. Only then are members of organizations encouraged to
learn and share ways of executing tasks, to be committed to problem solving, and to take
initiative (Hu et al., 2009). The pivotal role of direct interaction among the actors toward value
co- creation is described in details by Grönroos & Voima (2012).

3 A model for mutualized R&D for services

The classic process of manufacturing innovation follows three steps in a linear manner:
research, R&D and finally manufacturing. In the field of Service Science the raw material of
service production is considered to be knowledge (explicit and implicit) and the customer is
also a co-producer. These two special features of service production make it impossible for
the innovation process to be strictly linear, as in manufacturing. Typically, a service is
performed in any order. As such, the logic of service innovation must consider that the
associated process cannot be linear, as it must include reciprocating patterns of interaction
among the participating stakeholders. R&D for production is based on the "make-to-stock"
principle. The goods are produced in batches, then stored and finally sold on the markets. On
the other hand, in a service environment, for instance university students will develop specific
skills by following a pedagogical script that includes regular interactions with professors,
assistants and researchers.

If we consider now the iterative "Living Lab" process with its four components: 1. co-creation,
2. exploration, 3. experimentation and 4. evaluation. This process differs from the R&D
process only in the co-creation component, which breaks the linearity of the process and
involves every stakeholder taking part in the innovation. Indeed, the living lab is often
presented as an innovation process at the junction of open innovation and user innovation.
The Living Lab represents then a “co-creation enabler” between user and open innovation. By
analogy, we could say that exploration, experimentation and evaluation, even if they are tools
that are service-oriented, have the same similar roles and purposes as the identification of
needs, prototyping and testing in the classical R&D manufacturing process. We propose a
mod el that combines the Living Lab process with the classical R&D process (see Figure 1).

Figure 1. Integrating living lab methods in the more formal R&D process

Both processes (i.e. R&D and LL) contribute to the overall innovation framework. The living
laboratory process integrates the necessary co-innovation components within service
operations. R&D enjoys very high acceptance among the stakeholders in the airport
ecosystem since its tools are grounded on engineering and scientific methods. Exploration,
experiment and evaluation could also be based on scientific methods, typically field studies
and laboratory testing in the context of services such as cultural anthropology, ethnography,
quasi-experiment … The living lab would then be more accepted by all stakeholders,
especially in an ecosystem like an airport where innovation is driven by security, safety and
compliance. In this context, a rigorous and systematic (i.e. scientific) process is of great
importance (e.g. mandatory crash test of an airplane that is required, monitored and evaluated
by the aviation authorities in order for the airport to obtain an operating license).

To understand how the R&D process is synonym to innovation in industry, we propose a

detailed account of the Frascati Manual. In the Frascati Manual (2015, p. 28), R&D is defined
as “creative and systematic work undertaken in order to increase the stock of knowledge –
including knowledge of humankind, culture and society – and to devise new applications of
available knowledge.” A R&D activity is characterized by five criteria: novel, creative,
uncertain, systematic, transferable and/or reproducible. The goal of R&D is to create new
knowledge. R&D is performed systematically, in “a planned way, with records kept of both the
process followed and the outcome”. (Frascati Manual, p. 46, 47). According to SNA (System
of National Accounts), in the Frascati Manual (2015, p. 28), “product refers to a good or a
service and process refers to the transformation of inputs to outputs and to their delivery or to
organizational structures or practices.” In the new edition of the Frascati Manual, a greater
emphasis is given to the R&D in the social sciences, humanities and the arts (2015, p. 44).
R&D in service is difficult to define. R&D project are rarely “specific to a service” and there is

no clear distinction “between R&D and other innovation activities”. Furthermore, in service
companies, R&D is not formally organized as in other industries (Miles 2007, Sundbo 1997).
Some indicators like collaboration with public research laboratories, staff with doctoral
degrees or doctoral students and the publication of research findings in scientific journals help
to identify R&D in service. (2015, p. 68, 69).

For Sawatani and Fujigaki (2015, p. 166-168), these characteristics are not enough to
describe R&D in service. They propose a Service R&D Model based on the S-D logic and
extended the Moeller’s model where “service processes are divided into facilities,
transformation and usage”. The new model has “three spheres, such as R&D, value co-
creation and site”. In this model, researchers benefit of ideas generated through the co-
creation interaction. The Living Lab approach, “an Open Innovation ecosystem”, have also
the goal to involve user in the R&D process as a co-creation value (Pallot, et al. 2011).
However, as Ballon et al. (2005) show in their framework, Living Labs are little concerned with
in-house R&D.

R&D for service needs a clearer definition and more formalized process to be identified easily.
In this context, design could play an interesting role. However, “design and R&D activities are
difficult to separate” (Frascati, p. 63, 64). Sanders and Stappers (2008) proposed a design
research landscape with two dimensions: the design research approach and the mindset.
Based on this work, Pallot et al. (2008) propose a landscape of Living Lab research with two
dimensions, the interaction mode and research type, to show “a progress form functional tests
and usability analysis toward user-co-creation” and proposed two complementary dimensions
to “better characterize the current R&D and innovation trends and evolution”. Another
dimension could show the shift from technological to sociological innovation (Pallot, 2010).
However, these landscapes do not fully define R&D for service.

Because Design Thinking process is not performed systematically, it means “in a planned
way, with records kept of both the process followed and the outcome” (Frascati Manual, p.
46, 47), it cannot be considered as R&D. The proposed methodology for service design in
four steps as the advantage to be more systematic and, so, can increase the knowledge and
be transferable and/or reproducible.
After the explanation of the overall concept of R&D for services based on living lab methods
(see Figure 1), we show practically how we implement the R&D part (exploration,
experimentation and evaluation).

The exploration phase, based on proven qualitative research methodologies, allows the
descriptive and analytical study of the field, the service and the problem (Crewsell, 2013).
Approaches such as ethnomethodology, ethology or phenomenology are carried out through
semi-directive interviews, focus group interviews, immersions, social experiments,
benchmarks and actor maps. Innovation is the capacity to generate novel and useful ideas,
and to transfer it to market. It is a critical component of successful design in today’s business
economy. Chan et al. (2011) argued that innovation can be best achieved in the ideation

phase, where concepts and ideas are generated either intuitively or through systematic
processes. Based on collected data, this analysis and design phase is based on tools from
service marketing, such as Customer Journeys, Service Blueprints, scriptwriting, user stories.
It allows the best representation of the utility of the service and its functionalities to generate
the tangibilisation of the service experience. Once the service has been designed and
scripted, it can be prototyped and staged using tools such as theatre or mock-ups. A prototype
is a visualization of the service containing an understanding of what the solution should be
and what it should do for users (Blomkvist, 2014). Initial user feedback can be collected to
improve the service prototype and to make the invisible visible). The employed methodology
at this level is based on quasi-experiment, where participants are not randomly assigned to a
case (Shadish et al., 2002), after which the results of the two groups are compared (Campbell
& Stanley, 2015). Knemeyer and Naylor (2011) have identified the necessary conditions for
quasi-experiments to establish the causality of two tested variables; first, with all else equal,
only the independent variable should be changed, and second, the independent variable has
to affect the participants, and the dependent variable has to intervene. The quasi-experiment
phase is also staged such as in a theater re-enactment in order to facilitate the scaling up of
the tested service (Fragnière et al., 2012). To allow clients to judge the quality of the service
and decide to market it, a 1:1 scale test is organized. This phase measures its quality through
feedback from users and potential auditors, evaluates its price and the willingness to pay
(Homburg, 2005). This is a scientific, evidence-based, approach (Carr et al. 2011). This
evaluation process is similar to a patient visit to a doctor in 3 steps (Simon, 1990):
Examination, including anamnesis (analysis of the initial situation and stipulation of functions)
consists in gathering and understanding information in order to identify the problem (internal
data as well as tacit knowledge). Each element is measured so that the main problem can then
be calculated and defined in the problem statement. Diagnosis (inventory and mapping,
problem modelling) is the problem-solving phase. The problem is structured by performing a
contradiction and error source analysis using the most effective problem-solving techniques.
The prescription corresponds to the solution evaluation phase. The ideas generated are
evaluated based on the evaluation of the solution. By giving priority to ideas and formulating
local constraints, the ideal solution will then be formulated. This final result consists of a list
of possible innovative conceptual solutions. If solutions are still not found after the elimination
of contradictions, or another new problem occurs after the evaluation of the solution, the
problem-solving process is reinstated in the first step to redefine the original situation.

After this phase services can be implemented for real and this corresponds to the service
production (see Figure 1).

4 An example of service innovation based on LL integrated in the

more formal R&D process

This case study is focused on the meso level proposed by Schuurman: “Living Lab Innovation

project” as it is intended to generate a managerial impact for the airport. The novelty is also
to integrate service design tools at the micro level, these tools have been tested in another
LL research (Papilloud & al., 2017) but are not often mobilized in Living Lab environments.

We have worked on a dozen of service design projects to enable the transformation of Sion
airport. In this paper, we concentrate on how to make the reception area more accessible,
welcoming and functional for crews and their passengers of business jets.
To illustrate that process, the case of regular Swiss flights that will land and depart from Sion
Airport offers a fitting example. Since the infrastructure currently in place cannot
accommodate such a large number of passengers, new operating modes offering a series of
services (e.g. security checks and dining facilities) that can accommodate more passengers
need to be designed, tested, and implemented. All these new operating modes must also be
orchestrated into a consistent experience.

The R&D for Sion airport is first a physical location. The lab is a large renovated theatre fully
IT equipped used to simulate services experiences for design and innovation. In our R&D
activities, we are especially interested in the way service companies are able to scale up their
new services once created. In the classical setting of a manufacturing company, we all know
that the innovation process is realized through the R&D function. In the case of services,
however, the approach is very different since a service is an intangible production, does not
follow a strict linear process, and usually involves different providers. As ultimately the service
will be experienced by customers, the innovation process resembles more closely practicing
music scales, training to play a sport, rehearsing a scene, or working out a dance routine. In a
service experience like going to the airport to take a plane, the routines are called “operating
modes” and are orchestrated in such a way that the passenger undergoes without any
difficulties a very complex process (baggage handling, security check, duty free shopping,
boarding). In these ways, the R&D function in the service sector certainly corresponds more
to a kind of general staging of both providers and clients. Consequently, in order to provide
services of good quality, the rehearsal of new services is critical to being able to innovate.

We have thus developed a conceptual framework grounded on theory building to enable the
acquisition and coordination of the necessary know-how (i.e., tacit knowledge) to foster service
innovation. Thanks to a Memorandum of Understanding with Sion airport, we are able to
conduct the conclusive experiments in real-life conditions. Thus, this research in general
typically corresponds to in-use research. Its originality lies in the fact that to our knowledge
no such conceptual framework for R&D exists today that has been tested in real conditions.

A Living Lab can be seen as a “living laboratory”, at the level of a region, in which users
participate in the development of innovative products and services (co-creation). Its main goal
is to explore the "insight,” that is, the salient features valued by a specific population, so that
new services emerge. It is also a test environment, open and benefiting from technological
and methodological tools. It is, therefore, an ecosystem allowing a participatory process,
using appropriate tools and methodologies (Liedtke et al., 2012). Service Design was not

integrated in the living lab research landscape (Pallot & al, 2011), it could play an important
role for innovation in the traditional service industry.

The methodology followed for the realizing this pilot project was to conduct, in a first stage, a
qualitative survey among passengers at Sion airport. To this end, 15 passengers were
interviewed on 18 February 2017 during their flights between the cities of Sion, London and
Zurich. Then a second qualitative study was carried out among Sion airport employees. In
March, 5 employees as well as the airport manager, were interviewed. A complementary
methodology used was immersion (participative and non- participative observation). Indeed,
the passenger experience from the entrance hall to boarding could be lived, in February,
during the actual flights. A transcript of these interviews was made in order to imagine a
scenario and to create an initial blueprint as well as an improved blueprint considering the
remarks and details received during the qualitative interviews and the immersion. Then based
on a "LivingLab" (co-creation) approach, we developed together an experiment design that
ultimately took place in our labs. Finally, in order to test the improved blueprint proposal, a
theatrical experiment was set up as well as a final 3D pilot project.

The objective of this work was to submit a pilot project for the implementation of new 3D
infrastructures for the passenger waiting area, in order to respond to this constant evolution
described above. The implementation recommendations are illustrated by a 3D pilot project
divided into several presentation themes (see Figure 2): the opening of a souvenir and coffee
shop, the creation of different spaces, the improvement of the partners' visibility, the
improvement of the information office - the C office and the addition of an interactive terminal.

Adding an interactive terminal

Airport work area for its partners

Passenger entrance and reception hall

Figure 2. Some photos illustrating the pilot project

5 Conclusion

In the knowledge economy, the experience becomes the central part of value co-creation and
“the market is becoming a forum for conversation and interactions between consumers,
consumer communities, and firms. It is this dialogue, access, transparency, and
understanding of risk benefits that is central to the next practice in value creation.” (Prahalad
& Ramaswamy, 2004). Thanks to our partnership with Sion Airport, we have the opportunity

to test Service Desing tools such as the blueprints in real Living Lab conditions, thereby
making the smooth transfer of findings into practice possible via practical operating modes.
Consequently, the results would be tangible and able to be disseminated in other contexts.
As mentioned, the business model for the tourism sector especially needs complete revision.
The conclusions we can draw from this living experience were not at all anticipated. The direct
integration of a living lab into the airport's R&D function implies that R&D is based on a
formally coherent scientific process (i.e. a systematic and rigorous approach) and accepted
in industry. As we have already mentioned in this article, the living lab is an interface between
open innovation and user innovation. What we have seen in this case study is that a living
lab, formally integrated into the R&D function, immediately gains credibility and legitimacy and
adds the missing component of traditional R&D, namely co-creation between the user and a
collective of service providers. Indeed, managing an airport requires the coordination of
various activities such as a tight schedule integrated into a very formal logistics driven by
goals of safety, security and compliance. User experience is certainly crucial, as for example
any response to an airport accident that involves the airport, the police, firefighters,
passengers, airlines and even the state, the municipality, journalists, social networks... to
achieve a complex problem solving to save people’s lives.

We thus propose a mixed innovation process where the living lab is integrated into the formal
framework of the R&D function. This mix of the two innovation processes offers the advantage
for R&D to integrate co-creation and the advantage for the living lab to be embedded in a
formalized process based on evidence-based approaches and which enjoys great credibility
and legitimacy in the industrial world.

In all, the project could represent an original, concrete initiative toward applying the logic of
R&D to services based on Living Lab method. Thanks to our partnership agreement with Sion
Airport, we have been able to develop expertise in service innovation dedicated to small
airports for tourism. In that sense, Sion Airport, previously a military airport that will essentially
become a commercial one, presents an interesting case for several reasons. Since all services
have to be created, tested, put into production, and priced, the airport will serve as a to-scale
laboratory that can enable us to develop and test a comprehensive method for service

What is missing from that cooperative symphony, however, even if the musical partition is
beautiful and the musicians talented, is a conductor who can make everyone—providers,
locals and tourists—play in unison. An airport is indeed a combination of multiples actors
(SMEs, public entities, passengers …). This is where Living Lab methods can play an
important role as a facilitator in the future of service innovation of typical network organizations
such as an airport. By applying the Living Lab precepts (co- creation approach), we develop
on a regular basis, involving all the stakeholders, experiments. A Living Lab can be seen as a
“living laboratory”, at the level of a region, in which users participate in the development of
innovative products and services (co-creation). Its main goal is to understand the "insight,”
that is, the specific needs of the customers, so that new services emerge. It is also a test

environment, open and benefiting from technological and methodological tools. It is, therefore,
an ecosystem allowing a participatory process using appropriate tools and methodologies.


Ambrosini, V., Bowman, C., Collier, N. (2009). Dynamic capabilities: an exploration of how
firms renew their resource base. British Journal of Management, 20, 9-24.
Ballon, P., Delaere, S., Pierson, J., Poel, M., Slot, M., Bierhof, J., Diocaretz M. (2005). Test
and Experimentation Platforms for Broadband Innovation: Conceptualizing and
International Best Practice. IBBT/VUB-SMIT Report.
Bartol, K.M., Srivastava, A. (2002). Encouraging knowledge sharing: The role of
organizational reward systems. Journal of Leadership & Organizational Studies, 9,
Baumard, P. (1999). Tacit knowledge in organizations. Sage, UK.
Blomkvist, J., 2014. Representing Future Situations of Service: Prototyping in Service
Design. Linköping University Electronic Press, Linköping.
Brown, J.S., Duguid, P. (1998). Organizing knowledge. California management review, 40,
Campbell, D. T., & Stanley, J. C. (2015). Experimental and quasi-experimental designs for
research. Ravenio Books.
Carr, V.L., Sangiorgi, D., Büscher, M., Junginger, S., Cooper, R. (2011). Integrating
evidence-based design and experience-based approaches in healthcare service
design. HERD: Health Environments Research & Design Journal 4, 12–33.
Chan, J., Fu, K., Schunn, C., Cagan, J., Wood, K., Kotovsky, K. (2011). On the benefits and
pitfalls of analogies for innovative design: Ideation performance based on analogical
distance, commonness, and modality of examples. Journal of mechanical design
133(8), 081004.
Chesbrough, H. (2004). Managing Open Innovation. Research-Technology Management,
47(1), 23-26.
Collins, J.D., Hitt, M.A. (2006). Leveraging tacit knowledge in alliances: The importance of
using relational capabilities to build and leverage relational capital. Journal of
Engineering and Technology Management 23, 147-167.
Creswell, J. (2012). Qualitative inquiry and research design: Choosing among five
approaches (3rd ed.). Thousand Oaks, CA: Sage. Paperback, 472 pp.
Daft, R.L., Weick, K.E. (1984). Toward a model of organizations as interpretation systems.
Academy of management review, 9, 284-295.
Debély, J., Dubosson, M., Fragnière, E. ( 2008). Pricing of the Knowledge-Based Services:
Insight from the Environmental Sciences. Journal of Services Research, Special
Issue, 167-181.
Den Hertog, P., Van der Aa, W., De Jong, M.W. (2010). Capabilities for managing service
innovation: towards a conceptual framework. Journal of service Management, 21,
Design council (2015) Design methods for developing service. Guide. UK Design Council.
Retrieved online on 08.07.2018
ds%20for%20d eveloping%20services.pdf.
Drucker, P.F. (2009). A century of social transformation: Emergence of knowledge society,
in: Managing in a Time of Great Change. 177-230, Harvard Business Press, NY.
Drucker, P.F. (1954). The practice of management. Harper & Row, New York.

Fragnière E., Nanchen B., Sitten M. (2012).Performing Service Design Experiments Using

Ethnomethodology and Theatre-Based Reenactment: A Swiss Ski Resort Case
Study. Service Science, 4, 1-13.
Gawer, A., Cusumano, M.A. (2002). Platform leadership: How Intel, Microsoft, and Cisco
drive industry innovation. Harvard Business School Press, Boston.
Gopalakrishnan, S., Bierly, P., Kessler, E.H. (1999). A reexamination of product and
process innovations using a knowledge-based view. The Journal of High Technology
Management Research 10, 147-166.
Grant, R.M. (1997). The knowledge-based view of the firm: implications for management
practice. Long range planning 30, 450-454.
Grant, R.M., (1996). Toward a knowledge-based theory of the firm. Strategic management
journal, 17, 109-122.
Grönroos, C., & Voima, P. (2013). Critical service logic: making sense of value creation and
co- creation. Journal of the Academy of Marketing Science, 41(2), 133-150.
Hardy, C., Phillips, N., Lawrence, T.B. (2003). Resources, knowledge and influence: The
organizational effects of interorganizational collaboration. Journal of management
studies, 40, 321- 347.
Holste, J.S., Fields, D. (2010). Trust and tacit knowledge sharing and use. Journal of
knowledge management, 14, 128-140.
Homburg, C., Koschate, N., Hoyer, W.D. (2005). Do Satisfied Customers Really Pay More?
A Study of the Relationship between Customer Satisfaction and Willingness to Pay.
Journal of Marketing 69, 84–96.
Hu, M.-L.M., Horng, J.-S., Sun, Y.-H.C. (2009). Hospitality teams: Knowledge sharing and
service innovation performance. Tourism management, 30, 41–50.
Kogut, B., Zander, U. (1996). What firms do? Coordination, identity, and learning.
Organization science, 7, 502-518.
Knemeyer, A. M., & Naylor, R. W. (2011). Using behavioral experiments to expand our
horizons and deepen our understanding of logistics and supply chain decision
making. Journal of Business Logistics, 32, 296-302.
Lee, G.K., Cole, R.E. (2003). From a firm-based to a community-based model of knowledge
creation: The case of the Linux kernel development. Organization science, 14, 633-
Lee, H., Choi, B. (2003). Knowledge management enablers, processes, and organizational
performance: An integrative view and empirical examination. Journal of management
information systems, 20, 179-228.
Liedtke, C., Welfens, M.J., Rohn, H., Nordmann, J. (2012). Living Lab: User-Driven
Innovation forSustainability. International Journal of Sustainability in Higher
Education, 132, 106-118.
Lin, C.-P. (2007). To share or not to share: Modeling tacit knowledge sharing, its mediators
and antecedents. Journal of business ethics, 70, 411-428.
Lubit, R. (2001). The keys to sustainable compe