Sarah Kounaves1, Louise Archer1, Heather King1 and Emma Pegram2

1 2
Natural History Museum, London, UK

Abstract. This study investigated how the use of mobile technologies in a natural history muse-
um might affect the science engagement experiences of adults. A mixed methods approach was
adopted over two phases. Phase 1 consisted of a questionnaire to adult visitors (n=60), seeking
to explore attitudes towards mobile technology use in museums, and understand usage habits.
The second phase consisted of an intervention. Adult visitors (n=20) were assigned to either a
s using their per-
sonal mobile devices. Using multiple analytical methods including a modified Science Engage-
ment Framework, preliminary findings suggest that contrary to findings in other studies, mobile
devices may not be facilitating higher levels of science engagement in this context, and may in-
stead be shaping visitor engagement experiences in other unforeseen ways. Given the current

to re-evaluate claims of
.

Keywords: mobile technologies, science engagement, adults, museum learning

1 Introduction
It has been argued that informal science learning settings such as museums have the potential to afford
unique opportunities for adults to engage with and learn science (Bell et al.2009; Stocklmayer et al. 2010;
Schwan et al. 2014). As personal mobile devices such as smartphones and tablets have become increasingly
ubiquitous in these informal learning settings, it has become all the more significant for us to understand the
ways in which these devices are interacting with the visitor learning experience. Although mobile learning
research has been conducted in a wide array of settings, experts in the field of mobile learning have increas-
ingly turned their attention towards places of informal science learning (e.g. museums, science centres, zoos,
aquariums). Yet when set within these contexts, mobile learning research has primarily focused on children
and students (Chen et al. i-
tionally, many of the mobile learning and informal science learning studies have brought up the term en-
gagement in conjunction with science learning, yet seldom investigated the concept of engagement itself,
with few exceptions (Rennie et al. 2003; Barriault and Pearson 2010).
Thus this research focused on investigating how the use of personal mobile technologies in a natural histo-
ry museum context might affect the science engagement experiences of adults. From this the following three
research questions were developed.
RQ1. ds and perceptions of mobile technology use in museums?
RQ2. How do adults naturalistically use mobile technologies in a museum context, and how does it compare to
how they use them in their daily lives?
RQ3. Does the prompted use of mobile technologies by adults in this context facilitate an increase in science
engagement?

Two phases of research were conducted using a mixed-methods approach, as part of a collaborative re-
Provisional
analysis of responses has shown that visitor dependence on and desire to use mobile devices decreases when
at a museum. Perhaps more significantly, however, preliminary findings have suggested that, contrary to

361
findings in other studies, mobile devices may not be facilitating higher levels of science engagement, and
instead may be shaping the overall visitor engagement experiences in other unforeseen ways.

2 Literature Review
What is Mobile Learning?
Mobile learning has been defined in a plethora of different ways since its inception. For example, some
studies have defined mobile learning in more techno-centric terms (Stevens and Kitchenham 2011), while
others have stressed the underlying learner experience (Sharples et al. 2007). Rather than defining mobile
learning, it is perhaps more useful to identify some shared characteristics of mobile learning from the litera-
ture, which impact on this study that it is mobile, personalised, and collaborative. Mobility refers to the

multiple contexts and on-the-go (Vavoula et al. 2009). Mobile learning is also personalised; learners are able
to use mobile technologies in ways that cater to their own individual experiences and history and that are
specific to the temporal and physical context of the learning moment (Kearney et al. 2012). Finally, mobile
learning has the ability to be collaborative (Clough et al. 2008). Whether connecting with other people in the
room or across the world, mobile devices have become increasingly capable of enabling collaboration. Re-
gardless of which conception of mobile learning they have used, however, studies have overwhelmingly
championed the use of mobile devices in enabling learning. For example, Hwang and Wu (2014) reported
that 83% of student-
learning achievements.

Engagement with Science in Informal Learning Settings

museum and informal science learning studies, research on the closely-related term engagement has been
much less prevalent. This study takes the position that understanding engagement is a fundamental precursor
to understanding learning. This has been echoed in the work of a small number of researchers who have ar-
gued for the significance of visitor engagement or highlighted a connection between learning outcomes and
engagement (Rennie et al. 2003; Rennie and Johnston 2004; Barriault and Pearson 2010). Most influential in
this study, has been the work of Barriault and Pearson (2010), who closely examined the meaning of en-
gagement by presenting a framework of seven observable visitor engagement behaviours in science centres,
grouped into three levels of engagement leading to learning. Modifications, however were necessary for this
project in order to address the particular context of the research a natural history museum instead of a sci-
ence centre (science centres have more of a focus on interactivity). Additionally, despite the framework be-
science within their framework.
Figure 1 presents the final modified version of the framework used to measure changes in the visitor en-
gagement experience.

Figure 7. Modified Science Engagement Framework

362
 3 Methods

Data Collection
The objective of this study was to investigate how the use of personal mobile technologies in a natural his-
tory museum context might affect the science engagement experiences of adults. Ethical approval was grant-
e (LRS-14/15-0766) prior to conducting the re-
search. Over the course of two phases a mixed-methods approach was used, specifically in order to attempt
to increase the validity and reliability of the data through methodological triangulation (although other at-
tempts to address validity and reliability were made as well). The research conducted in Phase 1 consisted of
a paper questionnaire administered at the Natural History Museum in London, aimed at answering RQ1 and
RQ2 while also helping to build an informed and effective Phase 2 intervention. The questionnaire was ini-
tially tested in a pilot phase (n=12). Pilot participants were briefly interviewed on their impressions and un-
derstandings of the questionnaire after filling it in. After revisions were made, the final questionnaire was
administered to adult visitors around the museum (n=60) May through June 2015. Visitors had to meet three
eligibility requirements, that they: were at least 18 years old, spoke relatively fluent English, and owned a
personal mobile device such as a smartphone or tablet.

l-

device to engage with an exhibit (such as taking a photo, making a note, looking up information, etc.), and
were then instructed to attempt to use their own mobile devices to engage in at least three different ways
within the exhibition. Twenty participants were recruited at the exhibition entrance, both pairs and individu-
als, February through April 2016. Participants had to meet the same three eligibility requirements as in Phase
1 (although visitors had to not only own a mobile device in Phase 2, but have it with them). Both groups
were observed and audio recorded while visiting the exhibition, and interviewed for 20-25 minutes after-
wards. The interviews sought to gain a better understanding of who the participants were, how they felt about
science and technology, and how they might normally use their own devices. The interviews also investigat-
ed engagement experiences (mobile and non-mobile) in greater depth. Using the visit field
notes and a series of exhibit photos path was re-traced through the exhibition. They were
asked questions about exhibits where at least level 2 or 3 engagement behaviours had been noted, such as
how they felt about the exhibits, why they stopped there, how they thought the exhibit related to science or
natural history, and if they had used their mobile devices why they chose to use their device at that mo-
ment, if they found it distracting, etc. Finally, participants were sent email questionnaires 3-5 months later to
determine if the participants recalled or had revisited any of their instances of engagement. To-date 8 follow-
up questionnaires have been received.

Data Analysis
Phase 1 data has begun to be analysed using basic statistical analysis, looking for overall trends in order to

include t-tests and factor analysis. The analysis of Phase 2 field notes, visit recording transcripts, interview
transcripts, and follow-up questionnaires has so far used two different approaches. First, they were each ana-
lysed using open coding analysis (using NVivo 11). This approach is somewhat related to grounded theory,
although it also acknowledges prior review of the literature and that research questions were in mind when
the analysis was carried out. Second, the interview transcripts, field notes, and visit recordings were analysed
using the Modified Science Engagement Framework, looking for evidence of the observable engagement

meta-

363
 4 Preliminary Findings

RQ1 and RQ2: Perceptions, Attitudes, and Usage Habits

The majority of participants in Phase 1 reported feeling dependent on their personal mobile devices in
general. However, when asked how dependent they felt while in a museum, the majority reported that they
did not feel dependent on them (see Table 1). Participants in Phase 1 were also asked to report on how often
they used their mobile devices in general and in the museum in eight different categories: social media, tak-
ing photos/videos, sharing photos/videos, communicating with others, looking up information, personal or-
ganisational purposes, entertainment, and learning (labelled a-h respectively in Figure 2). Although the most
popular method of mobile device use changed whether the participants were responding about general usage
or usage in the museum, in every category the overall reported usage decreased in the museum context.

Phase 1 and Phase 2 findings also illuminated the importance of photo-taking to participants. Nearly all of
the participants in both phases reported using their personal mobile devices to take photos to some extent. In
fact, in the majority of mobile engagement instances in Phase 2 (both prompted and unprompted) partici-
pants chose to involve photo-taking in some way (snapchat, Instagram, etc.) Finally, Phase 2 qualitative
analysis also yielded further insight into participant attitudes towards mobile devices in museums. The idea
that museums were not a place that participants felt they needed or wanted to use their devices was consist-

right? You

RQ3: Changes in Science Engagement

Although the Phase 2 data is still in the process of being analysed, from the data analysed so far using the
Modified Science Engagement Framework, there is a clear correlation between higher engagement levels
and the use of personal mobile devices to engage. Comparing all of the instances of mobile engagement, the
majority achieved either level 2 or 3 as the highest level of engagement. However, through further analysis,
it was found that in the majority of those high level mobile engagement instances, that highest level was
reached prior to the use of the mobile device, and no level increase was found during or after the usage,
thereby eliminating a causal effect: mobile devices generally did not cause visitors to more deeply engage
with the science content. Rarely was there any additional engagement with the science while or after using
the device, and if there was, it was not usually a change in level, but rather the appearance of additional be-
haviours from that level.
Additionally, during the in-person interviews participants were asked how they felt about the engagement
instances, and specifically if they felt the mobile device usage changed their experience or enhanced their
understanding of the science. In the majority of mobile engagement instances the participants reported that
they did not feel as if the devices made any difference to their scientific understanding of the exhibits. This
corresponded with the results of the Modified Science Engagement Framework analysis described above.
Interestingly, preliminary results from the 8 post-visit questionnaires received so far (3-5 months after-
wards), show that mobile devices might potentially be helping visitors to have more memorable experiences

364
overall. Those visitors who used their mobile devices to engage tended to be able to recall in greater detail
general things they may have felt or observed, their memories of their visit to the exhibition, and how the
exhibit related to their lives or other topics they were interested in. However, the responses did not seem to
suggest that visitors using mobile devices were any better at recalling the science (such as scientific facts or
concepts) they had come across and discussed during the interviews. In fact the majority of post-visit ques-
tionnaires did not include references to any science from the exhibition. Additional visitor post-visit ques-
tionnaire responses (still pending) will help to gain further insight into this preliminary finding.

5 Discussion and Conclusion

Given the preliminary findings above, did the use of personal mobile devices change the science engage-
ment experience of adults at the Natural History Museum? These results suggest that there is a correlation
between mobile device use at the exhibits and higher levels of science engagement, however, the use of mo-
bile devices did not facilitate an increase in the level of science engagement. One explanation for the correla-
tion might be that visitors chose to use their mobile devices in instances where they were already interested,
not instances where they felt they needed their device to learn. Perhaps visitors see their mobile devices as
tools for expression of interest rather than facilitation. If so, this might explain why the devices were used
predominantly durin
change during or after mobile device use. Additionally, from analysis of interviews, it appeared as if partici-
pants used their personal devices with three specific motivations: to capture, personalise/take ownership of,
and to share. Interestingly, these three motivations focussed predominantly on either the cultural, historical,
artistic, or personal connections with the exhibits, suggesting that while mobile devices did not facilitate in-
creased levels of engagement with scientific facts and concepts, perhaps they had a greater effect on more
general engagement. The initial results of the post-visit questionnaires discussed above may indeed help to
further support this notion.
In addition to completing analysis, there are still many issues that need to be further explored from the
perspective of both academia and museum practice. Is interest expression separate from engagement, or
could expression just be another aspect of engagement? Should the practitioners shaping the informal science
learning experience focus on helping people engage with science specifically, or, given this research, aim to
simply help visitors broaden their overall engagement experiences? Should museums look to provide visitors
with digital offerings that tap into the capturing, sharing, and personalisation motivations, or continue to fo-
cus on developing apps to try and facilitate science learning specifically (as opposed to general engage-
ment)? Given the
research highlights not only a need to potentially re-visit claims of mobile devices facilitating learning, but to
re-evaluate the ways in which mobile technologies might best be utilised to improve engagement in informal
science learning settings.

6 References
Barriault C and Pearson D (2010) Assessing Exhibits for Learning in Science Centers: A Practical Tool. Visitor Studies,
13, 90-106.
Bell P, Lewenstein B, Shouse AW, Feder, MA (2009) Learning Science in Informal Environments: People, Places, and
Pursuits, National Academies Press.
Bryman, A (2012) Social Research Methods, Oxford, Oxford University Press.
Chen YS, Kao TC, Sheu JP (2003) A mobile learning system for scaffolding bird watcching learning. Journal of
Computer Assisted Learning, 19, 347-359.
Clough G, Jones AC, McAndrew P, Scanlon E (2008) Informal learning with PDAs and smartphones. Journal of
Computer Assisted Learning, 24, 359-371.
Evans C (2008) The effectiveness of m-learning in the form of podcast revision lectures in higher education. Computers
& Education, 50, 491-498.
Hedberg JG (2014) Extending the pedagogy of mobility. Educational Media International, 51, 237-253.

365
Hwang GJ, Wu PH (2014) Applications, impacts and trends of mobile technology-enhanced learning: a review of
2008 2012 publications in selected SSCI journals. International Journal of Mobile Learning and Organisation, 8, 83-
95.
Kearney M, Schuck S, Burden K, Aubusson P (2012) Viewing mobile learning from a pedagogical perspective.
Research in Learning Technology, 20.
Rennie LJ, Feher E, Dierking LD, Falk JH (2003) Toward an agenda for advancing research on science learning in out-
of-school settings. Journal of Research in Science Teaching, 40, 112-120.
Rennie LJ and Johnston DJ (2004) The nature of learning and its implications for research on learning from museums.
Science Education, 88, S4-S16.
Schwan S, Grajal A, LeWalter D (2014) Understanding and Engagement in Places of Science Experience: Science
Museums, Science Centers, Zoos, and Aquariums. Educational Psychologist, 49, 70-85.
Sharples M, Taylor J, Vavoula G (2007) A Theory of Learning for the Mobile Age. In: Andrews R and
Haythornthwaite C (eds.) The Sage Handbook of Elearning Research. London: Sage.
Stevens D and Kitchenham A (2011) An Analysis of Mobile Learning in Education, Business, and Medicine. In:
Andrew K (ed.) Models for Interdisciplinary Mobile Learning: Delivering Information to Students. Hershey, PA, USA:
IGI Global.
Stocklmayer SM, Rennie LJ, Gilbert JK (2010) The roles of the formal and informal sectors in the provision of effective
science education. Studies in Science Education, 46, 1-44.
Vavoula G, Sharples M, Rudman P, Meek J, Lonsdale P (2009) Myartspace: Design and evaluation of support for
learning with multimedia phones between classrooms and museums. Computers & Education, 53, 286-299.

366