Sie sind auf Seite 1von 5

692 PROCEEDINGS of the HUMAN FACTORS AND ERGONOMICS SOCIETY 45th ANNUAL MEETING- 2001

MEASURING ENGAGEMENT IN VIDEO GAMES: A QUESTIONNAIRE

Daniel K. Mayes and James E. Cotton


University of Central Florida

Video games continue to grown in popularity and presently account for annual revenues in the billions of
dollars. Although the technologies underlying a modern video game are well understood, the
characteristics of a game that make it a success are not. The Engagement Questionnaire (EQ) is introduced
in an attempt to capture those dimensions that are thought to influence the degree to which a user becomes
engaged while playing a video game. Our goal was to develop a metric that could be applied to a broad
range of video game genres (e.g., action, adventure, strategy, sports) and video game users (e.g.,
expert/novice, male/female, young/old). Factor analyses of data representing 243 participants suggested
the existence of five stable factors among 46 questionnaire items. These dimensions were labeled Interest,
Authenticity, Curiosity, Involvement and Fidelity. Though further empirical testing is necessary, we expect
that the questionnaire will prove to be a useful tool for the appraisal of video games, as well as for the
identification of critical differences between those who play them.

INTRODUCTION Usoh, & Steed, 1994). Unfortunately, presence metrics were


not developed with video games in mind. For example, the
Background Presence Questionnaire (Witmer & Singer, 1998), a
commonly used metric, was developed by Army research
Owing to vastly improved computer hardware and scientists for the purpose of measuring the quality of a virtual
software, modern video games are capable of delivering environment (VE) in military training scenarios. As a result,
highly realistic, compelling experiences to their players. As a the Presence Questionnaire focuses largely on how natural, or
result, game production has grown into a multi-billion dollar realistic the VE and VE-interface is. This approach, however,
industry - one that presently eclipses the motion-picture is not necessarily relevant for an entertainment simulation
industry. While consumer demand for video games remains such as a video game. For example, many game themes are
strong, an explanation for the demand is not so clear. What founded in fantasy, with a story occurring in a distant world
causes people to choose one game over another and what with weapons and creatures only known by science fiction.
elements of a game contribute to its appeal? Is it simply a Measurement for such attributes would not be found on the
function of the improved realism (e.g., graphics and Presence Questionnaire.
interactivity), or are there other, equally important variables?
While successful marketing sometimes predicts success, an Engagement
understanding of the nature of games, and the psychological
and behavioral qualities of those who choose to play them, Mayes, Cotton, and Sims (2000) have proposed a
may be even more relevant. For example, reliable data construct labeled "engagement" as appropriate for determining
suggesting relationships between player and game variables the quality of a video game experience. Engagement is
would be useful to the entertainment industry as a means to defined as how fun, involving, and motivating a task is. The
enhance success rates of new releases. In addition, for construct was conceived in consideration of the outcome that
military agencies that are presently considering the use of is desired most by game developers: an enjoyable game. It is
commercial video games to support their training agendas, hypothesized that enjoyable games lead to strong product
research relating game design to transfer-of-training and sales, recommendation of the game to others, and longer and
soldier performance could be equally valuable. Unfortunately, more frequent play by the end user.
the manner in which a game's entertainment value is currently The Engagement Questionnaire (EQ) introduced here
assessed is limited to items such as expert feedback, marketing was created in an attempt to capture those dimensions that are
surveys and game sales figures. What is lacking are thought to influence the degree to which a user becomes
systematic, empirical measures that would provide more direct engaged while playing a video game. The initial goal was to
and reliable data about a game's characteristics, develop a metric that could be applied to a broad range of
Instead of relying on intuition-based designs, what is video game genres (e.g., action, adventure, strategy, sports)
needed is an empirical method of predicting the success of a and video game users (e.g., expert/novice, male/female,
particular game design, across a range of unique users. In young/old).
order to make predictions, however, we must first be able to
measure the quality of a particular game along a variety of Constructs Targeted for the Questionnaire
dimensions. There are currently no published metrics to serve
this purpose. The nearest equivalents are questionnaires Upon its initial administration, the EQ questionnaire
developed to measure the construct of presence, or the consisted of 55 items that were derived from various sources.
subjective sense of being physically situated in the While the construct "engagement" incorporates many of the
environment specified by a simulation (Heeter, 1992; Slater, elements thought to contribute to presence (such as
PROCEEDINGS of the HUMAN FACTORS AND ERGONOMICS SOCIETY 45th ANNUAL MEETING- 2001 693

"authenticity"), it also considers those dimensions considered They responded to each EQ item on a 7-point Likert scale with
to be relevant to intrinsic motivation and reward. According labels for the extreme and mid points of the scale. In addition,
to Malone (1981), these include the dimensions of challenge, each participant also provided biographical data.
fantasy, and curiosity. These additional dimensions help to
address those content items (e.g., story, theme, objectives) that Procedure
are integral to most modem video games, yet are absent in
existingpresencemeasures. Participantscompletedthe materialnearthe endof a
The majority of questions relating to "authenticity" class period and returned the completed questionnaire
(e.g., level of interactivity, naturalness, ease of use, and immediately. Two EQs and a biographical data form were
consistency) were adapted from the Presence Questionnaire provided for each participant. The respondents were asked to
(Witmer & Singer, 1998). Other questions, including several recall both their favorite and least favorite video game and
relating to "involvement" and "fidelity" were also adapted then answer identical questions about each game.
from the Presence Questionnaire. The remaining questions Respondents were also instructed to provide the title of each
related to "curiosity," "fantasy," and "interest" were game. Separating the two copies of the EQ was a biographical
developed exclusively by the authors based on a review of the data form. Question and form order (either for favorite or
psychological literature, leading publications that track the least favorite game) were counterbalanced to control for order
latest video game design techniques, pilot data from focus effects. Participants were debriefed and received extra credit
groups, and personal experiences with playing video games. It upon return of the survey material.
is proposed that a relationship exists between and among these
dimensions, andengagement. RESULTS

METHOD Before factor analysis could proceed, a dependent


samples t-test was performed to determine which of the 55
Participants questionsdidnotdistinguishbetweenfavoriteandleast
favorite game. The results indicated that nine questions did
All participants were undergraduate students enrolled not meet the criterion alpha level of.001. These questions
in introductory psychology courses at a large, southeastern- were excluded from further analyses but are described in the
state university. Extra credit was exchanged for their discussion.
participation. Data from the 243 participants (94 males and Exploratory factor analysis using varimax rotation
149 females) were used for the analyses, was performed on the 46 remaining items using SPSS
Windows 10.0. Checks for outliers, multicollinearity, and
Measures factorability of the correlation matrices did not reveal
deviations from acceptable limits. Tabachnick and Fidell
All participants completed two versions of the EQ, (1996), among others, point out the benefits, limitations and
one each for their favorite and least favorite game. applications of the statistical techniques utilized in this study.

Table 1. Factor Ioadings for exploratory factor analysis using varimax


rotation
Item F1
a F2 F3 F4 F5
23 .541
28 .637
29 .666
30 .657
32 .623
33 .554
37 .577
38 .583
41 .679
46 .595
54 .523
694 PROCEEDINGS of the HUMAN FACTORS AND ERGONOMICS SOCIETY 45th ANNUAL MEETING- 2001

Table I (Continued). FactorIoadings for exploratoryfactor analysis using


varimax rotation
Item F1a F2 F3 F4 F5
1 .768
2 .747
3 .831
4 .607
7 .657
12 °563
14 .619
5O .609
10 .5O9
24 .673
25 ,587
34 .645
35 .679
36 .719
39 .644
40 ,699
15 .648
16 .603
17 ,703
18 .672
20 ,581
21 .556
27 -.662
55 .512
5 .719
6 ,776
8 .644
9 .674
10" .547
11 .626
19 .599
% of Variance 14,67 14.27 13,83 12.61 11.55

* Item appears in F3 and F5


a Suggested factor labels: F1 = Interest, F2 =Control, F3 = Curiosity, F4 = Involvement, and F5 - Fidelity

Six factors were extractedfollowinganalyses. After DISCUSSION


rotation, one factor consisting of two seemingly unrelated
questions was excluded from further investigation. Overall, The present factor-analytic approach appears to have
variables were well defined by this factor solution, provided the information necessary to initiate an empirical
Communality values tended to be at reasonable levels. A cut approach to video game engagement. Nine questions of the
of o50was used for inclusion of a variable in interpretation of original EQ did not discriminate between favorite and least
a factor. As such, three variables failed to load on any factor, favorite games and were not included as a part of the factor
and one variable loaded highly on two factors. A second analysis. The results of the factor analysis on the remaining
review of the factors using a cut of.40 resulted in multiple 46 items suggested the existence of five stable factors.
complex variables that were less defined by the factor Additionally, one factor was dropped because it was under-
solution. Because of this, the original cutoff of .50 was used identified and unstable across solutions. Below is a description
for interpretation. Loadings on factors and percent of variance of each of the factors as well as a brief account of the
accounted for are shown in Table 1o Loadings under .50 are questions that did not contribute to engagement.
not shown. Interpretive labels are suggested for each factor in
a foomote and are discussed further below.
PROCEEDINGS of the HUMAN FACTORS AND ERGONOMICS SOCIETY 45th ANNUAL MEETING- 2001 695

Factor 1. Labeled "Interest" Questions Not Included in the Factor Analysis

Items in this factor addressed subjective impressions As previously mentioned, nine questions failed to
of how interesting, appealing, and exciting the content of the reach the significance criterion of 12< .001 for the t-test
game was. Questions focused on the general theme, between liked and disliked games and were excluded from the
objective(s), characters, social structure, and objects in the factor analysis. The majority of these items included
computer-generated environment, questions regarding how difficult/challenging the game was
(e.g., "How would you rate the game's difficulty level?,"
Factor 2. Labeled "Authenticity" "Were there times that you felt overwhelmed by the amount of
information that you had to remember in order to successfully
The majority of items that loaded heavily on this complete some objective?").
factor had been derived and modified from the Presence Several other excluded items were related to what
Questionnaire (Witmer & Singer, 1998). The central theme of Witmer and Singer (1998) referred to as control factors (e.g.,
this factor included the degree to which events in the game "How much did the game's interface (i.e., the controls,
could be controlled, the responsiveness of the game to actions display, layout, etc.) interfere with or distract you from
that were initiated by the user, and the ease/naturalness with playing the game?, and "How much delay did you experience
which one could move or manipulate objects in the game. between your actions and expected outcomes?"). Finally, one
Usability of hardware interfaces, as well as the degree of question of a general nature did not satisfy the inclusion
interactivity built into the game would likely contribute to criterion: "How interested are you in video games?"
scores in this factor.
FUTURE RESEARCH
Factor 3. Labeled "Curiosity"
The purpose of the developing the Engagement
This factor included items about one's interest with Questionnaire was to create a tool that could be used to
exploration of the scene and its characters. These items quickly and inexpensively assess salient information about
seemed to cluster around the concept of curiosity, which can video game interactions. Five factors were identified that
be defined as the intrinsically motivating desire for could be labeled along the dimensions of Interest,
information (Lowenstein, 1994). Since previous research Authenticity, Curiosity, Involvement, and Fidelity.
indicated that humans seek out environmental variability, the Interestingly, "challenge" was not represented although we
desire for novel and changing stimuli in a video game had hypothesized that this construct would be particularly
interaction is not surprising (Jones, Wilkinson, & Braden, important. Though it may be reasonable to expect that these
1961). factorsaffectvideogameengagement, considerable empirical
research is necessary prior to drawing such a conclusion.
Factor 4. Labeled "Involvement" In addition to assessing existing and future video
games, the questionnaire can also be used to identify critical
This factor was comprised of items that asked differences between those who buy and play video games
directly, "Was the game fun?" and "How involved were you in frequently, and those with little video game interest and/or
the game?" Other items inquired about the degree of focus a experience. Finally, a number of gender issues can be studied
user experienced, as well as preferences about length of play. using the questionnaire, e.g., "What makes a game interesting
It is interesting that questions conceming focus and to females, and to males?" We have already begun this line of
involvement were grouped with others asking about how research, and will present those results at future meetings.
much fun and excitement was experienced.
REFERENCES
Factor 5. Labeled "Fidelity"
Heeter, C. (1992). Being There: The subjective
The fidelity factor consisted of questions regarding experience of presence. Presence, l(2), 262-271.
the graphical and auditory components of the game. The
realism, familiarity, and consistency of the graphical and Jones, A., Wilkinson, J.J., & Braden, I. (1961). Information
auditory components were important aspects of this factor, deprivation as a motivational variable. Journal of
Items included in this factor asked whether one could you Experimental Psychology, 62, 126-137.
easily identify sounds and graphics, as well as how consistent
were the game's images and sounds. Hardware issues such as Lowenstein, G. (1994). The psychology of curiosity: A
image resolution and sound quality, as well as review and reinterpretations. Psychological Bulletin,
design/modeling issues such as level of scene detail, are 116(1), 75-98.
expected to be determinants of a game's perceived fidelity.
Malone, T.W. (1981). Toward a theory of intrinsically
motivating instruction. Cognitive Science, 5(4),
333-369.
696 PROCEEDINGS of the HUMAN FACTORS AND ERGONOMICS SOCIETY 45th ANNUAL MEETING- 2001

Mayes, D., Cotton, J., & Sims, V. (2000). A conceptual model


of engagement in video games. Presented at the First
Annual Florida Human Factors and Ergonomics
Student Meeting, Orlando, FL.

Slater, M., Usoh, M., & Steed, A. (1994). Depth of presence


in virtual environments. Presence, 3(2), 130-144.

Tabachnick, B.G. & Fidell, L.S. (1996). Using multivariate


statistics (3ra ed.). New York: Harper Collins.

Witmer, B.G., & Singer, M.J. (1998). Measuring presence in


virtual environments: A presence questionnaire.
Presence, 7(3), 225-240.