HCI and Game Design: From a Practitioner’s Point of View
ABSTRACT
The intent of this paper is to present an overview of the
many aspects of Human-Computer Interaction (HCI)
research and practices, show their relationships with
computer games, and discuss the possibility of using some
of the HCI processes, methodologies, and tools for game
design.
Keywords
Human-Computer Interaction, game design, usability,
gameplay, user interface
INTRODUCTION
Game designers face the challenge of creating games that
can be easily learned, effectively played, and emotionally
enjoyed by gamers. With very limited theoretical
foundation and little research or data on gamers, they have
to solely rely on their intuition and experience. This is why
about 80% of games fail on the market each year [1].
While technologies have improved rapidly, game design
has evolved slowly. No guidelines or principles have been
established to help game designers do their work.
On the other hand, HCI has been a thriving field in recent
years where a lot of innovative ideas have been generated.
These ideas lead to a variety of processes, methodologies,
techniques, and tools being developed and successfully
practiced in other software industries. They have helped
software developers solve many of their problems. Many
of these problems are closely related to the problems of
game design. Once they are learned and used by game
designers, these processes and methods can greatly improve
their work.
However, little is known about HCI in the game industry,
partly because of the prevailing culture of the game
industry that game designers dislike anything from
academia. They tend to think that researchers in the
universities know nothing about games so that what they
are doing or talking about does not matter. This is simply
wrong! Researchers may not know games as well as game
designers do, but they do know computer and behavioral
sciences well. If we agree that computer and behavioral
sciences form the very foundation of games, we should
admit that game designers can learn a lot from their
counterparts in academia.
Let us, then, look at some of the areas of HCI, examine
their relationships with the corresponding areas of game
design, and see how they two can work together.
USABILITY AND GAMEPLAY
“Usability” has been a buzzword among HCI practitioners
and enthusiasts in recent years. It is at the center of the
whole HCI phenomenon - many of the HCI processes and
methods have been developed around this concept. The
practice of these processes and methods has had a
tremendous impact on how users use software and how
developers develop software. However, it is not easy to
describe what usability really is. As Jakob Nielsen
described [2], usability is not a single, one-dimensional
property of a software product. It has multiple components
as figure 1 illustrates.
Usability
Memorability
Learnability
Satisfaction
Efficiency
Errors
Figure 1. The multiple components of usability
Game designers may impatiently point out that, in the game
industry, they have their own buzzword - gameplay, alias
“playability,” which was coined by game designers and
given extensive attention in the game industry almost 10
years before usability began gaining any exposure in other
software industries. Since the mid 80s - the golden age of
Nintendo’s 8-bit and 16-bit consoles – the enormous
importance of good gameplay has been recognized by game
designers, and thusly good gameplay has become the
ultimate goal pursued by anyone who wishes to remain in
the business of game design.
But after all these years of hard work, can game designers
give a clear description of what gameplay really is? Can
they show what specific efforts have been made to improve
the many aspects of it? No! Today “gameplay” still
remains as a very vague concept. It is commonly known in
software engineering that if you cannot define a property of
a software product or process, you cannot measure it. And
if you cannot measure something, you cannot improve
upon it. Thus, game designers have failed to develop any
useful methods or tools to evaluate and improve gameplay.
By contrast, HCI researchers and practitioners have taken a
systematic and quantitative approach to not only define
usability, but also develop formal processes and methods to
improve the many aspects of it, and create a large set of
tools and databases that can help designers do their work.
Game designers, then, need to first define what gameplay is
and what attributes it has, and second, find ways to measure
these attributes. Only after achieving these goals can they
develop processes and methods to improve these attributes.
In order to do this, a layered structure may help to represent traditional communication design or UI design. In fact,
gameplay as a complex property of games. As figure 2 most game designers tend to focus exclusively on game
shows, this structure treats usability as the foundation of mechanics, where they have most expertise, and ignore the
gameplay, since usability is a universal property of all UI altogether. Under these circumstances, it is not
kinds of software. The second layer includes game-specific surprising that bad UI designs are everywhere in the game
attributes that make a game distinctive from other industry today. Here are two examples of the UIs from
application software products. These attributes are genre- some popular games currently on the market. Both show
neutral. The third layer introduces the idea of game genre. that even the most successful games can have serious flaws
Depending on different genre categories, different in term of UI design.
attributes are introduced and studied. For example, unit
balance might play an important role in RTS (real-time
strategy) games, while level and magic systems are more
critical to RPG (role-playing game) games. The topmost
layer is a highly subjective layer that deals with the nuances
and uncertainties of gameplay, in short, this layer details
whether or not the gamer likes the game. It corresponds to
satisfaction in the definition of usability.

Emotion layer

Figure 3. An information panel of The Age of Empires, a

Genre layer
Genre-specific attributes highly successful RTS game from Ensemble Studios. It was
poorly designed without a sense of balance – icons and labels
Game-specific attributes are crowded over to one side.
Game layer
Usability attributes
Usability layer

Figure 2. A layered structure can better represent gameplay

This layered structure can help game designers better
understand the many aspects and different levels of
gameplay, so that they can direct their efforts more
effectively toward them. This structure also forms the
strategic framework that can be used by game companies to
optimize their design processes and organize their teams.
USER INTERFACE DESIGN
User interface (UI) plays an important role in games. We
can look at a game as a layered system with a core and a
shell. The core is the game mechanics, which determine
the rules of the game - how high the character can jump,
which directions he can go, how many enemies he will Figure 4. The server configuration screen of EverQuest, an
encounter at a specific level, etc. The shell is the UI, online fantasy/adventure game. Gamers have to choose a
because through it the gamer controls the character, server from the list, and click on the PLAY button, which is
receives directions and feedback, fights with enemies, and the huge button with an overlaying graphics on the extreme
has fun. The shell is what the gamer actually sees, and right, to play. Of the 20 players we tested, 16 of them could
significantly influences the player’s impression of the not recognize it and felt confused simply because the size of
game, no matter how well the core works. The shell hides the PLAY button is different from the three normal buttons to
the left. Also, the lack of a clear label combined with the
the core from the gamer, thus making games playable, as
misuse of a graphic and font makes it more like a logo than a
well as more fun and challenging – the gamer has to figure button.
out the internal mechanics while competing with it through
the course of playing. Most of the design flaws can be easily corrected with a
little help from the expertise of UI designers. The balance
However, the concept of UI design has been long neglected of the information panel in Figure 3 could be improved by
by the game industry. In most game companies, there is no using a grid system. The confusion caused by the PLAY
such position as a “user interface designer”. Most UIs are button in Figure 4 could be easily detected by a simple
conceptualized by game designers and created by graphics usability test conducted by UI designers before the
artists. Typically, neither the game designers nor the shipment of the game. Game design needs UI designers’
graphics artists have knowledge or skills in either participation. A few companies in the game industry are
moving in this direction. Larger companies such as
Ensemble studios and its parent company, Microsoft, have
hired HCI graduates to work as UI designers in their game
divisions. Other game companies need to catch up.
Ideally, each game project team should include at least one
UI designer to work with the game designers to help tackle
on the UI design issues. UI designers should use their
talent and knowledge to create better UIs, and most
importantly, stress the importance of UI through their work
and across the game industry.
Although bad designs prevail, a few good designs do stand
out. They may not all root from HCI research, but we can
clearly see some connections. Figure 5 and 6 show two
examples.
Figure 5. The elegantly designed in-game interface of
Neverwinter Nights, an upcoming RPG game from Black Isle.
The designers used the idea of pie menu proposed in some
research papers in the late 80s to achieve natural efficiency in
menu selection. The clean and highly transparent interface
does a great job of preserving a sense of immersion without
disturbing gamers’ attention from the “virtual” game world.
Figure 6. Game designer Peter Molyneux developed an
innovative and intuitive gesture recognition interface for
Black & White.
ITERATIVE DESIGN AND PROTOTYPING
Most game designers use an iterative approach to designing
games and are devoted to extensive testing, which they’ve
dubbed “playtesting,” to help them reach a solid game
design [3]. This sounds ideal to HCI practitioners.
However, game designers face two major problems. First,
they have difficulties fitting the iterative design approach
into the whole game development process. Most
companies use either a modified waterfall model or an
incremental model to develop their games. Game
development runs almost parallel to game design. Thus
there are always contradictions, conflicts, and
inconsistencies among programmers and designers. A
better synchronization scheme needs to be developed to
address this problem. Also there are debates on whether or
not game design should be completed before game
development starts. Second, it is very hard to make
prototypes for games. Games are extremely time-critical
and unlike applications software, require high fidelity
prototypes – a slow primitive prototype made using Visual
Basic or Director cannot faithfully represent the gameplay
of the real game and its usefulness is questionable at best.
Although a few successful efforts, such as the “Miyamoto
Box” [4], have been made for some specific genres, overall
prototypes haven’t gained much popularity among game
designers because they are too expensive to build and too
limited in their usefulness.
These two problems, especially the latter, have greatly
impaired the effectiveness of the iterative approach taken
by game designers. Without good prototypes, game
designs cannot be effectively evaluated and tested.
Realizing the problem, progress has been made by a few
3D software vendors to provide 3D prototyping tools to the
game design community. Compared to the high-end 3D
software used for animation movies and special effects,
these prototyping tools are simpler, much easier to use, and
yet they are powerful enough to create convincing game
prototypes. Most of them also include embedded scripting
languages that help game designers create more
complicated behaviors of the characters and game
mechanics.
GAMERS’ ROLE IN GAME DESIGN
Software development efforts can be classified into three
categories, or three generations, which are engineer-
centered approach, designer-centered approach, and user-
centered approach. Game design is still at the designer-
centered stage - ideas are generated by game designers,
evaluated and tested by game designers, then implemented
by programmers. Only after the game has been put into the
market will the game designers know whether or not
gamers will accept it. Many game designers are now
worried about this situation. Some have expressed the
belief that gamers should be more involved in game design,
which means moving game design from a designer-
centered approach to user–centered approach. Nintendo’s
Shigeru Miyamoto, arguably the most prominent (and
successful) game designer in the world, said in an interview:
“We design the games from the players’ point of view” [5].
Another famous game designer, Peter Molyneux, says that
he brings gamers to his company every week to test his
design ideas. By getting gamers actively involved at all
stages of game design, game designers will have a better
chance to understand gamers’ needs and create better
games.
Of course, game designers cannot just sit mindlessly next to
gamers and watch them play – that is not what the user-
centered approach is all about. Game designers need to
restructure their design processes, redefine their design
strategies, and reorganize their teams to reflect these new
ideas. This is where HCI can offer its help. There are two
basic questions that need to be answered in order to get real
benefits out of the user-centered approach. First, when
should gamers be involved in the game design process, and
second, how should gamers get involved? To answer the
first question, a structured collaboration framework is
needed, such as the user-centered design (UCD) process [6].
This would reveal when gamers could be most effectively
used within the design process. To deal with the second
question, game designers need trainings on how to observe
users, how to interpret their behaviors, how to record test
data, and how to get the most out of the test data. The
usability test methods, such as think-aloud protocols [7],
are particularly helpful in these situations.
USABILITY ENGINEERING METHODS
A major focus of HCI is in usability engineering methods
[8]. These methods can be analytical or empirical,
formative or summative. Their goal is not only to gauge
the success of software systems with respect to measurable
criteria, but most importantly, to guide redesign within an
iterative framework. A number of usability engineering
methods can be adapted for game design. Some work for a
specific genre, while others can be applied to a broad range
of problems and domains of game design.
Among the usability engineering methods, the so-called
“low-cost” inspection and walkthrough methods are
particularly useful [9]. These are relatively inexpensive
and not too intimidating to learn by game designers. For
example, cognitive walkthrough can be used to evaluate
level designs of action adventure games. Action adventure
games consist of a set of levels with various goals. The
player has to explore a level to achieve that goal, then
proceed to another level for another goal. This scenario
matches the idea behind the cognitive walkthrough very
well, the intent of which is to evaluate a design for its ease
of learning through exploration [10]. Heuristic evaluation
is a “discount” usability engineering method [11] that can
be used to solve genre-specific issues in game design.
Game designers working on a specific type or series of
games have accumulated a lot of knowledge of this
particular genre. They can build a list of gameplay
heuristics for this type of game, which are principles that
describe common properties of good designs of this genre.
Other game designers can examine their designs and judge
its compliance – and thereby discover its relative worth –
with those recognized heuristics. Therefore the heuristics
become a general guideline that helps game designers share
their knowledge, and benefits the whole game design
community by allowing designers to learn from the
experience of others. Table 1 shows a list of heuristics for
RTS games [12].
Name Description
Balance The rock-paper-scissors paradigm
Simplicity The number of building types should be
limited under 15.
Tech tree The tech tree should be broad instead of
deep.
Micromanagement Avoid complicated micromanagement.
Resources Resources should be dispersed as
equally as possible.
Table 1. A partial list of example heuristics for RTS games.
This list was developed by game designers at Ensemble
Studios.
Figure 7. Cognitive walkthrough can be used to evaluate the
level designs of action games such as Mario 64. As the
walkthrough proceeds, game designers should ask themselves
questions like, “Will the gamer know what goal is to be
achieved?” and “Will the gamer realize the actions and
strategy required to achieve that goal?”
COGNITIVE MODELING
Cognitive Modeling – the technique to build models to
simulate human behaviors – has been used by both the HCI
community and game designers, but in two very different
ways. HCI researchers use cognitive models, such as
Model Human Processor and GOMS, to predict task
performance times of different designs [13]. By contrast,
game designers mainly focus on using simple cognitive
models to create more realistic behaviors of synthetic
characters in virtual game worlds, like Sega’s Seaman.
Many game designers have noticed that while some game
genres attract a wide variety of gamers, from young to old,
casual to hardcore, others are played exclusively by a small
group of dedicated gamers at a very high skill level. FPS
(first-person shooter) games are a good example. Due to
the nature of FPS games, gamers playing them need to have
good eye-hand coordination and have fast reflexes.
Therefore gamers become skilled users, playing games
becomes a routine cognitive skill, and performance issues
become very important. GOMS are designed specifically
to measure performance issues. Thus it can be deployed to
analyze the efficacy of FPS games.
Figure 8. Sega’s Seaman made use of several cutting edge
technologies - artificial life, speech recognition, and cognitive
modeling - to create a virtual sea world populated with
synthetic characters.
Figure 9. GOMS can be used to analyze the efficacy of FPS
games such as Half Life: Counter Strike.
Other genres, such as RPG, are played in a more casual
setting that mainly involves exploration and learning, to
which GOMS does not apply well. But the Model Human
Processor can still help game designers to improve the
efficiency of their UI designs.
CONCLUSTION
We have explored the many aspects of HCI research and
practices and shown their close ties to the corresponding
areas of game design. For all the challenges facing game
designers, we can always find useful help from the many
facets of HCI. The effort to integrate HCI processes and
methods into game design is still at a primitive stage and
demands a lot of attention. The game industry is so young
that it lacks both sound disciplines and useful tools. We
believe that by combining HCI and the raw experiences of
game designers, we have a great opportunity to create the
tools of the trade for the game design community and
establish disciplines that can guide game design in the new
century.
REFERENCES
1. Game Software Industry Report. In AlienBrain product
catalog. NxN software. 2001.
2. Jakob Nielsen. Usability Engineering. Academic Press,
San Diego, CA 1993.
3. Chuck Clanton. An Interpreted Demonstration of
Computer Game Design. In Proceedings of CHI. 1998.
4. Randy Angle, and William Dwyer. Effective Project
Management. In Proceedings of Game Developers
Conference. 2001.
5. Marc Saltzman. Game Design: Secretes of the Sages.
Macmillan Publishing. 2000.
6. John M. Carroll. Human-Computer Interaction in the
New Millennium. ACM press. New York, NY, 2002.
7. HCI lecture notes: think-aloud protocols. Available at
http://staff.psy.gla.ac.uk/~steve/HCI/cscln/trail1/Lecture
5.html
8. John M. Carroll. Human-Computer Interaction in the
New Millennium. ACM press. New York, NY, 2002.
9. Jakob Nielsen, and Robert L. Mack. Usability
Inspection Methods. John Wiley & Sons, Inc. 1994.
10. Jakob Nielsen, and Robert L. Mack. Usability
Inspection Methods. John Wiley & Sons, Inc. 1994.
11. Jakob Nielsen. Usability Engineering. Academic Press,
San Diego, CA 1993.
12. Mickey Kawick. Real-Time Strategy Game
Programming Using MS DirectX 6.0. Wordware
Publishing, Inc. Plano, TX, 1999.
13. Gray, W. D., John, B. E., & Atwood, M. E. (1993)
"Project Ernestine: A validation of GOMS for
prediction and explanation of real-world task
performance." Human-Computer Interaction, 8, 3, 237-
209.