Auditory Information In Video Games

Case: All Ghillied Up, Call of Duty 4: Modern Warfare (Activision, 2007)

Student: Bas Lievens, 3025845. Course: Game Studies, MA New Media and Digital Culture, Utrecht University. Teacher: David B. Nieborg Date: 08-07-2011

Abstract.
The interactive nature of video games has important implications for the way that audio behaves (Collins 2008). There are different perspectives on the function of audio in video games and how players understand it. I argue that in order to interpret the auditory stimulation a player has to have knowledge about the specific situation in which it appears. This knowledge arrives from the rules and context in which audio occurs. The theoretical and methodological framework of this essay stems from the scholars, Zach Whalen, Karen Collins and Kristine Jrgensen. The aim of this essay is to construct a conceptual framework which can be used to understand the function of audio in computer games. I will investigate the context of the thirteenth level, All Ghillied Up, from the single player of the first-person shooter Call of Duty 4: Modern Warfare and try to determine the function of auditory information and the way in which players understand these signals. Furthermore this essay can be used as a starting point for analyzing the use of audio in other video game genres.

Keywords: audio, first-person shooter, diegetic- and non-diegetic sound, musical sound and sound effects, in-game context and auditory cognition.

Introduction
A lot of academic work on video games focuses on video aspects rather than auditory aspects. This is remarkable for both aspects construct the game world. The interactive nature of video games has important implications for the way that audio behaves (Collins 2008). While navigating inside the virtual environment players activate specific audio events (Collins 2008). They are considered as nonlinear texts which distinguishes them from other forms of media (Collins 2008, 3). Therefore I argue we should examine audio in its specific interactive connotations. Zach Whalen, a PhD student at the University of Florida, has a particular interest in Video Game Genre Theory, and argues that audio evokes cognitive meaning (Whalen 2001). He believes that the sound signals function as a communication device which help the player understand the game environment and master the game scripts (Whalen 2001). Karen Collins, Canada Research Chair at the Canadian Centre of Arts and Technology at the University of Waterloo, argues that audio is adaptable to the structure of gameplay (Collins 2008); the actions, strategies and motives of the player which are supported by a specific narrative (Oxland 2004, 7). Because there are different genres, platforms and players audio has a symbolic function that helps: to provide a sense of cohesiveness (Collins 2008). I would argue, however, that such a semiotic or technological perspective is too deterministic for they surpass the specific in-game context in where auditory information is transmitted and received. I argue that in order to interpret the auditory stimulation a player encounters in video games, they need to have knowledge about the specific situation. This knowledge arrives from the rules and context in which audio occurs. Therefore I will investigate the context of the thirteenth level, All Ghillied Up, from the single player of the first-person shooter Call of Duty 4: Modern Warfare and try to determine the function of auditory information. My research question is: What is the function of auditory information in the thirteenth level, All Ghillied Up, from the single player of the first-person shooter Call of Duty 4: Modern Warfare (Activision, 2007), and how do players understand these auditory signals? In writing this essay I want to create insight in the functions of audio in video games, and in particular in single player first-person shooters. The theoretical and methodological framework of this essay derives from the scholars, Zach Whalen, Karen Collins and Kristine Jrgensen. The aim of this essay is to construct a conceptual framework which can be used to understand the function of audio in computer games. Next to that this essay can be used as a starting point for analyzing the use of audio in other video game genres. This essay starts with an inventory of the discourse on video game audio which will define several analytical perspectives on the function of audio in video games and players understanding these signals. In the next chapter I will present the analysis of the specific

case and will try to determine how players interpret auditory information in single player first-person shooters. In the conclusive chapter I will summarize my findings.

The discourse on audio in video games

Zach Whalen argued that: cognitive theories of perception and questions of immersion versus engagement as means of understanding flow or pleasurability in games, allows for a richer understanding of the complex communication involved in videogame music (Whalen 2001). What Whalen thus argues is that video game music is a form of communication. Whalen perceives video games as hypertexts or interactive narratives in which flow is the marker for interaction quality between the game and the player (Whalen 2001). He clarifies the concepts of immersion, engagement and flow in order to fortify his statement that music is a complex communication system (Whalen 2001). In his argument, Whalen uses traditional Film Music Theory to explain which cognitive processes are involved in understanding audio in videogames (Whalen 2001). He then uses the basic idea that there is a difference in diegetic sound and non-diegetic musical sound. Diegetic sound, according to Whalen, is every sound that populates an environment. Diegetic sound, is sound that is represented to originate form a source within the virtual world, like for example voices of characters or sounds made by objects in the narrative (Carlsson sd). However, I would argue that Whalens argument is more convenient to what he calls non-diegetic sound (Whalen 2001). Non-diegetic sounds, are sounds of which the source is not visible on the screen, thus off-screen, and which have not been triggered by the player. For instance the theme that is played in the background or the sound effects (Carlsson sd). Whalen argues that the video game music system supports and improves the narrative experience of gameplay: the music in videogames is one of several elements that make game play a compelling visual and aural experience which immerses players in a fictional space (Whalen 2001). He goes on to say that while navigating thought the virtual environment the player explores other sections that all embody their own musical signature, and therefore video game audio is connected to different game states (Whalen 2001). His main point is that the sound system, both diegetic and non-diegetic sound, evokes a narrative, cognitive meaning. Therefore audio is motivational and functions as communication device in different parts of the game to characterize fictional spaces in game environment and assists the player in his mastery of the game scripts (Whalen 2001). Karen Collins argued that game audio have been affected by the nature of technology (Collins 2008, 5). From a technological-economical perspective she claims that innovation in hardware and software technologies, and structural changes like design and marketing affected audio development in videogames (Collins 2008, 123). Collins argues that audio in video games is constrained by genre 4

and audience expectations, by the formal aspects of space, time, and narrative, and by the dynamic nature of gameplay (Collins 2008, 6). She also argues that game genre defines audio in arguing that different genres have different characteristics and that these typical features have a: distinct impact on the ways in which audio functions in terms of its relationship to the player, and to the games narrative or diegesis (Collins 2008, 124). However, this would also mean that players can interact with audio in numerous ways. Collins argues that audio in games serves different functions: dependent on genre, platform, players and players familiarity with a particular game (Collins 2008, 127). She argues that sound signals can function as cues for the player. Collins calls this the prepatory function which: alert the player to an upcoming event or to forewarn the player of approaching enemies (Collins 2008, 130). Collins seems to agree with Whalen that audio communicates to the player, whereupon he or she can anticipate action. This knowledge of so-called sound signals (sound effects, dialogue and recurring musical themes) can help the player to complete a specific objective and be successful in playing the video game (Collins 2008). Her main argument is that: music can create the structure of the gameplay (Collins 2008, 131). Audio therefore is adaptable and has a symbolic function that helps to: provide a sense of cohesiveness, and help to guide the player along in terms of the narrative and his or her location in the game matrix (Collins 2008, 137). I would argue, however, that there seems to be significant shortcoming in both the semiotic perspective of Whalen and technological perspective of Collins while analyzing audio in video games for both perspectives pre-assume determinism. I would argue that there is such a thing as in-game context where auditory conception takes place. Professor of the Department of Music Theory of the Schulich School of Music in Canada, Stephen E. McAdams and Emmanuel Bigand, a professor on cognitive psychology at the University of Bourgogne argue that there are different phases of processing in auditory cognition. In order to understand audio, there are: elementary phases of processing what means that we have to be able to perceive audio through our sensory apparatus (McAdams en Bigand 1993, 1). But, there are also: higher-level processes like mental representations and interpretations that: are necessary to elaborate a coherent representation of the sound world (McAdams en Bigand 1993, 1). What they explain is that the perception system (of the player) analyses the situation with the use of knowledge that a player has received from auditory information in the game world. So, in order to interpret audio a player encounters in video games, they need to perceive it, and have knowledge about the specific situation in which it appears. The Norwegian Kristine Jrgensen, a postdoctoral scholar in Information Sciences and Media Studies at the University of Bergen has a similar contextual perspective. She argues that audio in a video game is tightly integrated with the actual rules and the context in which a specific sound occurs (Jrgensen 2008). Although she seems to agree with Whalen and Collins that there are specific 5

auditory signals that support gameplay, she argues that the interpretation audio in videogames is highly textual and the player therefore has to understand the specific situation, or gameplay context, in order to understand the auditory signal (Jrgensen 2008). Jrgensen argues that the game structure exists in a virtual environment and constructs a location where information becomes combined with the sense of being positioned in the virtual world. This information can also be provided through sound signals and it is therefore necessary to investigate what kind of information the gameplay of Call of Duty 4 Modern: Warfare provides for the player. Jrgensen argues that sound signals serve two functions; it affects the choice of action and the understanding of the situation by the player (Jrgensen 2008). She thus argues that due to the dualistic function that audio provides to the player we cannot identify certain sound signals as related to one specific information system, like for instance Whalen does, and it therefore depends on context. In this essay, this context arises from the single player of the first-person shooter Call of Duty 4: Modern Warfare and in particular level thirteen; All Ghillied Up. In order to understand how audio is used and what the implications are for the player I will investigate how players are organized and what the in-game context is.

An auditory analysis of All Ghillied Up

All Ghillied up is a flashback of an earlier mission when Captain Prince, the main character, still was a Lieutenant. The scene is set in Prypiat, Ukraine. In this level Prince has to follow Captain MacMillan to a specific location where they can assassinate the Russian terrorist Imran Zakheav. In order to reach this location they have to move stealthily alongside Russian guards, the Ultranationalists, and avoid the radioactive zones from the Chernobyl disaster. The mission ends when they reach the location (All Ghillied Up sd). The single player mission All Ghillied Up is part of the first-person shooter Call of Duty 4: Modern Warfare. A first-person shooter is a video game which is perceived from a first person perspective and where the player has total freedom of movement. In order to control a in-game character the player has to learn the controls. These instructions are usually given through a mouse and keyboard. The Portuguese Doctor on Philosophy Elias Herlander, from the University of Beira Interior, argues that in order to understand the first-person shooter we must, as a player, accept the entrance in the trap of the virtual (Elias 2009). This virtual world is a combat zone in which a player has to complete specific objections following a simple ret of rules (Jrgensen 2008). The methodology of Jrgensen provides a useful tool in analyzing how understanding of audio is constructed. She argues that the player needs to have knowledge of the circumstance that produces the sound, what it means in a particular context, and which object it produces in order to grasp the strategic importance (Jrgensen 2008). Before I can determine how players interpret these sounds I need to find out which sounds are used in the All Ghillied Up level. Due to the interactive 6

nature of videogames different sounds can be triggered throughout the gameplay. Therefore I analyzed four videos of players (Attila16, JohnyBravoSix, Kiarash32, Alieninvader2) which uploaded their practice on Youtube.com1.

Sound generated by players

These diegetic sounds are caused by the action of the player and provide information which should trigger the player to response. In the All Ghillied Up- level the diegetic sounds that are triggered by the player and populate the virtual environment are footsteps of the player while moving around, running and jumping. Other sounds stem from gun action; gunshots, reloading and the sound that is made when the player zooms in with a scope.; the player than triggers a heartbeat sound in while the character hold its breath. These player generated sounds can help the player to determine where he or she is in the virtual environment (Jrgensen 2008).

Sound generated by allies and enemies

Sounds generated by allies and enemies are also diegetic. The source of the information arrives from within the virtual environment (Jrgensen 2008). Jrgensen argues that it is crucial for the player to determine whether the sound is produced by an ally or an enemy, for these sound signals provide spatial information and facts about the physical state of in-game characters. The action of the player does not create these sounds for they are created off screen (Jrgensen 2008). In the thirteenth level of Call of Duty 4: Modern Warfare, allied sounds are made by Captain MacMillan. For instance the first spoken sentence: Too much radiation, Well have to go around. Follow me and keep low. These sounds signals give the player spatial and identifying information and also learn the player how to navigate and how to behave; Stay low and move slowly. Well be impossible to spot in our ghillie suits (Jrgensen 2008). These sound signals also provide information on the location of enemy characters; Dont move. Weve got a lookout in the church towerand a patrol coming from the north. The objectives arrive from the sound signals that Captain MacMillan provides. There are also sounds of enemies in the form of dialogue when the player approaches them. These dialogues are spoken in a specific dialect and provide information about the location and movement of enemies. Next to that, they trigger the player to be pay attention for action is to come (Jrgensen 2008).

Sounds generated by the game system

System generated sounds are non-diegetic sounds that are not generated by a specific command of any player but still provide information about changes in avatar state of game state (Jrgensen 2008). Jorgensen argues that the game system, and not the player, produces these sounds. The
1

The sources are in the Appendix.

source of these sounds is usually not clear, but they can be heard by all the characters in the game. In the All Ghillied Up -mission there are a lot of system generated sounds. First of all, the theme of this mission, the musical sound, also influence the player. The theme is somewhat military with rhythmic drums, and an occasional have base. In tradition of Whalen and Collins we could argue that this musical sound is simply creating an particular atmosphere. I would however argue that in the case of mission thirteen Call of Duty 4: Modern Warfare, the musical does also provide information, and understanding for the gameplay in a specific in-game context. The tempo and rhythm of the theme enhances when a player navigates towards an hostile in game environment, making the player aware of danger. A similar statement can be made about the sound effects. For instance, in this mission there is a sound of a overflying helicopter which triggers the player to seek for cover.

Results
The analysis shows that auditory cognition is a complicated occupation. In Call of Duty 4: Modern Warfare audio is understood from a first-person view. The listener, or player, has to find out which sounds are appropriate and useful in a specific situation in order to comprehend the conclusive situation. In this mission not all the sounds are evenly important. Therefore the player has to work out which ones are relevant. In order to comprehend the meaning of a particular sound in a specific context the player needs to be able to distinguish the dissimilarity between the source of the sound and the generator of the sound: knowing this is essential for the players interpretation of sound (Jrgensen 2008). The function of audio thus is dependent of the specific situation in which it occurs. As argued, sound can function as communication device, especially when the visual aspects of the video game are reduced (Jrgensen 2008). Sound generated by the player is diegetic and provides information which should trigger the player to response. Diegetic sounds from allies are not produced by the player but support the player in navigation and behavior inside the virtual environment. Sound generated by enemies are also diegetic and provide information on the location of enemy characters. System generated sound are non-diegetic and provide information about changes in avatar state of game state (Jrgensen 2008). Due to the complex diegetic and nondiegetic character of audio it becomes difficult for the player to comprehend the usability. Therefore, knowledge about the specific in-game context in which a sound occurs proves to be useful in the thirteenth level, All Ghillied Up, from the single player of the first-person shooter; Call of Duty 4: Modern Warfare.

Conclusion
In this essay I tried to create insight in the function of audio in video games, and in particular in single player first-person shooters and examined how players make sense of audio in a specific context, like the All Ghillied Up-level from the single player of the first-person shooter Call of Duty 4: Modern Warfare. The interactive nature of the case had important implications on the way that audio behaves in the virtual environment. Whalen argued that audio in video games supports the player and thus functions as a communication system. He argues that there are different game states on which this auditory communication system adapts: game audio is closely connected to specific game states (Whalen 2001). Collins argued that audio in video games can serve different functions: dependent on genre, platform, players and players familiarity with a particular game (Collins 2008, 127). Due to the diversity in gameplay; the actions, strategies and motives of the player, audio has to be able to adapt (Collins 2008). Collins therefore argues that audio is adaptable and has a symbolic function that helps to provide a sense of cohesiveness, and help to guide the player along in terms of the narrative and his or her location in the game matrix (Collins 2008, 137). I noticed, however, that these perspectives surpass the specific in-game context where auditory information takes place and argued that, in order to understand the audio in video games, players need to perceive it and have to understand the sound source and the situation in which the sound signal occurs. Jrgensen has a similar contextual perspective in arguing that audio in a video game is tightly integrated with the actual rules and the context (Jrgensen 2008). The analysis showed that the audio in Call of Duty 4: Modern Warfare is part of the game structure for it has a direct relation to the players actions. In the case, the sounds generated by allies and enemies provide instance response. As Jrgensen argued audio also affects player behavior by providing spatial cues and information about changes in game state and character state (Jrgensen 2008). Thus, the auditory information which is provided by the game structure, helps the player understand the rules of the game, improving his gameplay. The player has to, however, comprehend the specific in-game context in which these sounds signals occur, in order to understand its connotations.

Bibliography
Aarseth, Espen. Playing Research: Methodological approaches to game analysis. Game Approaches. Melbourne: University of Bergen, 2003. 1-8. Adams, Ernest. Fundamentals of Game Design. . Berkeley: New Riders, 2010. All Ghillied Up. http://callofduty.wikia.com/wiki/All_Ghillied_Up (geopend 06 10, 2011). Carlsson, Sven E. FilmSound.org. http://filmsound.org/terminology/diegetic.htm (geopend 06 05, 2011). Collins, Karen. An Introduction to the History, Theory, and Practise of Video Game Music and Sound Design. Cambridge: The MIT Press, 2008. Elias, Herlander. First Person Shooter. The subjective cyberspace. . Covilha: LabXom Books, 2009. Jrgensen, Kristine. Audio and Gameplay: An Analysis of PvP Battlegrounds in World of Warcraft. Game Studies. The international Journal of Computer Game Research, 2008. Manovich, Lev. the language of new media. Cambridge: MIT Press, 2001. McAdams, Stephen, en Emmanuel Bigand. Introduction to Auditory Cognition. In Thinking in Sound: The Cognitive Psychology of Human Audition, door Emmanuel Bigand Stephen McAdams, 1-9. Oxford: Oxford University Press, 1993. Oxland, Kevin. Gameplay and Design. Essex: Pearson Education Limited, 2004. Raessens, Joost. Computergames as participatory media culture. In Handbook of computer game studies, door Joost Raessens & Jeffrey Goldstein, 373-388. London: The MIT Press, 2005. S. Egenfeldt-Nielson, S.J. Smith & S. P. Tosca. Understanding video games. The essential introduction. New York: Routledge, 2008. Whalen, Zach. Play Along - An Approach to Videogame Music. The International Journal of Computer Game Research, 2001. Young, Bryan-Mitchel. Changing Views - Worlds in Play. . Gaming Mind, Gaming Body: The Mind/Body Split For a New Era. Vancouver: Authors & Digital Game Research Association, 2005. 1-6.

10

Appendix
Attila16: Call of Duty 4 Modern Warfare Walkthrough 13 All Ghillied Up http://www.youtube.com/watch?v=DwGV9Nl4RjE JohnyBravoSix= Call of Duty 4 [HD] Modern Warfare All Ghillied Up part 1 [1920x1200 Ultra High] http://www.youtube.com/watch?v=7WCkqmbXMfM Kiarash32 = Call Of Duty 4 - All Ghillied Up - Risky Gameplay Part 1 (HD) http://www.youtube.com/watch?v=1WrXB-oGpLk Alieninvader2 =All Ghillied Up - Walkthrough - Part 1 http://www.youtube.com/watch?v=psMegTIsZkM

11