Spoors, G. (2001). Cognitive Work in Computer Gameplay. ANZCA Proceedings. Available at: http://praxis.massey.ac.nz/fileadmin/ praxis/papers/GSpoorsPaper.pdf
Cognitive Work in Computer Gameplay
Abstract
Turkle (1984, 1995) draws from Piagetian
cognitive psychology to identify three stages in
childrens’ relationships with computer games,
and two dominant styles, or modes, that
characterise adults’ use of computers. This
essay reviews her approach, arguing that
Piaget’s four stages of cognitive development—
sensorimotor, preoperational, concreteoperational
and formal-operational—are
usefully identified as modes that are combined
and traversed during gameplay.
Introduction
The following paper suggests a methodology
for analysing computer gameplay that reworks
Turkle’s (1984, 1995) largely Piagetian-based
research. For Piaget, of course, cognitive
development is explained through the
“assimilation” of perceptions within existing
cognitive “schema,” and the “accommodation”
of new perceptions through the modification or
creation of schema. The child is seen as a naive
scientist, formulating increasingly complex yet
relatively consistent cognitive schema in a
process which Piaget refers to as “equilibration.”
This process passes through four general stages:
sensorimotor (birth to 2 years), preoperational
(2-7 years), concrete-operational (7-12 years)
and formal-operational (12 years to adulthood)
(Wadsworth, 1989).
Turkle (1984) draws from Piaget’s work to
identify three general stages in children’s
relationship with computers. In the first stage,
children show a metaphysical interest in the
status of the computer as alive or dead; in the
second stage, children are concerned with
mastering the machine; in the third stage,
children use the computer as a way of thinking
about, or defining, their identity. Turkle also
identifies two kinds of mastery, “hard” and
“soft” (p. 102). The “hard” style of mastery is a
top-down approach that involves progressively
breaking down tasks into component functions,
as in procedural programming (1995, p. 32). The
“soft” style of mastery is an unstructured,
bottom-up approach to problem-solving in
which users are unconcerned with the
underlying rules of the machine, and are content
to experiment, make mistakes, and reconsider
different approaches until a workable model is
found.
Turkle initially refers to “hard” and “soft”
mastery as “ideal types” (1984, p. 11), but
subsequently argues that the analogous styles of
“hobbyiest” and “hacker” subcultures are “best
understood as different modes of relationship
that one can have with a computer” (1995, p. 33)
and which may be combined. However, Turkle
is more concerned with broad cultural shifts than
the dynamic ways in which modes are combined
or traversed during mainstream computer
gameplay. Ultimately, her distinction between
“hard” and “soft” mastery—or, rather, between
“abstract” and “concrete” modes of thought (p.
55)—is too simplistic for any close analysis of
the cognitive activity of players.
The concept of “modes of thought” is
founded on a conflict in cognitive sciences
between the perceived “psychic unity of
[hu]mankind” (Tambiah, 1990, p. 84) and the
obvious diversity of human cultures. While early
anthropologists categorised modes of thought in
various ways, the predominant concern has been
the presence or absence of “rationality” in
particular cultures, and, consequently, a
distinction between prelogical and logical
thought (Horton & Finnegan, 1973, p. 17). It is
now recognised, however, that different
“modes” of rationality coexist in all cultures and
individuals (Tambiah, 1990, p. 84). This is
evident in that Piaget’s stages of development
describe a movement from prelogical through to
logical (or “rational”) thought in the individual.
Most importantly, Piaget’s stages overlap, and
are often incompletely traversed, such that
“normal” adult cognition has characteristics of
earlier stages (McInerny & McInerny, 1998, p.
30). Consequently, we can view Piaget’s stages
as assimilated cumulatively by an individual as
relatively distinct modes of cognition.
Viewing Piaget’s stages as modes is, of
course, a heuristic gesture, and more detailed
analysis would need to address the cognitive
processes that give rise to and cross modes.
Nonetheless, just as looking at development in
terms of Piaget’s stages is a useful way to begin
analysing a child’s overall developmental
process (Wadsworth, 1989), it is useful to begin
an analysis of teenage or adult gameplay in
terms of shifts between or across cognitive
modes. We might define the modes identified
below, then, as cognitive tactics or processes
that recapitulate the stages of cognitive
development, and which players choose from or
combine when interacting with a computer.
Sensorimotor Mode
Sensorimotor activity generally moves from
reflexive movements towards intentional and
coordinated interaction with the physical world,
and is characterised by an increasing ability to
use objects creatively and in combination
(McInerny & McInerny, 1998). We can identify
a sensorimotor mode of activity in the way
players master computer game interfaces. For
example, players of an arcade game like
Dungeons and Dragons: Towers of Destiny
(Figure 1) may initially look from the controls to
the screen and back again to determine the
relationship between controls and action on the
screen. Through the trial and error use of the
buttons and joystick, players will identify which
character can be mo ved, where it can be moved
to, and the ways it can be moved. The player
will also learn that combining joystick
movements and buttons will produce special
“moves,” such as a whirlwind sword strike. If
players see a character perform a previously
unseen move, they may experiment with the
controls until they can repeat the move. Indeed,
it is customary in two-player fighting games for
the more experienced player to hold off
attacking while the other player tries to
familiarise him/herself with moves.
Figure 2. King’s Quest X: Mask of Eternity
Once the interface is mastered as sensorimotor memory—when the actions may be reproduced without attention to the game controls and feedback comes principally from the screen—players’ sensorimotor problems extend to their characters’ agency in the game world, such as how to pass obstacles or acquire objects immediately out of reach. Roberta William’s King’s Quest X: Mask of Eternity a 3D adventure/role-playing game (RPG), provides a useful example of this. Let us say that a player has rudimentary sensorimotor control over the interface through familiarity with similar games, and, soon after starting the game, moves his/her character into a building with a grinding mill (Figure 2). The millstone rotates around a stone base, and to its left is a lattice that leads to a second-storey balcony. If the player moves the mouse cursor over a rope on the balcony, a message states that the rope “would be useful.” The game is here reduced to what we might call an in-game sensorimotor problem: How can the player direct the character to retrieve the rope? The player may attempt to climb the lattice by holding the “? ” and the <SPACE> bar simultaneously on the basis of the controls found in similar games. If so, s/he will find that the character will jump, but not climb, and may realise that there is no command for climbing anything except ladders. The player may then notice a gap in the lattice above the millstone, and deduce that if the character jumps onto the millstone it will then be able to jump through the gap onto the balcony. However, if the character jumps onto the millstone it will be knocked back and injured. The player might then deduce that the problem is a matter of timing the jump so that the millstone does not hit the character, but if the attempt is repeated often enough the character will keep getting knocked back and will die.
While retrieving the rope is part of a larger strategy of passing the game, sensorimotor goals are best characterised by their independence. That is, while it is possible to speak of a succession of sensorimotor problems, or of sensorimotor skills being combined, the processes associated with sensorimotor mastery are governed by a particular task associated with the physical activity of a character, such as: “How do I get my character to move?” or “How do I access or manipulate that item?” Any overarching strategy, or goal, is necessarily punctuated by attempts to resolve the sensorimotor component(s). It is not, however, simply a matter of players mastering sensorimotor problems and then moving into another cognitive mode, as many computer games are characterised by the repeated presentation of sensorimotor problems: How do I jump this ledge, How do I move that stone, How do I kill this opponent? The same problems are often presented with a variation in difficulty, through the use of time limits, more complex environments to navigate, or an increase in the speed, number, or strength of opponents. Gameplay, then, often involves not just the mastering of sensorimotor problems but the constant proving of sensorimotor mastery through repeated performance of problems with incremental increases in difficulty. Nonetheless, the resolving of sensorimotor problems often requires a shift into other modes, and sensorimotor processes provide or reinforce perceptual units that may be taken up by other modes of cognition during gameplay.
Preoperational Mode
During the preoperational stage, children begin to know things not only through physical actions but through representations, and this facilitates the development of their logical ability. However, reasoning is not logical but prelogical in that perception is privileged over reason in the evaluation of problems (Wadsworth, 1989). For example, children do not attend to all aspects of a stimulus, as when they evaluate the volume of liquid in a glass only in terms of the height of the glass, not its width or shape. Preoperational thought is also characterised by “phenomenalistic causality,” “nominal realism,” and “physiognomic perception” (McInerny & McInerny, 1998, pp. 22-23). Phenomenalistic causality describes the tendency to mistake the conjunction and contiguity of events as a causal relation. For example, a child may see the act of opening of blinds in the morning as causing the sun to appear. Nominal realism refers to the belief that signs are related to what they stand for, as if signs possess the qualities of the objects for which they stand. For example, a quality of the sun, such as heat, may be seen as residing in the word itself. Physiognomic perception refers to the projection of psychological properties to inanimate objects. For example, a child who has hurt himself with a hammer may ascribe intention to the hammer (“Why did you do that?”). This attribution largely reflects an inherent “egocentrism,” in that the child confuses an inner state of being (being hurt by a hammer) with an external situation (the hammer did it on purpose). More broadly, the child sees him/herself as the primary cause or focus of reality, and thereby fails to acknowledge the mental life and agency of others (Wadsworth, 1989).
Figure 4. Ral Partha RPG pewter figurines.
These preoperational processes are evident in computer gameplay in several ways. As Friedman (1995) observes, when a player enters into a cybernetic relationship with a computer the player may view the computer as an extension of his/her physical and mental agency and experience a blurring between his/her inner world and the (virtual) physical environment. In Piagetian terms, whenever the interface is mastered as sensorimotor memory, such that the player is able to easily manipulate the game world, the player may experience a form of physiognomic perception in which s/he attributes him/herself to the computer. This can lead to a form of nominal realism, in that an egocentric player may view interaction as a form of “invocation” that translates thought into deed, with no awareness that signs at the interface are representations of computational processes. In short, digital signs, such as the character, game world, and opponents, will be treated as the objects of the player’s actions. This may, in turn, lead to phenomenalistic perceptions, in that the conjunction of a particular action (pressing the fire button) and an effect at the interface (an explosion) may be seen as causally related, ignoring the possibility that the effect may have a hidden cause (the character may have walked over a “trigger,” or the explosion may have been a random event).
For Piagetians, conflict during social interaction forces children to acknowledge and accommodate the role and thoughts of others, and thereby lose egocentrism (Wadsworth, 1989). Similarly, computer games undermine the egocentrism of players by frustrating their attempts to control action in the game world. In many computer games, for example, a mouse cursor in the shape of a hand or gauntlet mediates between the player and a character on screen (as in Diablo, Figure 3). This may recall the direct physical manipulation of miniatures that occurs in the wargames and table-top RPGs (Figure 4) upon which games like Diablo are partly based. However, computer games limit the power of players to control their characters. There is, for example, rarely complete freedom of movement. In Diablo, the apparently seamless isomorphic background is in fact broken up into small, square units, and while the character may seem to walk between these units, it always stops in the middle of them in the same way that chess pieces are confined to their squares. The character is also unable to walk across or interact with certain objects, such as the trees, barrels and houses in Figure 3. Most obviously, characters are confronted by gamecontrolled opponents which may block or kill them. Such basic limitations, or conflicts, force players to shift from an egocentric impression that they are the sole or principal causal agent in the game. However, such a shift is not permanent, because whenever a player reasserts his/her agency through mastering some element of the interface or game world, a characteristically preoperational egocentrism may be re-established.
Concrete-operational Mode
During the concrete-operational stage, children’s logical ability develops such that they gain an understanding of the conservation of object properties, as well as the ability to organise items into an order (“seriation”) and construct classes of objects (“classification”) (McInerny & McInerny, 1998, p. 27). However, these logical operations are applied only to concrete objects and situations, not to hypothetical situations, purely verbal problems, or abstract concepts (Wadsworth, 1989).
Conservation is evident in computer game interfaces inasmuch as the same object is often represented in multiple forms. In early 2D sprite animation, changes in the state of game objects were highly visible and well-defined. In Figures 5 and 6, for example, the sprite of the main character in Dug Dug obviously changes from being alive to being dead. Many adventure and CRPGs like Diablo and Might and Magic VI have residual 2D sprite elements, in that the image of an object will change when picked up. For example, a sword may be represented by one sprite in the main game screen (Figure 7), and a different sprite in the player’s inventory (Figure 8). A player in a preoperational mode may see this as an actual transformation, whereas a player in a concrete-operational mode will recognise that the intrinsic qualities of the object have not changed. However, games are increasingly characterised by 3D animation, which allows for more dynamic simulations of physical transformations and changes in spatial orientation. Players may have to apply greater attention to issues of conservation, in that image variation in 3D games may reflect either the transformation of an object or a change of perspective as the character moves through the game world.
Figure 5. Sprite of character in Dug Dug.
Figure 6. Different sprite of same character in a different state.
Figure 7. Sword in Might and Magic VI game world.
Figure 8. Same sword in character’s inventory.
In most action-oriented computer games,
classification and seriation is evident in the way
players advance through a series of sensorimotor
problems organised into levels of increasingly
difficulty. Players may, for example, identify
that monsters on each level range from weak
through to the “boss,” but that the boss of each
level is more difficult than the basic opponent of
the following level. This classification and
seriation applies to game objects, such as
weapons and armour, which are organised
hierarchically according to the bonuses or skills
they give characters. Weaker weapons are
usually found on the lower levels, and more
powerful weapons are usually found on the
higher levels, but sometimes a powerful weapon
will be placed in a lower level to give the player
a temporary advantage. In Level 2 of Diablo, for
example, a player’s character may fight and kill
the Butcher and take his Cleaver, which does
more damage than other weapons on the
surrounding levels.
Since players are only able to act and
improvise within the boundaries defined by the
program, it can be argued that players repeatedly
utilise a concrete-operational mode to identify
the rules of a game and/or the state of its data.
Game-world logic is different from real-world
logic (games, for example, do not have to
respect the rules of gravity), and while games of
a similar genre may be governed by similar
rules, each game’s code or data is different. That
is, the logic of a game may only be valid in the
context of the game, to the extent that players
may be required to accept game- logic that
contradicts their own logic or the logic of
another part of the game. In the RPG The Bard’s
Tale, for example, the player controls characters
in the city of Scara Brae, and there is unusual
street which, if characters walk along it, soon
seems to be longer than the width of the city
walls. If characters keep walking down this
street, they will not come to its end, but keep
travelling an infinite distance. This is illogical in
terms of real-world logic (a space cannot be
larger than the space that contains it) and the
game-logic that governs character movement
through the rest of Scara Brae (if you move
along a street, you will come to its end).
However, no matter how far the characters walk,
if they turn around and take a few steps they will
come back to the start of the street. This logic is
only applicable to the street in that game, and
the player must conform to it if they wish to play
the game.
Formal-Operational Mode
Formal-operational thought is characterised
by the ability to reason deductively and
inductively, with a more fully developed
understanding of causation (Wadsworth, 1989).
Individuals who utilise formal-operational
thought may evaluate the logic of an argument
without reference to a context, that is, they may
draw conclusions that go beyond observable
results. We can see this kind of activity in
computer games when players attempt to apply
real-world logic to a game or try to anticipate or
reason through objects and event probabilities
that have not been encountered and/or are not
immediately present.
This is evident in the problem in the grinding
mill in King’s Quest X. If the player clicks the
cursor over the grinding stone, a text message
appears: “How can I stop the mill?” This c(l)ue
may re-orient the player’s cognitive activity in
terms of the following questions: “Have I seen
an object big enough to stop the mill, and, if not,
where might one be?” The player may consider
a range of objects with formal properties
appropriate to the function of stopping the mill:
an object equal to or larger than the millstone (a
heavy object, most likely made of rock), or a
small, hard object (such as a dagger). The player
could then hypothesise about where to find such
an object by relating it to its customary context.
If s/he had earlier passed the chicken coop
nearby, and remembers that axes are used to kill
chickens, s/he might reason that an axe is there.
However, the player of a computer game must
not only test his/her hypotheses through
observation, they must test it through physical
interaction if they are to finish the game, and
this requires shifts across cognitive modes.
For example, King Quest X’s realistic 3D
game world may motivate players to use realworld
logic in a formal-operational mode, in the
way indicated above. However, a player would
have to test their hypothesises about the axe by
interacting with the game world to discover its
logic, which requires a shift into a concreteoperational
mode. Their hypothesis would
subsequently be invalidated, in that there is no
axe at the coop, and no object can be placed on
the mill to stop it. Players familiar with
adventures and CRPGs may remember that
buttons or levers are the principal means of
causing objects to move in such games, and it is
a lever that actually stops the mill. Since the
player is rewarded for ignoring the perceptual
aspects of the problem (the size, shape and
material of the mill) and focusing on a logical
solution (the lever) the game may be (again)
seen as motivating a formal-operational mode.
However, the simplistic causal relation between
the lever and mill may be seen as motivating a
preoperational mode because there is no
attention to the mechanism of gears or
counterweights, that is, to the entire causal
sequence of the mill’s operation. Indeed, it is
possible that a player may find the sensorimotor
aspects of the problem so absorbing that other
cognitive modes are bracketed out, and failure to
resolve the problem might lead him/her to
abandon the game.
Suggested Research
Since Piaget delineated his stages in more
detail than space allows us to discuss here, an
obvious direction for future research is more
detailed analysis of the cognitive activity in
computer games. Such an analysis might
provide a useful way of categorising computer
games and genres. To take the most basic
example, action/arcade games may be seen
principally as a succession of sensorimotor
problems, whereas CRPGs may be seen as
governed by inferential and deductive processes
in a formal-operational mode as players
assimilate and accommodate game data to create
a schema for the developing story (Myers,
1990). Analyses of the cognitive processes in
particular genres or games could be related to
player demographics to identify significant
correlations (Durkin, 1999).
Existing cognitive analyses of film
(Bordwell, 1985; Branigan, 1991), which focus
upon the mental activity of the viewer, could
also be synthesised with a Piagetian approach,
which accounts for the combination of physical
and mental activity found in computer
gameplay. The relationship between cognitive
modes and narrative codes (Spoors, 2001) would
be of particular importance. For example, while
sequences of gameplay organised around
sensorimotor problems might be easily
accounted for in cognitive terms, any
understanding of story-based CRPGs like
Baldur’s Gate would need to clarify if certain
cognitive modes are preconditions for certain
narrative work to be performed, or if they affect
the work of certain narrative codes. Further
research along these lines would be invaluable
in developing an adequate account of computer
gameplay.
References
Bordwell, D. (1985). Narration in the fiction
film. London, Great Britain: Methuen.
Branigan, E. (1992). Narrative comprehension and
film. London: Routledge.
Durkin, K., & Aisbett, K. (1999). Computer games
and Australians today. Sydney: Office of Film
and Literature Classification.
Friedman, T. (1995). Making sense of software:
Computer games and interactive textuality. In
S. G. Jones (Ed.), Cybersociety: Computermediated
communication and community (pp.
73-89). Thousand Oaks, CA: Sage Publications.
Horton, R. & Finnegan, R. (1973). Modes of
thought: Essays on thinking in western and
non-western societies. London: Faber & Faber.
McInerny, D. M., & McInerny, V. (1998).
Educational psychology: Constructing learning
(2nd ed.). Sydney: Prentice Hall.
Myers, D. (1990). Computer game genres. Play
and Culture, 3, 286-301.
Spoors, G. R. (2001). Narrative and interaction
in computer games: Transdisciplinarity. In
Program, Abstracts, and Refereed Papers.
Annual Conference of the Australian and
New Zealand Communication Association
(pp. 210-24). Perth: Edith Cowan University.
Tambiah, S. J. (1990). Magic, science, religion
and the scope of rationality. Cambridge:
Cambridge University Press.
Turkle, S. (1984). The second self: Computers and
the human spirit. London: Granada Publishing.
Turkle, S. (1995). Life on the screen: Identity in
the age of the internet . New York: Touchstone.
Wadsworth, B. J. (1989). Piaget’s theory of
cognitive and affective development. New
York: Longman.
