The Control of Instrumental Action Following Outcome Devaluation in Young Children Aged Between 1 and 4 Years

Journal of Experimental Psychology: General
Copyright 2008 by the American Psychological Association
2008, Vol. 137, No. 1, 39 –51
0096-3445/08/$12.00
DOI: 10.1037/0096-3445.137.1.39
The Control of Instrumental Action Following Outcome Devaluation in
Young Children Aged Between 1 and 4 Years
U. M. H. Klossek, J. Russell, and A. Dickinson
University of Cambridge
To determine the role of action– outcome learning in the control of young children’s instrumental
behavior, the authors trained 18- to 48-month-olds to manipulate visual icons on a touch-sensitive display
to obtain different types of video clips as outcomes. Subsequently, one of the outcomes was devalued by
repeated exposure, and children’s propensity to perform the trained actions was tested in extinction. On
test, children with a mean age greater than 2.5 years performed the action trained with the devalued
outcome less than those trained with the still-valued outcome, thereby demonstrating that their actions
were mediated by action– outcome learning. By contrast, the instrumental responses of younger children
(mean age
2 years) were resistant to outcome devaluation and may have been elicited directly by the
icons associated with each response, rather than mediated by a specific action– outcome expectation.
Keywords: goal-directed action, instrumental learning, outcome devaluation, children
Most human adults rarely regard the capacity for intentional,
bust, as the novel response was retained for as long as 5– 6 days by
goal-directed instrumental action as a remarkable achievement.
2- to 3-month-olds in the absence of any further training or
We do it all the time: We are able to perform an action in the belief
exposure to the training situation (Rovee-Collier, 1983). However,
that it will bring about a goal or a change in the world that fulfils
most standard instrumental learning paradigms are designed to
our needs and desires. Critically, a course of intentional action can
assess learning simply in terms of response acquisition. Therefore,
be selected and initiated even though at the time of performing the
although demonstrating that infants possess remarkable learning
action, the goal is not physically present. By being able to rely on
capacities even in the first few months of life, these studies do not
internal representations of the desired goal or outcome and of the
permit inferences to be drawn regarding the intentional status of
causal action– outcome relationship, intentional, goal-directed ac-
these actions. Establishing the goal-directed status of an action
tion enables agents to transcend their current physical environment
requires the demonstration that it is performed specifically to bring
and extend their sphere of influence to objects and events beyond
about a particular outcome, rather than just that it is acquired when
the immediate present and local space.
it is followed by an attractive effect.
Although the capacity for goal-directed action in normal human
Determining the intentional status of an instrumental action in
adults does not require empirical demonstration, the intentional
the absence of a reliable verbal report from the agent is, however,
status of young children’s instrumental actions is less clear. Stud-
not an easy matter. To address this issue empirically, students of
ies of instrumental learning in infancy have shown that infants, and
animal learning have developed a behavioral assay for intentional,
even newborns, are sensitive to the contingent consequences of
goal-directed action by devaluing the outcome following instru-
their behavior (Kalnins & Bruner, 1973; Rovee & Rovee, 1969;
mental training. The concept of goal-directedness assayed by this
Siqueland, 1969; see also Hauf, Elsner, & Aschersleben, 2004) and
procedure is one in which the action is goal-directed if perfor-
that they reliably acquire a new instrumental action when it im-
mance is mediated by the interaction between two representations
mediately causes an attractive outcome, but not when an equiva-
(Dickinson, 1985, 1989): The first is a representation of the causal
lent outcome is presented independently or noncontingently (e.g.,
or contingent relationship between the action and the outcome,
Rovee-Collier, Morrongiello, Aron, & Kupersmidt, 1978;
whereas the second is a representation of the current value of the
Siqueland & DeLucia, 1969). This learning was remarkably ro-
outcome. It is mediation by the action– outcome representation that
ensures that the action is directed toward obtaining the outcome,
whereas the dependence on a representation of the outcome value
U. M. H. Klossek, J. Russell, and A. Dickinson, Department of Exper-
ensures that the outcome functions as a goal in controlling perfor-
imental Psychology, University of Cambridge, Cambridge, England.
mance.
This research was supported by the Biotechnology and Biological Sci-
In the prototypical outcome devaluation experiment (e.g., Ad-
ences Research Council, Medical Research Council, and the University of
ams & Dickinson, 1981), animals are initially trained to perform
Cambridge Behavioural and Clinical Neuroscience Institute, supported by
an instrumental action to gain access to a particular outcome,
a joint award from the Medical Research Council and the Wellcome Trust.
which is subsequently devalued. Although the effectiveness of the
We thank M. R. F. Aitken for his assistance.
devaluation treatment depends on the nature of the outcome, a
Correspondence concerning this article should be addressed to A. Dick-
common procedure is the induction of specific satiety for the
inson, Department of Experimental Psychology, University of Cambridge,
Downing Street, Cambridge CB2 3EB, United Kingdom. E-mail:
outcome through simple exposure (Balleine & Dickinson, 1998b;
ad15@cam.ac.uk
Colwill & Rescorla, 1985). This devaluation treatment must occur
39

---

40
KLOSSEK, RUSSELL, AND DICKINSON
in the absence of the opportunity to perform the instrumental
Experiment 1
response to prevent the formation of any direct association be-
tween the instrumental action and the devaluation treatment. Fi-
To ascertain that the touch response could be brought under
nally, the impact of the outcome devaluation is assessed through
instrumental control, we trained children of the youngest age used
in our experiments to reach out and touch two different screen
the subsequent performance of the instrumental action. If the
areas marked by red and green butterfly icons. During acquisition,
action is goal-directed, performance of the action trained with the
one of the actions was followed by short video clips featuring
now devalued outcome should be reduced in this test relative to a
colorful animated scenes, whereas the other produced no outcome.
control condition in which the outcome is not devalued. The
Given that the video clips were effective rewards for the children,
critical feature of this test is that postdevaluation performance is
the children should have learned to perform the rewarded action in
measured in extinction and therefore in absence of the outcome.
preference to the nonrewarded one if they were sensitive to the
Testing in extinction ensures that performance is mediated by the
action– outcome contingency.
representations the instrumental contingency established during
The second phase assessed the effect of reversing the instru-
training rather than through the direct impact of the devalued
mental contingency so that the action rewarded during acquisition
outcome during the test.
no longer produced the outcome, whereas the previously unre-
We applied the same logic to investigate whether young chil-
warded response did. It was important to determine whether the
dren represent their own instrumental actions in terms of specific
target actions were sensitive to changes in the action– outcome
action– outcome relationships. There is currently little evidence
contingency. Unless the children adapted to the shift in contin-
that young children are able to select a specific action on the basis
gency by reversing their preference for the two actions, it was
of the current value or utility of the past outcome of the action. In
possible that response perseveration would have masked any sen-
fact, studies that examined problem solving or planning, using, for
sitivity to outcome devaluation.
example, child-appropriate versions of standard adult planning
tasks, such as the Tower-of-Hanoi disc-transfer puzzle (Simon,
Method
1975), have often found that 2- to 3-year-olds perform rela-
tively poorly (e.g., Klahr & Robinson, 1981; Kopp, O’Connor,
Participants
& Finger, 1975; O’Sullivan, Mitchell, & Daehler, 2001; Welsh,
1991). However, these tasks were designed to assess relatively
Participants were 5 boys and 3 girls, aged between 19 and 26
complex cognitive skills—such as reasoning, planning, and
months, with a mean age of 22.8 months (SD
2.5), who were
rule-guided behavior—and it remains unclear whether young
recruited from independent day care centers. Four children did not
children experience difficulty in acquiring and performing the
complete the session because they became distracted (2) or be-
more basic form of goal-directed action assayed by the outcome
cause of interference by other children (2) and were therefore
devaluation procedure.
replaced.
Moreover, there are reasons for believing that infants and young
children might have difficulty selecting an action on the basis of
Apparatus and Stimuli
the current value of the outcome. For example, they perform
The experiment was run on a Sony Vaio laptop computer
reaching responses and other actions that supposedly reflect goal-
(F808K and GRX315MP) connected to a stand-alone flat panel
directedness, problem solving, and means– end planning, such as
Taxan CV600 LCD monitor with an effective display area of
pulling a cloth to retrieve an out-of-reach toy, even when there is
21.5
16 cm and a screen resolution of 640
480, which was
no goal, that is, no toy to be retrieved (e.g., Smith, Thelen, Titzer,
equipped with a Microtouch capacitive touch-screen system. Two
& McLin, 1999; Willatts, 1999). Young children also have the
external computer speakers (HK 195) connected to the laptop were
tendency to reach perseveratively in object retrieval tasks and
situated behind the touch screen. The software used for controlling
frequently repeat a learned response even though the action con-
stimulus presentation and recording of responses was written and
tinues to be ineffective in obtaining a desirable outcome (e.g., Fox,
compiled using Microsoft Visual Basic Professional 6.0. Two 9
Kagan, & Weiskopf, 1979; Gratch & Landers, 1971; Schutte &
7.5-cm icons of a red and green butterfly on a white background
Spencer, 2002; Spencer, Smith, & Thelen, 2001).
were displayed on the screen, one on the left side and the other on
The purpose of the present series of experiments was therefore
the right side. These icons acted as targets for the touch response.
to determine whether instrumental actions performed by young
Eight short video clips from a children’s cartoon were used as the
children are sensitive to devaluation of the outcome. Experiment 1
outcomes. During pretraining, four 10-s clips were used, whereas
demonstrated that children are sensitive to the instrumental con-
four different 24-s clips constituted the outcomes during acquisi-
tingency between the target response (touching an icon on a touch
tion and reversal. All clips were in color and included music and
sensitive screen) and the outcome (the presentation of a short video
other sounds but had no explicit verbal content.
clip). Experiments 2 and 3 then investigated whether devaluing the
outcome by specific satiety had an effect on the subsequent in-
Procedure
strumental behavior of children ranging between 18 and 48 months
of age. Finally, Experiment 4 confirmed that the sensitivity to
In this and all of the following experiments, the children were
outcome devaluation was mediated by the instrumental contin-
tested while seated at a table within easy reach of the touch-
gency between action and outcome rather than being due to the
sensitive monitor in a familiar room in their regular day nursery.
predictive significance of the icons to which the responses were
They were introduced to the apparatus and the investigator by a
directed.
member of the nursery staff well-known to the child. The child’s

---

OUTCOME DEVALUATION IN CHILDREN
41
carer and the investigator then sat next to the child at the table for
the duration of the session. In some cases, the child sat on the
carer’s lap for part or all of the session.
Pretraining.
On pretraining trials, only one icon was displayed
in alternation on either the right or left side of the screen. Touching
the butterfly picture always caused the picture to disappear and
triggered a 10-s video presentation. After observing the experi-
menter complete two demonstration trials, one with the left icon
and the other with the right icon, the child completed two practice
trials that were identical to the demonstration trials. The order of
icon presentation (i.e., left icon first vs. right icon first) and icon
color were counterbalanced across participants.
Training and reversal.
At the start of each acquisition trial,
both butterfly icons were presented, thus providing the child with
a choice of two possible responses. The actual response– outcome
contingency assignments were such that only one of the responses
was rewarded with a video outcome, whereas the other response
Figure 1.
Mean discrimination ratios during two-trial blocks of acquisi-
produced no outcome. The first correct touch response on the
tion and after reversal. Error bars represent standard error.
activated icon therefore always completed a trial. For half of the
children, touching the left icon was rewarded, whereas the remain-
ing children needed to touch the right icon to obtain a video
In summary, children of the youngest age used in the present
reward. As soon as the rewarded response had been performed, the
series of experiments showed an increasing and significant ten-
butterfly pictures disappeared and a 24-s video clip was shown.
dency to perform the action that caused the video outcome to
Once a video clip ended, the video display disappeared, the but-
occur. Moreover, when the action– outcome assignments were
terfly pictures were shown again, and the next trial began. After
changed, the children reversed their responding accordingly. This
the eight acquisition trials had been completed, the response–
experiment has therefore shown that video outcomes are effective
outcome contingency was reversed for a further eight trials.
rewards for children aged between approximately 1 and 2 years
and that their touch responses are sensitive to the instrumental
response–reward contingencies. After establishing these necessary
Results and Discussion
prerequisites, the next experiment was designed to examine
whether these instrumental actions are sensitive to outcome deval-
Acquisition and reversal performance was assessed by a dis-
uation.
crimination ratio in the form of the reciprocal of the total number
of responses per trial. Because the first correct response always
Experiment 2
terminated a trial, a ratio of 1 represents perfect discrimination,
with lower values indicating poorer discrimination. The ratios
In the second experiment, children were initially trained to touch
were analyzed by a mixed analysis of variance (ANOVA), with the
both colored butterfly icons. Touching one butterfly produced
between-children response variable differentiating performance
video clips from one cartoon series as the outcome, whereas
when right or left responses were correct during acquisition. The
touching the other icon produced clips from a second cartoon
within-child variables of phase and trial distinguished performance
series. Following this instrumental training, one of the video
during acquisition and reversal and on the different trials of each
outcomes was devalued by presenting it repeatedly to reduce its
phase, respectively. The reliability of the effects in this and all
incentive value by inducing satiety for this video. To assess the
subsequent analyses was assessed against a Type I error rate of .05.
effectiveness of this devaluation treatment, we gave half of the
The analysis revealed no significant effects of the response
children a reacquisition test in which they were given the oppor-
variable, and therefore we collapsed the discrimination ratios
tunity to obtain further video outcomes under conditions identical
across this variable for presentation. As Figure 1 illustrates, the
to training. If the incentive or goal value of the video outcome had
relative performance of the correct, rewarded response increased
been reduced by the devaluation procedure, the children should
over trials, both during acquisition and following reversal of the
have responded less for this devalued video than for the other
original response– outcome relationships on Trial 9. This descrip-
outcome, which should have retained its incentive value.
tion was confirmed by the statistical analyses. The significant main
The critical condition to assess whether children’s instrumental
effect of trial, F(1, 7)
3.82, MSE
0.08, did not interact with
responses were goal-directed was the postdevaluation extinction
phase, F(7, 42)
1.09, p
.386, MSE
0.01. Separate analyses
test experienced by the remaining children. During the extinction
for linear trend using the coefficients
7,
5,
3,
1, 1, 3, 5, 7
test, children were also given the opportunity to perform the two
revealed a significant linear trend for response reciprocals both
instrumental responses on the same display that was used during
during acquisition, F(1, 42)
4.41, MSE
0.49, and reversal,
training. In this condition, however, no outcomes were presented
F(1, 42)
4.16, MSE
0.12, thereby demonstrating that there
to ensure that children’s performance reflected the knowledge that
was a significant increase over trials in children’s tendency to
had been acquired during instrumental training. If performance
perform the correct rewarded response relative to the incorrect,
was mediated by knowledge about the specific response– outcome
unrewarded response.
relationships experienced during training, and if children were able

---

42
KLOSSEK, RUSSELL, AND DICKINSON
to integrate this knowledge with the information concerning the
Instrumental training.
During the 9-min instrumental training
changes in outcome value, they should have performed the re-
phase, the two butterfly icons were shown side by side in the center
sponse trained with the devalued outcome less than the response
of the touch screen display. Initially, the investigator directed the
that had been trained with the still valued outcome.
child’s attention to the screen display and then demonstrated each
To assess whether sensitivity to outcome devaluation varied
touch response once, thereby triggering the appropriate video presen-
with age, we recruited the children into three 10-month age bands
tation. Thereafter, the child was encouraged to touch the pictures and
between 16 and 48 months.
received verbal encouragement and praise for doing so. Every touch
on one of the pictures always triggered a clip from the same video
Method
series so that one response was always followed by a clip from one of
the two cartoons and the other response by a clip from the other
Participants
cartoon. The assignment of the cartoons to the responses was coun-
terbalanced across children within each group. Simultaneous re-
Seventy-two children aged between 17.7 and 46.7 months were
sponses on both pictures did not trigger a video, and children were
allocated to the extinction condition (18 girls, 18 boys) and the
only allowed to use one hand at a time to touch the pictures. Succes-
reacquisition condition (17 girls, 19 boys). Within each condition,
sive touch responses on the same icon triggered the three successive
participants were divided into three 10-month age bands, each
clips from the appropriate cartoon in a fixed order, regardless of
comprising approximately equal numbers of boys and girls (see
whether clips from the other cartoon were produced in between.
Table 1): 16 to 26.9 months (Age Group 1), 27 to 36.9 months
Outcome devaluation.
Following training, the butterfly icons
(Age Group 2), and 37 to 48 months (Age Group 3). An additional
disappeared and the three 24-s video clips from one of the two
43 children (10 in Age Group 1, 17 in Age Group 2, and 16 in Age
cartoons were repeated four times, each with an interval of 3 s
Group 3) did not complete a full session and were replaced. The
between clips. For half the children, these clips were from one
reasons were the following: interference by other children, the
cartoon, with the remaining children being shown the clips from
child’s carer, or other events (17); technical problems or experi-
the other cartoon. Because response– outcome assignments and
menter error (13); spontaneously abandoning the task because of
icon location were counterbalanced across participants, this also
distraction (11); failure to acquire the required touch responses (2).
meant that for half of the children in each group, it was the video
outcome associated with left responses that was devalued, and for
Apparatus and Stimuli
the remaining half the video outcome associated with right re-
The touch-screen apparatus and butterfly icons were the same as
sponses was devalued.
those used in Experiment 1. Three 24-s video clips from each of two
Extinction and reacquisition tests.
During the tests, the chil-
different and highly discriminable children’s cartoons were used as
dren again had the same opportunity to touch the red and green
outcomes. All clips were in color and included music and sounds
butterfly icons as during instrumental training. For children in the
produced by the characters, but had no explicit verbal content.
extinction test, none of their responses triggered any video pre-
sentations for a period of 2 min. Thereafter, the icons were acti-
Procedure
vated again for a final 4-min period, which was not part of the test
but ended the session on a positive note by allowing the children
All except 2 of the children were tested at their regular day
once again to produce the videos.
nursery in the same way as described in Experiment 1. The other
For the children in the reacquisition condition, touching the
2 children completed the task at our departmental testing facility
butterfly icons triggered video presentations in the same way as
accompanied by their mothers. Unless stated otherwise, the gen-
during training. To compensate for the time taken up by the video
eral procedure was the same as in Experiment 1.
presentations, we designed the instrumental test in the reacquisi-
tion condition to last 6 min.
Results and Discussion
Table 1
Age Profile of the Children (in Months)
Instrumental Training
The number of responses per minute was calculated for the time
Variable
n
M
SD
Range
during which the children had opportunity to respond to the
Experiment 2
butterfly icons. Because the variance of the response rates in-
Extinction condition
creased with the mean, a square-root transformation was applied to
Age Group 1
12
22.8
2.4
17.7–26.0
these rates in this and all subsequent analyses before they were
Age Group 2
12
32.4
2.8
28.2–36.9
evaluated by a mixed ANOVA. The between-children variables
Age Group 3
12
41.1
3.5
37.5–46.7
Reacquisition condition
were age and condition, which distinguished between performance
Age Group 1
12
22.8
2.6
18.5–25.9
of the children in the extinction and reacquisition conditions. The
Age Group 2
12
32.1
2.5
28.1–35.7
within-child variable of devaluation contrasted performance of the
Age Group 3
12
41.4
2.5
38.7–45.9
action trained with the video outcome that was subsequently
Experiment 3
Age Group 1
24
22.5
2.6
16.2–26.3
devalued with performance of the action trained with the outcome
Experiment 4
that was still valued at the time of testing.
Age Group 2
16
31.0
2.2
27.2–34.4
As Table 2 illustrates, the mean rate of responding increased
Age Group 3
16
40.1
2.2
36.1–43.9
systematically with age, F(2, 60)
14.0, MSE
0.81. More

---

OUTCOME DEVALUATION IN CHILDREN
43
Table 2
Mean Responses per Minute (Square-Root Transformed) During Instrumental Training
To-be-devalued response
Valued response
Variable
M
SD
(M SD)
M
SD
(M SD)
Experiment 2
Extinction condition
Age Group 1
4.6
3.5
(2.0 0.8)
6.4
5.0
(2.4 1.0)
Age Group 2
8.0
7.2
(2.6 1.2)
8.4
5.0
(2.8 0.8)
Age Group 3
11.1
5.0
(3.3 0.7)
9.4
5.3
(3.0 0.8)
Reacquisition condition
Age Group 1
5.0
3.3
(2.1 0.7)
3.7
3.2
(1.8 0.7)
Age Group 2
7.6
4.2
(2.7 0.8)
6.7
4.2
(2.5 0.8)
Age Group 3
9.9
3.3
(3.1 0.5)
10.3
5.7
(3.1 0.8)
Experiment 3
Age Group 1
4.6
4.0
(2.0 0.8)
4.5
4.0
(2.0 0.8)
Experiment 4
Age Group 2
3.6
1.9
(1.8 0.5)
3.5
1.9
(1.8 0.5)
Age Group 3
3.3
2.3
(1.7 0.6)
2.8
1.2
(1.6 0.4)
importantly, however, there was no evidence that the response
represents the number of valued responses as a ratio of the total
rates for the actions trained with the “to-be-devalued” and valued
number of responses (r
.36, p
.033).
outcomes differed reliably during training. There was no signifi-
cant main effect of devaluation, nor did this variable enter into any
Reacquisition Test
significant interactions (Fs
1).
The pattern of responding during the test in the reacquisition
condition shows that the absence of a devaluation effect in the
Extinction Test
extinction test for the youngest children was not due to a failure of
the devaluation treatment to reduce the incentive value of the
Because the response rates during instrumental training varied
exposed video clips. As the bottom panel of Figure 2 and Table 3
as a function of age, the rates during the tests were expressed as a
percentage of the absolute training rate to minimize the contribu-
tion of between-children variance. In the extinction condition,
omission of the video outcome on test produced rapid extinction of
responding, and some children failed to respond at all in the second
minute of the extinction test. Therefore, test rates are based on
responding during the first minute of the extinction test. The top
panel of Figure 2, which presents responding in the extinction test
as a percentage of the training rate, shows that a substantial
devaluation effect was observed in the two older groups in that
these children performed the response trained with the now deval-
ued outcome at a lower relative rate than the response trained with
the valued outcome. By contrast, there was no evidence that the
youngest children were sensitive to outcome devaluation, as the
relative rates at which they performed valued and devalued re-
sponses were not markedly different.
This description was confirmed by statistical analysis. Because
the variance of the percentage responding during the tests in-
creased with the mean (see Figure 2), this measure was square-root
transformed prior to analysis in this and the subsequent analyses
(see Table 3 for the transformed data). There was a significant
Age
Devaluation interaction, F(1, 30)
4.8, MSE
27.1, and
analyses of the simple main effects showed that the children
performed the valued response more than the devalued one in Age
Group 2, F(1, 11)
12.0, MSE
25.4, and Age Group 3, F(1,
11)
19.5, MSE
25.5, but not in Age Group 1 (F
1).
Figure 2.
Mean percentage responding by the different age groups during
Furthermore, evidence that the magnitude of the devaluation effect
the test in the extinction condition (top panel) and reacquisition condition
varied with chronological age comes from the significant positive
(bottom panel) for the responses trained with the valued and devalued
correlation between age and discrimination ratio, where the latter
outcomes. Error bars represent standard error.

---

44
KLOSSEK, RUSSELL, AND DICKINSON
Table 3
Experiment 3
Mean of the Square-Root Transformed Percentage Responding
During the Tests
In a series of studies, Rovee-Collier and colleagues (for a
review, see Rovee-Collier, 1997) have shown that a failure of
Devalued
Valued
instrumental performance by young infants does not always reflect
response
response
an absence of learning because the presentation of a reminder cue
in the form of stimuli from the training context could reinstate
Variable
M
SD
M
SD
performance. The presence of the reminder cue aided retrieval of
Experiment 2
the training memory and the deployment of this memory in the
Extinction condition
control of responding. Therefore, it is possible that the youngest
Age Group 1
9.4
5.2
9.6
6.3
children experienced difficulty in retrieving information about the
Age Group 2
6.5
5.4
13.8
7.5
Age Group 3
6.1
3.1
15.3
6.8
outcome devaluation experience during the extinction test. The
Reacquisition condition
fact that the youngest children showed a substantial devaluation
Age Group 1
9.9
3.9
17.7
10.6
effect in the reacquisition condition suggests that actually present-
Age Group 2
8.1
3.2
13.5
6.0
ing the video outcome during the test may have aided retrieval of
Age Group 3
4.7
2.8
10.6
6.3
information of the relative values of the two outcomes, thereby
Experiment 3
Age Group 1
9.3
6.4
10.2
7.6
allowing the youngest children to manifest differential responding
Experiment 4
in accordance with outcome value. Therefore, we examined the
Age Group 2
6.9
4.4
9.9
4.7
effect of providing children in the youngest age group with a
Age Group 3
6.6
4.7
11.4
4.7
reminder cue in the form of a still picture from either the devalued
or valued video during the extinction test in the hope that this cue
would facilitate retrieval of information about the relative values
of the two outcomes. If the absence of a devaluation effect in
illustrate, when the video outcomes were presented contingent on
Experiment 2 was due to the fact that the youngest children had
responding during reacquisition, children of all ages performed the
difficulty in retrieving this information, then the children should
response that produced the devalued video less often than the one
have been more likely to perform the devalued response less than
which produced the valued video, F(1, 30)
22.9, MSE
30.6.
the valued one in an extinction test conducted in the presence of a
Moreover, the fact that the F ratio for the interaction between age
reminder cue.
and devaluation was less than 1 indicates that the magnitude of the
devaluation effect did not vary with age. There was also a signif-
icant effect of devaluation in each age group, Fs(1, 11)
4.9, and
Method
in contrast to the extinction condition, there was no significant
correlation between age and relative performance of the valued
Participants, Apparatus, and Stimuli
response during the reacquisition test (r
.21, p
.22).
Twenty-four children (13 boys and 11 girls) matching the age
In summary, the instrumental responses of children aged be-
profile of the youngest age group in Experiment 2 (see Table 1)
tween 2 and 4 years appeared to be goal-directed. The significant
were trained and tested with the same apparatus, stimuli, and video
devaluation effect observed in Age Groups 2 and 3 demonstrated
outcomes as in Experiment 2. An additional 16 children did not
that these children could integrate instrumental knowledge ac-
complete a full session and were replaced for the following rea-
quired during training with information about a subsequent change
sons: abandoning the task spontaneously (5); fussing or crying (4);
in the value of one of the outcomes and adapt their behavior
equipment malfunction or experimenter error (3); interference by
appropriately.
other children, carers, or other events (2); failure to perform the
Unexpectedly, however, the present results did not provide
required touch responses during instrumental training (2).
evidence for goal-directed action in the 18- to 27-month-olds, as
the children in this age group responded equally for valued and
devalued outcomes in the extinction test. The performance of the
Procedure
youngest reacquisition group confirmed that the absence of a
The instrumental training and outcome devaluation procedures
significant devaluation effect in the extinction condition was not
were the same as those used in Experiment 2. The children were
due to a failure of youngest children to discriminate between the
trained to touch the two butterfly icons in order to be presented
different types of video outcome, nor a result of the ineffectiveness
with different video outcomes, before the video that acted as the
of the devaluation procedure. When allowed to earn video out-
outcome for one of the responses was devalued by repeated expo-
comes directly in the reacquisition test, the youngest children did
sure. The children then received an extinction test that was also
show a significant preference for the response whose outcome had
identical to that in Experiment 2, except for the presence of a
not been devalued, indicating that the repeated presentations of the
reminder cue. This 5
7-cm cue was a still picture taken from one
relevant video series prior to the test had been successful in
of the video cartoons and located above the two butterfly pictures
reducing the incentive or goal value of the video outcome for this
in the center of the screen. For half of the children, the cue was
age group. Experiment 3 examined a possible reason why our
from the devalued video cartoon; the remaining half was presented
devaluation procedure may have failed to detect the goal-directed
with a cue from the valued cartoon. Touching the cue picture
nature of the actions of the youngest children.
during the extinction test had no programmed consequences.

---

OUTCOME DEVALUATION IN CHILDREN
45
Results and Discussion
touched the valued reminder cue without explicit training or in-
struction demonstrates that attractive visual stimuli can directly
As the response rates during instrumental training did not differ
elicit touch responses. At issue, therefore, is whether the children
significantly for the children presented with the valued and deval-
touched the butterfly icons because they learned that these re-
ued reminder cues during extinction testing (F
1), the rates were
sponses produced valued outcomes or because the icons them-
collapsed for presentations. As Table 2 illustrates, these rates were
selves became attractive stimuli through their association with the
very similar to those produced by the youngest children in the
video outcomes and, as a result, became capable of directly elic-
extinction and reacquisition conditions of Experiment 2, and in-
iting the response. In other words, the question is whether icon
deed, the F ratio for the contrast between the response rates of
touching was an instrumental response generated by the specific
children in the present experiment versus the youngest groups in
action– outcome contingency or an elicited response generated by
Experiment 2 was less than 1. As in Experiment 2, the two
the predictive relationship between the butterfly icons and their
responses were performed at the same rate during training (F
1).
associated outcomes. By our criteria for goal-directedness (Dick-
During the extinction test, the presence of the reminder cues in
inson, 1985, 1989; Dickinson & Balleine, 1993), only in the
the present experiment did not induce a devaluation effect. Neither
former case would we characterize the touch response as goal-
the main effect of the reminder cue valence on the square-root
directed.
transform of the percentage responding relative to the training
The fact that touch responding by the older children was sensi-
rates (F
1) nor the Cue Valence
Devaluation interaction, F(1,
tive to outcome devaluation does not decide the issue because
20)
1.1, p
.31, MSE
37.7, was significant, so the test
there is extensive evidence that a cue associated with an intrinsi-
performance was collapsed across the valence of the reminder cue
cally attractive event also loses its attractiveness and hence its
for presentation. The mean percentage response rates for the val-
capacity to elicit approach responses when the associated event is
ued and devalued responses were 125.8 (SE
35.4) and 158.8
devalued (e.g., Colwill & Motzkin, 1994). To determine whether a
(SE
39.6), respectively; which, importantly, did not differ sig-
behavior is under instrumental control, one must show that it is
nificantly following the square-root transformation (F
1; see
mediated by the relationship between the action and its specific
Table 3).
outcome rather than by an association between the outcome and a
The absence of a reliable devaluation effect was not due to the
discrete stimulus, such as the butterfly icon. To this end, the
failure of the valence of the reminder cues to transfer to the
relationships between the stimuli and outcomes need to be held
extinction test. During this test, the children touched not only the
constant, so only the specific action under investigation is uniquely
target butterfly icons but also the reminder cues and did so more
related to the outcome that it produces. This condition is instanti-
frequently in the case of the valued cue compared with the deval-
ated in the so-called bidirectional paradigm (Grindley, 1932),
ued cue, t(1, 22)
2.17. The mean number of contacts with the
which is illustrated by the modification of our procedure used in
devalued cue was 1.0 (SE
0.5), but 3.5 (SE
1.0) with the
Experiment 4.
valued cue. Seven of the 12 children presented with the devalued
During instrumental training, the children were presented with a
cue did not touch it, whereas all but 1 of the 12 children touched
single icon in the center of the screen that they first had to touch
the valued cue at some point during the test.
and then drag, either to the left or to the right, in order to produce
Given these data, we think that it is unlikely that absence of a
video outcomes. Dragging the icon to the left produced video clips
devaluation effect for the youngest children was due to a failure to
from one cartoon and dragging it to the right produced clips from
retrieve information about the relative values of the two outcomes
the other cartoon. Although the initial touch response directed at
during the test. Rather, it is more likely that the process controlling
the icon could have been elicited by the conditioned attractiveness
their touch responding was insensitive to representations of the
of the icon through its association with the outcomes, the contin-
current values of the outcomes. As mediation by a representation
gency between each drag response and its video outcome was
of outcome value is one of our criteria for goal-directedness (see
entirely arbitrary. Consequently, the direction of the drag response
the introduction), responding by the youngest children was not
could not have been mediated by a stimulus– outcome association
goal-directed by this criterion. We consider the nature of this
and must have reflected a sensitivity to the instrumental contin-
process in the General Discussion. However, the differential re-
gency. At issue, therefore, is whether these actions were sensitive
sponding to the valued and devalued reminder cues not only
to outcome devaluation when tested in extinction. As only the
strengthens the interpretation of the insensitivity of the youngest
older children showed an outcome devaluation effect in Experi-
children to outcome devaluation but also raises interpretative is-
ment 2, we assessed the effect of the devaluation on the bidirec-
sues for the outcome devaluation effect observed in the older
tional drag response in children in the two older age bands.
children.
Method
Experiment 4
Participants
The key finding of Experiment 2 was the significant devaluation
effect observed in the two older age groups, which suggests that
Thirty-two children (16 girls and 16 boys) aged between 27 and
the children of approximately 32 months and above were capable
48 months were divided into two 10-month age bands correspond-
of goal-directed action that was based on knowledge of the instru-
ing to those for Age Groups 2 and 3 in Experiment 2 (see Table 1).
mental action– outcome contingency. Strictly speaking, however,
An additional 17 children in Age Group 2 and 13 children in Age
the procedure employed in Experiment 2 does not warrant this
Group 3 did not complete a full session and were replaced for the
conclusion. The fact that children in Experiment 3 spontaneously
following reasons: interference by others or external events (8),

---

46
KLOSSEK, RUSSELL, AND DICKINSON
failure to acquire the required instrumental actions (6), failure to
devaluation was assessed after each response had earned no less
complete the minimum number of responses during instrumental
than seven outcomes but not more than nine.
training (6), equipment malfunction and experimenter error (5),
Outcome devaluation.
The outcome devaluation was similar
abandoning the task because of distraction or fussing (5).
to that used in Experiment 2. One set of three 12-s video clips was
repeated five times with an interclip interval of 3 s. The devalued
video outcome was counterbalanced with respect to the response to
Procedure
which it was assigned, left or right, and the cartoon from which it
came.
Unless otherwise stated, the apparatus, stimuli, video outcomes,
Extinction test.
Following outcome devaluation, the children
and general procedure were the same as those used in Experiment
were free to perform both actions for 2 min. However, neither
2. In the present experiment, the instrumental actions required
response was followed by a video outcome. After the extinction
moving a small butterfly-shaped icon in different directions across
test, the responses once again generated their respective outcomes
the display. The display used during both training and test showed
for 2 min.
a small, 2
2-cm butterfly picture in the center of the screen,
which served as a response manipulandum for the bidirectional
Results and Discussion
action. When this square was touched, a smaller, red butterfly
shape appeared under the child’s finger that could then be dragged
Instrumental performance was assessed during the period of
across the touch-sensitive display as long as contact with the
instrumental training, in which the children were free to make both
screen was maintained. If the butterfly icon was dragged far
responses. As Table 2 shows, prior to devaluation there was no
enough onto one of the two invisible trigger areas (each represent-
difference in the rate at which children performed the two actions.
ing a vertical 5
16-cm strip) on the far left and far right of the
Following square-root transformation of the response rates, the F
display and then released, a video clip was presented. If the icon
ratios for the effect of devaluation and the interaction of this
was dragged and released anywhere else on the screen, it disap-
variable with age were both less than 1. Age also did not signifi-
peared and had to be picked up again from the center, and no video
cantly affect responding, F(1, 28)
1.78, MSE
0.17, p
.19.
clip was shown. However, drags to the left and right that fell just
As Figure 3 and Table 3 illustrate, the children showed a
short of the trigger areas and thus did not produce a video outcome
devaluation effect in the extinction test by performing the response
were recorded as an instrumental response if they fell within the
trained with the valued outcome relatively more than those trained
immediately adjacent area, which was of the same height as and
with the devalued outcome, F(1, 28)
7.86, MSE
29.6. How-
half the width of each trigger area.
ever, we can only be confident about the reliability of this deval-
Action– outcome assignments and type of video devalued were
uation effect for Age Group 3. Although there was no reliable
counterbalanced across children in each age group but were always
interaction between age and devaluation (F
1), planned com-
consistent for individual children so that, for example, a left drag
parisons revealed that the effect of devaluation was significant for
would always produce clips from one cartoon, whereas a drag to
the older children, F(1, 14)
6.5, MSE
27.1, but not for those
the right would start a video clip from the other cartoon.
in the younger Age Group 2, F(1, 28)
2.14, MSE
31.98,
Pretraining.
To train the children to perform the required
p
.17.
instrumental actions, the experimenter demonstrated and explained
The significant overall devaluation effect, which did not interact
the movement sequence to the child on the first two practice trials.
with age, is consistent with the claim that the children’s instru-
Thereafter, the child completed three practice trials with each
mental performance was primarily mediated by knowledge of the
action. If necessary, the experimenter demonstrated the icon
pick-up and icon drag sequence again during these trials. During
pretraining, only two short, 8-s clips, one from each series, were
used as outcomes.
Instrumental training.
After pretraining, three new 12-s clips
from each cartoon were used as instrumental outcomes. Initially,
each response was trained separately. Half of the children were
first trained to perform drags to the left, whereas the remaining half
performed drags to the right first. In each case, one half of the
screen was blacked out, so that only one of the two responses could
be performed until five video outcomes had been obtained by
dragging the icon in the relevant direction. After both responses
had received this separate training, the whole screen was shown,
and the child was free to make responses in either direction until
at least four video outcomes from each series had been triggered or
4 min had elapsed, depending on which criterion was met first. In
the latter case, to proceed to the next stage, a child had to produce
at least two outcomes for each response to ensure that she or he
had experienced both action– outcome contingencies. Including
Figure 3.
Mean percentage responding by the different age groups during
the five outcomes earned by each response during the single
the test for the responses trained with the valued and devalued outcomes.
response training, this criterion ensured that the effect of outcome
Error bars represent standard error.

---

OUTCOME DEVALUATION IN CHILDREN
47
action– outcome contingencies. Even when both actions were di-
the presence of activated representations of both outcomes so that
rected toward a common response manipulandum, children over 3
devaluation of either of the outcomes should have had a compa-
years of age were able to encode a particular outcome in relation
rable effect on both responses.
to a representation of the specific action that had produced it
This problem does not arise in version of two-process theory
during training. Although the apparent devaluation effect in the
espoused by Pavlov (1932) and his students (Asratyan, 1974;
test performance of children with a mean age of 31 months leads
Gormezano & Tait, 1976), who assumed the O 3 R association
to the same conclusion for the younger children, the fact that the
arises directly from the pairing of each response with its outcome.
effect was not statistically reliable for Age Group 2 in the absence
As a consequence, each association is both outcome and response
of an interaction with age makes the results difficult to interpret. If
specific. A devaluation effect with the bidirectional response is
action– outcome learning did take place during training in both age
also not problematic for the associative– cybernetic model of goal-
groups, this learning was apparently less effective in controlling
directed behavior (Dickinson, 1994; Sutton & Barto, 1981;
instrumental performance in the younger compared with the older
Thorndike, 1931). According to this theory, the sight of the but-
group.
terfly icon causes the child to think of the two available responses,
a left drag and a right drag, which would each in turn retrieve a
General Discussion
thought of the associated outcome through the learned R 3 O
associations. If the outcome is valuable, the theory assumes that
The main finding from this series of experiments is that children
the activation of the outcome representation feeds back on the
over 3 years of age are capable of goal-directed action as assessed
response representation or thought to cause the child to perform it.
by the outcome devaluation paradigm. In the absence of the
Therefore, this theory assumes that a goal-directed action is me-
outcomes during the extinction tests, these children performed
diated through an S 3 R 3 O associative chain, in which the
both stimulus-directed responses (Experiment 2) and arbitrary
activation of the response by the stimulus is not sufficient to
bidirectional actions (Experiment 4) at a reduced level if the
generate overt responding without excitatory feedback from an
training outcome had been previously devalued. To do so, at least
activated and positively evaluated outcome representation. When
in the case of the bidirectional action, they must have encoded the
expressed more colloquially, the R 3 O association enables the
specific contingency between action and outcome during training
child to evaluate covertly the consequences of each action before
before integrating this information with a representation of the
deciding which one to perform.
current value of the outcome. This capacity meets the criteria for
Cognitive theories of goal-directed behavior also come in a
goal-directed action offered by Dickinson and colleagues (Dick-
variety of forms. There are a gamut of action theories (Greve,
inson, 1985, 1989; Dickinson & Balleine, 1993).
2001), which assume that human goal-directed behavior is medi-
There are two classes of psychological accounts of goal-directed
ated by explicit, propositional-like representations of instrumental
behavior, the associative and the cognitive, both of which come in
contingencies, whereas causal model theory argues that agents
a variety of forms. Associative two-process theories assume that
construct models of the causal efficacy of their actions (Waldmann
the pairing of the stimulus (S) with the outcome (O) establishes a
& Walker, 2005). The scope of these cognitive theories is not
stimulus– outcome association (S 3 O) through a Pavlovian learn-
restricted to human action and, in one form or another, they have
ing process. The second, instrumental learning process then gen-
also been applied to goal-directed behavior of nonhuman animals
erates an outcome–response association (O 3 R) through experi-
(Blaisdell, Sawa, Leising, & Waldmann, 2006; Dickinson, 1980;
ence with the response– outcome contingency, thereby allowing
Heyes & Dickinson, 1990). It remains an important research issue
performance to be controlled by an S 3 O 3 R associative chain.
whether the capacity of young children for goal-directed action
On the basis of the assumption that the devaluation treatment
depends on associative or cognitive processes.
generally decreases the excitability of the outcome representation,
In contrast to the oldest children, the sensitivity of the two
this model predicts reduced responding following devaluation.
younger age groups to outcome devaluation was less conclusive.
When expressed in terms of folk psychology, the sight of a
Although the same overall pattern of results was obtained in the
butterfly icon made the children think of the associated outcome,
27- to 36-month-olds, planned subgroup analyses in Experiment 4
which in turn made them think of the response that had produced
failed to yield a significant devaluation effect in this group when
that outcome. If outcome devaluation decreased the likelihood that
a bidirectional response procedure was used. The rationale for
the children were thinking of the devalued outcome, they would
using a bidirectional assay was to establish the mediation of
have been less likely to perform the devalued response. Whether or
responding by a representation of the action– outcome relationship
not two-process theory can explain the devaluation effects ob-
rather than a stimulus– outcome association. Given that the deval-
served in the present experiments, and especially Experiment 4,
uation effect was highly reliable for Age Group 2 in Experiment 2,
depends on the nature of the instrumental learning process that
which used a procedure that may have engaged stimulus– outcome
generates the O 3 R link of the chain. According to Trapold and
learning, one possible interpretation is that the period between 2
Overmier (1972), this link arises from the reinforcement of an
and 3 years of age brings about a transition in behavioral control
association between the activated representation of the outcome
from stimulus– outcome learning to fully intentional goal-directed
that is excited by the stimulus and the response through the classic
action.
stimulus–response reinforcement mechanism (Hull, 1943;
For the youngest children, on the other hand, we could find no
Thorndike, 1911). However, the demonstration of a devaluation
evidence that their responding was sensitive to outcome devalua-
effect with the bidirectional paradigm in Experiment 4 is problem-
tion. Only when their responses actually produced the outcomes,
atic for this account. As the single visual icon was equally paired
as in the reacquisition test in Experiment 2, did they respond less
with both outcomes, each response would have been reinforced in
for the devalued outcome than for the valued one, which estab-

---

48
KLOSSEK, RUSSELL, AND DICKINSON
lished that the devaluation treatment was effective for the 18- to
perform the instrumental actions on test, their knowledge about the
27-month-olds. Because the presentation of the outcomes during
current incentive value of the outcomes did not seem to impact the
the reacquisition test may have reminded these children of the
processes responsible for the control of responding. As already
different values of the outcomes, we presented reminder cues for
noted, such resistance to outcome devaluation has often been
the video outcomes during the extinction test in Experiment 3 to
observed in nonhuman animals following instrumental training
parallel the conditions in the reacquisition test. However, the
even in the presence of the devalued outcome, as in Experiment 3
results indicated that the absence of a devaluation effect in the
(Dickinson, Nicholas, & Adams, 1983), and especially following
youngest group was not due to a failure of the outcome devaluation
overtraining (e.g., Adams, 1982; Dickinson et al., 1995; Holland,
to transfer to the test of performance in extinction, as devalued and
2004). On the basis of these findings, Dickinson and colleagues
valued responses were performed at equivalent rates even when
(Dickinson, 1985, 1989; Dickinson et al., 1995) have argued that
children’s responses to the reminder cues concurrently registered
instrumental training engages concurrently two different learning
the differential value of the outcomes.
systems, one underlying goal-directed action and the other medi-
Although our procedure in Experiment 3 therefore showed
ating stimulus–response learning, which does not encode the iden-
effective transfer of current outcome value, it is possible that
tity of the outcome. In human adults, the operation of this latter,
presenting the reminder cues throughout the instrumental test may
habitual mechanism is clearly revealed when we make a slip-of-
not have been the most effective way to aid retrieval of children’s
action because a particular stimulus elicits a well-trained but
training memories. Effective reminder treatments have involved a
unintended response (Reason, 1992). Therefore, according to this
pre-cuing procedure (e.g., Spear & Parsons, 1976), using either a
theory, the goal-directed and habitual systems are in competition
salient cue from the original training context as in the present study
for the control of behavior, with the particular training history and
(e.g., Rovee-Collier, Sullivan, Enright, Lucas, & Fagen, 1980), or
test conditions favoring one system over the other.
a brief period of retraining administered in the original training
The distinction between the goal-directed and habitual systems
context (e.g., Campbell & Jaynes, 1966; see also Adler, Wilk, &
has recently been validated in animal studies by their dissociation
Rovee- Collier, 2000).
in the rat prefrontal cortex (Balleine & Dickinson, 1998a; Corbit &
It is also possible that the youngest children’s insensitivity to
Balleine, 2003; Coutureau & Killcross, 2003; Killcross & Coutu-
outcome devaluation reflected the fact that they did not receive
reau, 2003; Ostlund & Balleine, 2005) and dorsal (Yin, Knowlton,
sufficient training to learn about the action– outcome contingen-
& Balleine, 2004, 2005) and ventral striatum (Corbit, Muir, &
cies. Although the duration of instrumental training was identical
Balleine, 2001). Critically, the acquisition of goal-directed action
in all three age groups, the older children in Experiment 2 re-
depends on the integrity of the prelimbic area of the rat prefrontal
sponded at a faster rate and therefore, on average, experienced a
cortex (Ostlund & Balleine, 2005), which is thought to be homol-
greater number of actions that were followed by an outcome (Age
ogous to dorsal prefrontal structures in the primate brain (Preuss,
Group 2 M
18.8, SE
0.6; Age Group 3 M
19.6, SE
0.3)
1995; Rushworth, Walton, Kennerley, & Bannermann, 2004).
than the youngest children (Age Group 1 M
16.9, SE
0.9).
Moreover, sensitivity to outcome devaluation is impaired by le-
Although the differences between these group means is not large,
sions of the orbital frontal cortex in monkeys (Baxter, Parker,
younger children may simply require relatively more training than
Lindner, Izquierdo, & Murray, 2000; Izquierdo, Suda, & Murray,
older children for reliable action– outcome learning. As a result,
2004).
the total number of responses made during training in the youngest
This analysis raises the possibility that the actions of the young-
group may still have failed to provide the critical level of exposure
est children were mediated by the habitual system, which accords
to the instrumental contingencies necessary for eff