The role of representation in experience-based choice

Judgment and Decision Making, Vol. 4, No. 7, December 2009, pp. 518–529
The role of representation in experience-based choice
Adrian R. Camilleri? and Ben R. Newell
School of Psychology, University of New South Wales, Sydney, Australia
Abstract
Recently it has been observed that different choices can be made about structurally identical risky decisions de-
pending on whether information about outcomes and their probabilities is learned by description or from experience.
Current evidence is equivocal with respect to whether this choice “gap” is entirely an artefact of biased samples. The
current experiment investigates whether a representational bias exists at the point of encoding by examining choice in
light of decision makers’ mental representations of the alternatives, measured with both verbal and nonverbal judgment
probes. We found that, when estimates were gauged by the nonverbal probe, participants presented with information
in description format (as opposed to experience) had a greater tendency to overestimate rare events and underestimate
common events. The choice gap, however, remained even when accounting for this judgment distortion and the effects
of sampling bias. Indeed, participants’ estimation of the outcome distribution did not mediate their subsequent choice.
It appears that experience-based choices may derive from a process that does not explicitly use probability information.
Keywords: decision making, decision from experience, judgment, description-experience gap, representation, uncer-
tainty, probability.
1 Introduction
were completely speci?ed in the form: “Choose between
(A) $3 for certain, or (B) $4 with a probability of 80%,
In recent years a quickly growing literature has emerged
otherwise zero”. Participants playing this description-
contrasting two different formats of choice — descrip-
based choice task tended to make decisions consistent
tion and experience — and the correspondence of deci-
with prospect theory’s four-fold pattern of choice —
sions observed in each (Rakow & Newell, 2010). A de-
risk-aversion for gains and risk-seeking for losses when
cision from experience (DfE) is one where the possible
probabilities were moderate or high, but risk-seeking for
outcomes and estimates of their probabilities are learned
gains and risk-aversion for losses when probabilities were
through integration of personal observation and feedback
small (Kahneman & Tversky, 1979). For example, 64%
from the environment (Hertwig & Pleskac, 2008). A typ-
of participants preferred the certain $3 in the decision
ical example might be the decision from where to buy
above.
your morning coffee as you make your way to work. By
In the experience format, participants were initially un-
contrast, a decision from description (DfD) is one where
aware of the outcomes and their respective probabilities
all possible outcomes and their probabilities are explic-
and had to learn this information by sampling from two
itly laid out from the outset (Hertwig & Pleskac, 2008).
unlabelled buttons. Each sample presented a randomly
A typical example might be the decision to bring an um-
selected outcome taken from an underlying outcome dis-
brella to work after hearing the morning weather forecast
tribution with the same structure as the problems pre-
and the chance of precipitation.
sented in the description format. Participants were free
Surprisingly, recent evidence has found that the deci-
to sample as often and in any order that they liked un-
sions made under these two different formats of choice
til they were ready to select one option to play from for
diverge. For example, Hertwig, Barron, Weber and Erev
real. Strikingly, participants playing this experienced-
(2004) presented six binary, risky choice problems to par-
based choice task tended to make decisions opposite to
ticipants in either described or experienced format. In
the four-fold pattern of choice. For example, only 12% of
the description format, outcomes and their probabilities
participants preferred the certain $3 in the decision above.
This apparent Description-Experience “gap” led some to
?Address for correspondence: Adrian R. Camilleri, School of Psy-
chology, University of New South Wales, Sydney, 2052, Australia.
call for the development of separate and distinct theo-
Email: acamilleri@psy.unsw.edu.au. This research was sup-
ries of risky choice (Hertwig et al., 2004; Weber, Sha?r,
ported by an Australian Postgraduate Scholarship and a UNSW Re-
& Blais, 2004). Fox and Hadar (2006), however, have
search Excellence Award to the ?rst author, and an Australian Research
argued that this conclusion is unwarranted in light of a
Council Discovery Project Grant (DP 0770292) to the second author.
The authors thank Jonathan Baron, Daniel Gottlieb, Robin Hau and
reanalysis of the Hertwig et al. data. Speci?cally, they
Kevin Bird for valuable help and advice.
found that prospect theory could satisfactorily account for
518

---

Judgment and Decision Making, Vol. 4, No. 7, December 2009
Representation in experience-based choice
519
Information
Mental
Choice
Observed
Presentation
Representation
Mechanism
Decision


Figure 1: A simple decision-making framework. Black chevrons represent external, observable events. Grey chevrons


represent internal, mental events.
the patterns of choice when based on participants’ expe-
ing compared to pineapples. Thus, this account is pri-
rienced distribution of outcomes, which, due to sampling
marily concerned with the level of information acquisi-
“errors”, was often different to the objective distribution
tion and the major prediction is that the gap should disap-
from which the sampled outcomes derived.
pear when information acquired in both the DfD and DfE
The crux of the debate centres on the relative impor-
paradigms are equivalent
tance of sampling bias. This issue has led investigators
In contrast, according to the psychological account, the
to employ a number of creative designs that have pro-
gap is something over and above mere sampling bias:
duced con?icting results (e.g., Camilleri & Newell, in
it re?ects different cognitive architecture at the level of
prep.; Hadar & Fox, 2009; Hau, Pleskac, & Hertwig,
choice. Description- and experience-based choices re-
2010; Hau, Pleskac, Kiefer, & Hertwig, 2008; Rakow,
cruit different evaluative processes that operate according
Demes, & Newell, 2008; Ungemach, Chater, & Stewart,
to different procedures. Thus, this account is primarily
2009). The purpose of this paper is to re-examine these
concerned with the level of choice and the major predic-
discrepancies in light of how choice options are repre-
tion is that the gap will remain even when information
sented in the mind of the decision maker.
acquired in both the DfD and DfE paradigms is equiva-
lent.
1.1 A framework for understanding the
A number of methodologies have been used to account
description-experience gap
for sampling bias and therefore provide a test between
the statistical and psychological accounts. Sampling bias
Figure 1 presents a simple framework of the steps in-
has been eliminated by yoking described problems to ex-
volved in making a decision, which is based on the two-
perienced samples (Rakow et al., 2008), conditionalis-
stage model of choice (Fox & Tversky, 1998). At the
ing on the subset of data where the objective and experi-
stage of information acquisition, the decision-maker at-
enced outcome distributions match (Camilleri & Newell,
tempts to formulate a mental representation or impression
in prep.), and obliging participants to take representative
of the outcome distributions for each alternative1. The
samples (Hau et al., 2008; Ungemach et al., 2009). The
two modes of information acquisition we are presently
?rst two of these studies found that elimination of sam-
concerned with are description and experience.
pling bias all but closed the gap. In contrast, the last two
There are two primary accounts for the Description-
of these studies found that even after accounting for sam-
Experience gap. According to the statistical, or informa-
pling bias there nevertheless remained a choice gap (see
tion asymmetry, account, the gap re?ects a population-
Hertwig & Erev, 2009, and Rakow & Newell, 2010, for
sample difference due to sampling bias inherent to the
good overviews). This mixed evidence has ensured that a
sequential-sampling, experience-based choice paradigm
level of controversy persists.
(Hadar & Fox, 2009). Speci?cally, the information ac-
quired, or utilised, by decision-makers through their sam-
pling efforts is not equal to the underlying outcome dis-
1.2 The stage of mental representations
tributions from which the samples derive. As a result
One way to reconcile these con?icting sets of observa-
of these unrepresentative samples, the experience-based
tions is to reconsider the framework presented in Figure
decision maker’s understanding of the outcome distribu-
1. The current methodologies accounting for sampling
tion is quantitatively different from the description-based
bias all attempt to equate information presented at the
decision maker’s understanding of the outcome distribu-
stage of information acquisition. That is, they all work
tion. The fact that a Description-Experience gap occurs
to ensure that decision makers have been exposed to the
is therefore relatively trivial because the gambles that
same information. There are two reasons for suspecting
decision-makers are subjectively (as opposed to objec-
that the information participants are exposed to may be
tively) choosing between are different. Apples are be-
unequal to the information participants actually use to
1Not all choice frameworks require the formation of mental repre-
make their decisions. First, it is not clear that participants
sentations (e.g., Busemeyer & Townsend, 1993).
construct representations of outcome distributions from

---

Judgment and Decision Making, Vol. 4, No. 7, December 2009
Representation in experience-based choice
520
all of the information they are exposed to. In the free sam-
conclude that the source of the gap is independent of dis-
pling paradigms, for example, participants may utilise a
torted representations of the outcome distributions; in-
two-step sampling strategy in which they begin by obtain-
stead, it must be due to sampling bias and/or inherent
ing a general overview of the outcomes of each alternative
to the choice mechanism processes. This conclusion is
(e.g., the magnitudes) before moving on to a more formal
perhaps premature for two reasons. First, there are con-
investigation of the probability of each outcome occur-
cerns regarding the methodology used to measure the ver-
ring. Partial support for this claim comes from obser-
bal representations. In the Hau et al. (2008) study 2, for
vations of recency, whereby the second half of sampled
example, participants were aware that, at least after the
outcomes, as opposed to ?rst half, better predicts choice
?rst problem, they would have to make relative frequency
(Hertwig et al., 2004; but see Hau et al., 2008). In the
judgments. It is possible that participants’ sampling ef-
forced sampling paradigm, moreover, it seems doubtful
forts were then at least partially driven by their attempt
that participants take into account, and linearly weight,
to accurately learn the contingencies, and crucially, rep-
information from up to 100 samples when forming a rep-
resent these contingencies in a verbal format. Ungemach
resentation due to memory and/or attentional limitations
et al. (2009) avoided this issue by presenting the judg-
(Kareev, 1995; 2000). Indeed, we suspect such limita-
ment probe as a surprise. However, the probe comprised
tions are responsible for the meagre amount of sampling
simply of participants stating how frequently the rare out-
typically observed in free sampling designs (e.g., a me-
come had been observed. This task is therefore quite dis-
dian of 15 samples in Hertwig et al., 2004).
tinct from participants appreciating the probability of the
Second, we know that when reasoning about uncer-
rare event being observed on the next sample, which, at
tainty, mathematically equivalent (external) representa-
the very least, additionally involves appreciation of the
tions of probabilities are not necessarily computation-
number of samples taken.
ally equivalent (Gigerenzer & Hoffrage, 1995; Hau et al.,
Second, there are concerns regarding the validity of
2010). For example, “80%” is mathematically equiva-
the verbal judgment probe in the context of experienced-
lent to “8 out of 10”, yet these two pieces of information
based choice. In the DfE task, the decision maker’s
can be used in non-equivalent computational ways, lead-
only goal is to decide which of the options is “better”.
ing to different decisions (see also the ratio bias effect;
Presumably, decision makers could use a “satis?cing”
Bonner & Newell, 2008). Importantly then, it should not
heuristic and attempt to make this decision with mini-
be assumed that what people are given (i.e., information
mal computational effort (Simon, 1990; Todd & Gigeren-
contained in a description or aggregated from experience)
zer, 2000). Therefore, in terms of mental representa-
is identical to what people take away. Viewing this point
tions, the minimalist requirement in this task is to form
within the framework presented in Figure 1 implies that
some sort of impression as to which option is “better”,
mathematically equivalent contingency descriptions and
irrespective of the magnitude of that superiority or the
experienced contingencies could nevertheless be repre-
speci?c probabilities of each outcome. Therefore, in the
sented differently depending on whether the information
experienced-based choice task, there is no inherent need
is acquired by description or experience. If true, the pos-
to formulate a propositional statement about the proba-
sibility then exists that even when sampling bias is objec-
bility of each outcome (as is presented in the description-
tively eliminated, there may still remain subjective differ-
based choice task). Given evidence that humans possess
ences in mental representations actually operated upon.
a nonverbal numerical representation system (Dehaene,
And of course, it is these actually operated upon mental
Dehaene-Lambertz, & Cohen, 1998), a nonverbal assess-
representations that we are most interested in.
ment probe may be better able to capture the summary
A small number of studies have attempted to examine
impression because it makes no reference to explicitly de-
these mental representations (Barron & Yechiam, 2009;
scribed verbal probabilities.
Hau et al., 2008; Ungemach et al., 2009). For example,
Pursuing this logic, Gottlieb, Weiss and Chapman
Ungemach et al. (2009) asked participants to verbally re-
(2007) used both a verbal and a nonverbal assessment
port the frequency of rare event occurrences. Similarly,
tool to probe decision makers’ mental representation of
Hau et al. (2008) asked participants to verbally estimate
outcome distributions in DfD and DfE (forced sampling)
the relative frequency (as either percentages or natural
paradigms. The verbal probe asked participants to com-
frequencies) of each outcome. The results of these stud-
plete the sentence “__% of cards were worth __ points”.
ies are consistent and suggest that people are largely ac-
The nonverbal probe consisted of a large grid composed
curate and, if anything, overestimate small probabilities
of 1600 squares whose density could be adjusted by
and underestimate large probabilities. The direction of
pressing on the up and down arrow keys of a normal
these estimation errors would actually have the effect of
keyboard. Participants were asked to adjust the density
reducing the size of the gap.
of the grid to match their belief as to the relative fre-
Based on this evidence, one might feel con?dent to
quency of each option. Interestingly, there was a disparity

---

Judgment and Decision Making, Vol. 4, No. 7, December 2009
Representation in experience-based choice
521
in judgment accuracy depending on whether judgments
were probed verbally or nonverbally. Similar to past stud-
ies, when probed verbally, participants’ judgment accu-
racy was best modelled by a linear function with fairly
good accuracy regardless of mode of information acquisi-
tion. In contrast, when probed nonverbally, participants’
judgment accuracy was best modelled by a second-order
polynomial implying underestimation of large probabil-
ities and overestimation of small probabilities. Impor-
tantly, there was an interaction suggesting that this dis-
tortion from perfect mapping was much stronger in the
description than in the experience condition.
Two details are particularly intriguing about these ?nd-
ings. First, the second-order polynomial curves obtained
with the nonverbal judgment probe were strikingly rem-
iniscent of the probability-weighting function described
by Prospect Theory (PT; Kahneman & Tversky, 1979).
If PT is taken as a process model of choice, then the

weighting function re?ects the mental adjustment that de-
cision makers apply to their calculation of expected utility
Figure 2: Screenshot of a default grid. The value in the
for each option. However, these ?ndings suggest that an
box corresponds to the outcome value provided by the
alternative explanation is that probability information is
participant.
distorted at the level of mental representation, and that
this distortion may be observed only with a nonverbal
2 Method
judgment probe. Second, accuracy when probed non-
verbally was worse for the description condition than in
the experience condition. This difference is surprising
2.1 Participants
because adjusting a grid’s density to that of an explicit,
The participants were 80 undergraduate ?rst year Uni-
known proportion would seem an easier task than ad-
versity of New South Wales psychology students (48 fe-
justing to an imprecise, non-speci?ed proportion gleaned
males), with an average age of 19.5 years and a range of
from sequential sampling. The difference potentially im-
18 to 36 years. Participation was in exchange for course
plicates judgment distortions as contributing to the gap
credit, plus payment contingent upon choices.
and, moreover, leads to suspicion that nonverbal probes
of mental representations may be a more sensitive form of
mental representation assessment for experienced-based
2.2 Materials
choice tasks
Choice problems. The eight choice problems used are
shown in ?rst three columns of Table 1. Each problem
1.3 The current experiment
consisted of two options: an option that probabilistically
paid out one of two values versus an alternative option
Primary explanations for the Description-Experience
that always paid out a single value. The expected value
choice gap have been statistical (the result of sample bias)
was always higher for the probabilistic option. The prob-
and psychological (the result of a weighting bias at the
lems were chosen to evenly split between the domains
time of choice). The current study examined whether
of gain and loss, and also to span a range of probabilis-
the gap could also be a representational phenomenon,
tic rarity (5%, 10%, 15%, and 20%). The option pre-
that is, the result of a distortion at the time of encoding.
dicted by Prospect Theory to be preferred was labelled
The speci?c aims of the current experiment were to test
the “favoured” option and the alternative option was la-
whether there exists a representational bias and whether,
belled the “non-favoured” option (Kahneman & Tversky,
when controlling for sampling and any representational
1979). Speci?cally, the favoured option was the option
bias, there remains a choice gap. To examine these ob-
containing the rare event when the rare event was desir-
jectives we employed the free-sampling, money machine
able (e.g., 14 is a desirable rare event in the option 14
paradigm (Hertwig et al., 2004) in combination with both
[.15] and 0 [.85]), or the alternative option when the rare
a verbal and nonverbal probe to assess participants’ judg-
event was undesirable (e.g., 0 is an undesirable rare event
ments of the outcome distributions (Gottlieb et al., 2007).
in the option 4 [.8] and 0 [.2]).

---

Judgment and Decision Making, Vol. 4, No. 7, December 2009
Representation in experience-based choice
522
Table 1: Percentage choosing the option predicted by Prospect Theory (Kahneman & Tversky, 1979) to be favoured.
Problem
Option
Percentage selecting the favoured option
number
Favoured
Non-favoured
Description
Experience
Gap
1
3 (1.0)
4 (.80)
68
54
14
2
?2 (1.0)
?50 (.05)
55
41
14
3
14 (1.0)
17 (.90)
71
42
29*
4
?3 (1.0)
?32 (.10)
47
49
?2
5
14 (.15)
1 (1.0)
57
49
8
6
?12 (.85)
?9 (1.0)
42
42
0
7
25 (.20)
4 (1.0)
51
33
18
8
?9 (.95)
?8 (1.0)
64
31
33*
Note: ? indicates signi?cant difference between description and experience conditions.
Decision task. The decision task was the free sam-
In both decision conditions, when the participants were
pling “money machine” paradigm, similar to the one em-
ready to make a one-shot decision, they pressed on a
ployed by Hertwig et al. (2004). In the description-based
“Play Gamble” button that allowed them to select the ma-
choice condition, two alternative money machines were
chine they preferred to play from. In all cases allocation
presented on screen. Each machine was labelled with a
of safe and risky options to the left and right machines
description of how that machine allocated points. All of
was counterbalanced and the order of the problems was
the safe option machines were labelled in the form “100%
randomised for each participant.
chance of x”, where x represents the outcome. All of
the risky option machines were labelled in the form “y%
Judgment probes Both the verbal and nonverbal judg-
chance of x, else nothing”, where y represents the prob-
ment probes asked participants to ?rst enter the number,
abilistic chance of a non-zero outcome, and x represents
and speci?c value, of each outcome paid out by each
the outcome.
machine. Contingent on this response, participants were
In the experience-based choice condition, the two al-
then subsequently asked to provide a probability estimate
ternative money machines were also presented on screen,
for each identi?ed outcome. Thus, participants were not
but they were labelled only with the letters “A” and “B”,
asked to make an estimate for an outcome they had not
respectively. Each of the machines was associated with
seen, and some participants did not make an estimate for
a distribution of possible outcomes in accordance with
an outcome they had seen (because they had not identi-
the objective probabilities as shown in Table 1. Samples
?ed this outcome initially).
from each machine were non-random draws from the re-
The verbal judgment probe asked participants to com-
spective outcome distributions that were selected by an
plete the sentence: “x is paid out by the machine __
algorithm to maximally match the objective probability
percent of the time”, where “x” refers to the outcome.
with the participants’ experienced distribution, thereby
In contrast, the nonverbal judgment probe presented a
minimising sampling variability.2
grid made up of 40x40 small squares, each containing
the number “x”, along with the instructions: “Adjust the
2On each sample, the participants’ experienced distribution was
frequency of x’s in the grid to match the frequency of x
compared to the objective distribution and the outcome that minimised
this difference was presented. This algorithm produced repeating pat-
paid out by the machine. You can adjust the density of
terns of outcomes. For example, when the objective probability was
the grid by pressing ‘up’ and/or ‘down’ on the keyboard
20%, the pattern of outcomes repeated itself in blocks of 5 outcomes.
until x ?lls the grid according to its frequency”. The de-
A typical approach to exploring the money machines in our data, based
on the median values, was to sample from the risky option seven times,
fault grid showed 50% of the squares, randomly dispersed
sample from the safe option twelve times, and then sample from the
(Figure 2). Each press of the key increased or decreased
risky option eight times before making a ?nal choice. Thus, the typi-
the frequency of squares by 1%, randomly over the grid.
cal sequence of outcomes for a participant playing problem 1 would be
For the purposes of analysis, the visual display was con-
something like 4, 4, 0, 4, 4, 4, 4, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 0, 4, 4,
4, 4, 0, 4, 4. It thus seems unlikely that participants in the current study
verted into a percentage after the participant made his or
were able to identify the repeating pattern.
her judgment.

---

Judgment and Decision Making, Vol. 4, No. 7, December 2009
Representation in experience-based choice
523
2.3 Design
scription vs. experience) and both judgment probe types
(percentage vs. grid).4 Inspection of the ?gure suggests
The experiment was a 2 x 2 x 2 within-subjects design
that there is an interaction between presentation mode
and counterbalanced such that participants completed one
and judgment probe type. Speci?cally, it appears that the
of the eight problems in each of the eight experimen-
verbal percentage probe produced better calibrated judg-
tal cells. The three binary independent variables were
ments for those in the Description condition (i.e., esti-
presentation mode (description or experience), judgment
mates closer to the identity line), whereas the non-verbal
probe type (percentage or grid), and judgment probe time
grid probe produced better calibrated judgments for those
(before or after choice). The two dependent variables
in the Experience condition.
were the choice made (favoured or non-favoured option)
We tested this interaction using a mixed model (using
and the accuracy of judged outcome probabilities (mea-
the lmer function of R [Bates & Maechler, 2009; R De-
sured as the average absolute difference between experi-
velopment Core Team, 2008], as described by Baayen,
enced3 and judged probabilities).
Davidson, & Bates, 2008, and Bates, 2005). This func-
tion is robust when designs are unbalanced, as is the case
2.4 Procedure
here as a result of omitted data. The dependent variable
An on-screen video tutorial explained that the experiment
was a measure of judgment error: the absolute value of
was about making decisions between different alterna-
the difference between, on the one hand, the experienced
tives, that the objective of the game was to maximise the
probability of the common event, and, on the other, the
amount of points won, and that at the end of the experi-
normalized judged probability of the common event (i.e.,
ment points would be converted into real money accord-
the judged probability of the common event divided by
ing to the conversion rate of 10 points = AUD$1. The tu-
the sum of that and the judged probability of the rare
torial combined written instructions with movements of a
event — the two often did not add to 100). The main
ghost player to demonstrate how to play the description-
predictors were presentation mode, judgment probe type,
and experience-based decision tasks and correctly answer
and their interaction. Problem number (as a nominal vari-
the verbal and nonverbal judgment probes. Participants
able or factor) was also included as a ?xed effect; it ac-
were informed that they could sample from each option as
counted for signi?cant variance, but judgment probe time
often and in any order that they liked. Thus, participants
(before vs. after choice) was excluded because it was
could take samples ranging in size from one to many hun-
never signi?cant in any analysis. Participant identity was
dreds. Instructions for the grid probe were: “You will see
included as a random effect. The interaction was signif-
small versions of the target value randomly superimposed
icant at p = .0042 (as assessed by Markov Chain Monte
on a square grid. You should adjust the density of the tar-
Carlo sampling). Thus, the magnitude of the difference
get value on the grid to match the frequency of the target
between participants’ experienced probabilities and their
value paid out by the machine.” In order to reduce poten-
judged probabilities varied depending on whether the in-
tial wealth effects, no feedback was given of the points
formation was acquired by description or experience. Ex-
that participants were awarded for their one-shot choice
amination of the ?tted mean errors revealed that partici-
for each problem.
pants in the Description conditions were relatively more
At the completion of the experiment a screen revealed
accurate with the percentage probe than the grid probe
the participant’s total points earned, as well as their corre-
(M = 0.98 vs. 6.64, respectively) compared to participants
sponding real money conversion. Participants that ended
in the Experience conditions (M = 3.22 vs. 5.70, respec-
up with negative point scores were treated as though
tively). Further inspection of the two bottom panels of
they had scored zero points. Finally, participants were
Figure 3 suggests that there is a difference in the slopes
thanked, debriefed, and then paid.
of the regression lines between the Description and Ex-
perience conditions.
In order to make this directional inference, we re-
3 Results
gressed an error term (common event judged probability
— common event experienced probability) on presenta-
3.1 Judgment
tion mode (description vs. experience) for cases where
the nonverbal grid judgment probe was used. After re-
Figure 3 plots judged probabilities against experienced
probabilities separately for both presentation modes (de-
4We collapsed across judgment probe time (before vs. after choice)
because this manipulation had no effect. Eighty-one trials (12.6%)
3In the description condition, the “experienced” probabilities were
were excluded because estimates were unreasonable (the average ab-
the objective probabilities. In the experience condition, the “experi-
solute difference between experienced and judged probabilities was 40
enced” probabilities depended on what outcomes had actually observed.
or higher) or the participant failed to make an estimate.

---

Judgment and Decision Making, Vol. 4, No. 7, December 2009
Representation in experience-based choice
524
Experienced percentage
Description
Experience
100
80
60
40
ercentage probe
P
20
y = 0.9941x + 0.4022
y = 0.989x + 0.5368
R2 = 0.9937
R2 = 0.9778
0
percentage
100
udged
J
80
60
id probe
Gr
40
20
y = 0.8939x + 1.544
y = 0.951x ? 0.906
R2 = 0.8917
R2 = 0.9088
0
20
40
60
80
100 0
20
40
60
80
100
Figure 3: Experienced percentages plotted against judged percentages as a function of presentation mode (description
on left panels, experience on right panels) and judgment probe type (verbal percentage in upper panels, nonverbal grid
in lower panels). The size of the plotted circles relates the number of identical data points. The solid line depicts the
least-square regression lines describing the relation between the experienced and judged probabilities.
moving one outlier, the interaction was signi?cant at p
for six of the eight problems.5 Two of these differences
= .0291. A similar analysis for cases where the verbal
were signi?cant by individual chi-square tests (p’s < .05).
percentage judgment probe was used was not signi?cant.
Indeed, the odds of selecting the favoured option in the
Thus, the tendency to overestimate rare events and under-
Description condition were more than 1.7 times the odds
estimate common events was much stronger in the De-
of selecting the favoured option in the Experience con-
scription condition, but only when assessed with the non-
dition. Although indicative, and commonly used in the
verbal probe.
literature, this rough analysis fails to properly assess the
role of presentation mode because it ignores the variance
in participants’ experience and judgments.
3.2 Choice
To test the effect of presentation mode on choice, we
The percentage of participants selecting the option pre-
used a logistic mixed model, with participant identity as a
dicted by Prospect Theory to be the favoured choice is
5Although a within-subjects design, the comparisons were all
displayed in Table 1. The difference between Description
between-subjects because participants made only one decision for each
and Experience conditions falls in the expected direction
problem in either the description or experience choice format.

---

Judgment and Decision Making, Vol. 4, No. 7, December 2009
Representation in experience-based choice
525
random effect, and including problem number as a ?xed
cision makers are equally able to nonverbally represent
effect (as before). The dependent variable was whether
a non-explicit, gist impression constructed from sequen-
or not the favoured option was selected. The main pre-
tial sampling and a numerical percentage explicitly pre-
dictors were presentation mode, judgment probe type, ex-
sented. Of course, this is not to say that judgments were
perienced probability and normalized judged probability
particularly accurate: they were not; participants in both
(as used before). Of these predictors, the only signi?cant
groups displayed a tendency to underestimate common
effects were of presentation mode (coef?cient ?.627, z =
events and overestimate rare events. This observation
?3.43, asymptotic p = .0006) and experienced probabil-
replicates Gottlieb et al.’s (2007) intriguing ?nding that
ity (coef?cient ?.071, z = -2.38, p = .0172). The odds
percentages are distorted when transformed into nonver-
of selecting the favoured option in the Description con-
bal estimates. The current study extends this observation
dition were more than 1.8 times the odds of selecting the
to a free sampling design where participants decided the
favoured option in the Experience condition. Importantly,
size of their samples. Admittedly, it is possible that at
the effect of normalized judgment was not signi?cant (z
least some of this bias is due to an anchoring effect at the
= ?.90). Thus, the effect of presentation mode on choice
probe-density starting point (50%). What is perhaps more
is apparently not mediated by its effect on judgment.
interesting, and not explainable in terms of anchoring, is
the fact that the distortion, this tendency to underestimate
In order to show this result graphically, we condition-
common events and overestimate rare events, was much
alised on the subset of data where participants’ experi-
greater for those in the Description conditions than those
enced and judged distributions were approximately equal
in the Experience conditions.
to the objective distribution.6 The subset of data com-
prised of just 28 experience- and 153 description-based
When participants made their judgment using a verbal
probe — entering a number to correspond to the relative
decision trials. Thus, the subset did not equally repre-
probability of each outcome — absolute judgment accu-
sent all participants, problems and conditions, and, hence,
racy was greater in the Description conditions. Contrary
inferential statistics were not conducted. Nevertheless,
to some previous research, there was little evidence that
the retained data do serve to visually represent the ma-
participants overestimated small probabilities and under-
jor ?nding of our regression analysis. Namely, as shown
estimated large probabilities (Barron & Yechiam, 2009;
in Figure 4, even within the subset of data without sam-
Hau et al., 2008). In fact, accuracy in both conditions was
pling or judgment errors, there remains a gap between
fairly high, which replicates some other studies that have
description- and experienced-based choices.
asked for probability judgments (Fox & Hadar, 2006;
Gottlieb et al., 2007; Ungemach et al., 2009), and were
4 Discussion
superior to those achieved by participants making judg-
ments via the nonverbal grid probe.
The greater absolute judgment accuracy observed
4.1 Judgment
when using the verbal probe may lead some to the con-
The current study attempted to take a representational
clusion that this type of probe should be preferred when
perspective in explaining the observation of a gap be-
assessing representations of outcome distributions We
tween description- and experienced-based patterns of
have three cautions. First, accuracy when using the ver-
choice. The ?rst aim was to examine whether there ex-
bal probe in the Description condition depended only on
ists representational bias, that is, an encoding distortion
memory, not judgment, and is therefore in?ated. Sec-
of the outcome distribution prior to choice. To that end,
ond, the nonverbal grid task was, on average, prone to
we asked participants to judge each problem’s outcome
greater variability because of the potential for super- or
distribution using either a verbal or nonverbal probe.
sub-additivity. Speci?cally, because one grid was pre-
When participants made their judgment using a non-
sented for each outcome identi?ed, participants’ summed
verbal probe — adjusting the density of a large grid to
judgments for the outcome probabilities for each option
correspond to the relative probability of each outcome —
often deviated from 100%. Super- and sub-additivity did
absolute judgment accuracy in the Description and Ex-
not occur when using the percentage probe because par-
perience conditions was approximately equivalent. This
ticipants could easily add up their estimates and ensure
result is particularly surprising because it implies that de-
that they totalled 100%. Third, even if decision-makers
can interpret and numerically report the content of their
6Speci?cally, we retained only those trials in which the experienced
mental representations when explicitly probed by a ver-
and (normalised) judged rare event probabilities were both within 10%
bal probe, if this is not the actual representation and in-
of the objective rare event probability. For example, in Problem 1,
formation used to make the decision, then such (albeit
where the objective probability for the rare event is .2, we retained only
those trials where the experienced and judged probability for the rare
accurate) information is non-diagnostic in the pursuit of
event were both between .18 and .22 (i.e., “within 10%” of .2 = ±.02).
understanding experience-based choice.

---

Judgment and Decision Making, Vol. 4, No. 7, December 2009
Representation in experience-based choice
526
100
All Data
Conditionalised Data
n
g
80
t
i
o
t
i
n
p
c
l
e

O
d
60
e
Description
r
e
t

S
u
Experience
n
o
40
e
v
a
r
c
e

F
e
20
P
t
h
0
Figure 4: The percentage of participants selecting the favoured option in the Description and Experience conditions.
The conditionalised data were those trials where the participants’ experienced and (normalised) judged rare event

probabilities were both within 10% of the objective rare event probability (see footnote 6). Error bars indicate the
standard error of the mean.
What then are we to conclude about nonverbal judg-
in judged outcome distributions across presentation mode
ment probes? Despite producing less accurate results
did not mediate the choice gap between description- and
overall, they uniquely discriminate between description-
experience-based choices. Importantly then, the choice
and experience-based formats of information acquisition.
gap appears to be being driven by something over and
Nonverbal judgment probes may therefore permit greater
above both sampling bias and judgment distortions. This
sensitivity to presentation mode when gauging mental
?nding supports the work of those that have obliged par-
representations. Potentially, this is because representa-
ticipants to sample until they have observed outcomes
tions of outcome distributions are themselves nonverbal
matching exactly or nearly exactly the objective outcome
(Dehaene et al., 1998).
distribution (Hau et al., 2008; Jessup, Bishara, & Buse-
meyer, 2008; Ungemach et al., 2009).
4.2 Choice
4.3 Implications
The second aim was to examine whether representational
biases constitute, in addition to sampling bias, a major
How can we explain the remarkable conclusion that par-
cause of the choice gap between description and expe-
ticipants’ own estimate of the outcome distribution does
rience choice formats. As described above, there does
not mediate their subsequent choice? It may be the case
appear to be a representational bias, at least when probed
that choices are made separately from judgment of the
nonverbally, and this bias is stronger when information
outcome distributions. Recently it has been noted that in
is acquired by description. Assuming choices are made
many situations, both inside and the lab and out, people’s
based on these differentially distorted outcome distribu-
choice behaviour is at odds with their judgment (Barron
tions, representational biases may be suf?cient to cause
& Yechiam, 2009). For example, immediately following
subsequent differences in choice.
a suicide bombing, people believe the risk decreases but
To begin, we again found a disparity in the patterns of
at the same time exhibit more cautious behaviour. Thus,
choice made to identical problems depending on whether
choice may not be made using representations of the out-
they were presented by description or experience (Her-
come distributions at all. Decision ?eld theory, for ex-
twig et al., 2004; Weber et al., 2004). The size of the
ample, models choice processes as the gradual change of
gap observed in our data, 14.4 percentage points, is rel-
preference between options and makes no reference to
atively small when compared to previous free sampling
a mental representation of each option’s outcome distri-
DfE paradigm studies (e.g., 36 percentage points in Her-
bution (Busemeyer & Townsend, 1993). This conclusion
twig et al., 2004). This is probably due to the relatively
has implications for the development of models of choice.
large amount of samples taken by our participants (me-
Speci?cally, our results suggest that models that incor-
dian of 28, compared to 15 in Hertwig, et al., 2004)
porate two stages, one at the level of representation and
coupled with our manipulation for sample outcomes to
one at the level of choice, may be unnecessary when it
track the objective probabilities as closely as possible (see
comes to predicting experienced-based choice. For exam-
Footnote 2).
ple, one of the leading two-stage choice models — cumu-
Even after accounting for sampling bias and judgment
lative prospect theory (Fox & Tversky, 1998; Tversky &
distortions, however, the mode by which information was
Fox, 1995) — fares no better at explaining our data when
acquired — by description or from experience — re-
based on judged, compared to experienced, outcome dis-
mained signi?cant. The differential distortions observed
tributions (Appendix). This result echoes the ?nding of

---

Judgment and Decision Making, Vol. 4, No. 7, December 2009
Representation in experience-based choice
527
Hau et al. (footnote 2, 2008). Our conclusion also seems
ear mixed-effects models using S4 classes.
R
to be consistent with the ?ndings from a recent choice
package
version
0.999375–32,
http://CRAN.R-
prediction competition. Whereas all models submitted to
project.org/package=lme4.
predict description-based choices assumed that outcomes
Bonner, C., & Newell, B. R. (2008). How to make a risk
were weighted by probabilities, the majority of models
seem riskier: The ratio bias versus construal level the-
submitted to predict experience-based choices were such
ory. Judgment and Decision Making, 3, 411–416.
that “the concept ‘probability’ did not play an important
Busemeyer, J. R., & Townsend, J. T. (1993). Decision
role” (Erev et al., 2010).
Field Theory: A dynamic cognition approach to deci-
With regard to the two primary choice gap explana-
sion making. Psychological Review, 100, 432–459.
tions — statistical or psychological — the current data
Camilleri, A. R., & Newell, B. R. (in preparation). The
lend support to the latter account. That is, that there ex-
description-experience ’gap’: Psychological or statisti-
ist true differences in the choice mechanics used to make
cal phenomenon? The University of New South Wales.
experience-based decisions that are over and above the
Dehaene, S., Dehaene-Lambertz, G., & Cohen, L. (1998).
effects of biased samples and judgment errors. What else
Abstract representations of numbers in the animal and
could be driving the gap? Hertwig et al. (2004) demon-
human brain. Trends in Neurosciences, 21, 355–361.
strated that recency, the tendency to rely more heavily
Erev, I., Ert, E., Roth, A. E., Haruvy, E., Herzog, S., Hau,
on more recently observed outcomes, was another in?u-
R., Hertwig, R., Stewart, T., & Lebiere, C. (2010). A
ence on experienced-based choice and hence the gap. In
choice prediction competition, for choices from expe-
our data, however, we observed no difference in success
rience and from description. Journal of Behavioral De-
when predicting choice from the mean value of the ?rst
cision Making, 23, 15–47.
versus second half of observed outcomes (56.2% ver-
Fox, C. R., & Hadar, L. (2006). “Decisions from experi-
sus 60.5%, respectively, t(560) = ?1.026, n.s.). Our in-
ence” = sampling error + prospect theory: Reconsider-
terpretation is that the gap derives from a probabilistic
ing Hertwig, Barron, Weber & Erev (2004). Judgment
focus in the description format and a non-probabilistic
and Decision Making, 1, 159–161.
focus in the experience format. Indeed, Rottenstreinch
Fox, C. R., & Tversky, A. (1998). A belief-based account
and Kivetz (2006) argue that non-probabilistic thinking
of decision under uncertainty. Management Science,
is more likely in situations where people partially con-
44, 879–895.
trol events and when there is relatively low salience of
Gigerenzer, G., & Hoffrage, U. (1995). How to im-
probabilistic cues. If Rottenstreinch and Kivetz’s inter-
prove Bayesian reasoning without instruction: Fre-
pretation is correct, the experience format in which prob-
quency formats. Psychological Review, 102, 684–704.
abilities are never explicitly mentioned is more likely to
Gottlieb, D. A., Weiss, T., & Chapman, G. B. (2007).
yield non-probabilistic thinking than the description for-
The format in which uncertainty information is pre-
mat in which probabilities are clearly presented. More-
sented affects decision biases. Psychological Science,
over, evidence from outside the lab also suggests that
18, 240–246.
executives’ decision-making rarely explicitly considers
Hadar, L., & Fox, C. R. (2009). Information asymmetry
outcome probability (Jeske & Werner, 2008). We feel
in decision from description versus decision from ex-
that the distinction between a probabilistic and non-
perience. Judgment and Decision Making, 4, 317–325.
probabilistic focus during choice is an interesting one for
Hau, R., Pleskac, T. J., & Hertwig, R. (2010). Decisions
further research to pursue.
from experience and statistical probabilities: Why they
trigger different choices than a priori probabilities?
Journal of Behavioral Decision Making, 23, 48–68.
5 References
Hau, R., Pleskac, T. J., Kiefer, J., & Hertwig, R. (2008).
The description-experience gap in risky choice: The
Baayen, R. H., Davidson, D. J., & Bates, D. M. (2008).
role of sample size and experienced probabilities.
Mixed-effect modeling with crossed random effects for
Journal of Behavioral Decision Making, 21, 1–26.
subjects and items. Journal of Memory and Language,
Hertwig, R., Barron, G., Weber, E. U., & Erev, I. (2004).
59, 390–412.
Decisions from experience and the effect of rare events
Barron, G., & Yechiam, E. (2009). The coexistence of
in risky choice. Psychological Science, 15, 534–539.
overestimation and underweighting of rare events and
Hertwig, R., & Erev, I. (2009).
The description-
the contingent recency effect. Judgment and Decision
experience gap in risky choice. Trends in Cognitive
Making, 4, 447–460.
Sciences, 13, 517–523.
Bates, D. (2005). Fitting linear mixed models in R. R
Hertwig, R., & Pleskac, T. J. (2008). The game of
news, 5, 27–30. http://journal.r-project.org.
life: How small samples render choice simpler. In N.
Bates, D. & Maechler, M. (2009).
lme4:
Lin-
Charter & M. Oaksford (Eds.), The probabilistic mind:

---