The coexistence of overestimation and underweighting of rare events and the contingent recency effect

Judgment and Decision Making, Vol. 4, No. 6, October 2009, pp. 447–460
The coexistence of overestimation and underweighting of rare
events and the contingent recency effect
Greg Barron?
Eldad Yechiam
Harvard Business School
Technion — Israel Institute of Technology
Abstract
Previous research demonstrates overestimation of rare events in judgment tasks, and underweighting of rare events
in decisions from experience. The current paper presents three laboratory experiments and a ?eld study that explore
this pattern. The results suggest that the overestimation and underweighting pattern can emerge in parallel. Part of
the difference between the two tendencies can be explained as a product of a contingent recency effect: Although the
estimations re?ect negative recency, choice behavior re?ects positive recency. A similar pattern is observed in the ?eld
study: Immediately following an aversive rare-event (i.e., a suicide bombing) people believe the risk decreases (negative
recency) but at the same time exhibit more cautious behavior (positive recency). The rest of the difference is consistent
with two well established mechanisms: judgment error and the use of small samples in choice. Implications for the
two-stage choice model are discussed.
Keywords: decision making, rare events, judgment, choice, learning, terror, probability.
1 Introduction
tant in light of the two-stage choice model (Fox & Tver-
sky, 1998) which assumes that choice can be predicted
Studies of human reaction to low probability (rare) events
from estimated probabilities.2 The main goal of the cur-
reveal an interesting difference between judgment and
rent paper is to improve our understanding of this pattern
decision-making in repeated settings. Judgments (prob-
and its implications.3
ability estimations) appear to re?ect over-sensitivity to
rare events. That is, the estimated probability of events
1.1 The contradicting results
that occur with probability below 0.5 tends to be higher
than the objective probability (see e.g., Erev, Wallsten &
Ample experimental and ?eld evidence suggests that sub-
Budescu, 1994; Zacks & Hasher, 2002; Viscusi, 1992).
jective probability and frequency estimates re?ect over-
On the other hand, decision-making from experience
sensitivity to rare events. In their study on the judged fre-
tends to re?ect underweighting of (insensitivity to) rare
quency of lethal events, Lichtenstein, Slovic, Fischhoff,
events (Barron & Erev, 2003; Hertwig, Barron, Weber
and Combs (1978) observed a consistent overestima-
& Erev, 2004; Weber, Blais, & Sha?r, 2004).1 That is,
tion of the probabilities related to the rarest causes of
decision-makers behave as if events that occur with prob-
death. A similar ?nding is that teens greatly overesti-
ability below 0.5 occur with smaller probability than their
mate the chances of death in the near future; they estimate
objective probability. The apparent discrepancy is impor-
the probability to be 18.6% when the actual probability
is 0.04% (Fischhoff, Parker, Bruine De Bruin, Downs,
?This research was supported by grants from the USA-Israel Bina-
Palmgren, Dawes, & Manski, 2000). Another remarkable
tional Science Foundation and from the Israel Science Foundation. The
example is that when Americans were asked to estimate
authors wish to thank Ido Erev for his signi?cant input and guidance.
the probability that a smoker would develop lung cancer
All remaining errors are our own. Address: Greg Barron, Harvard
Business School, Baker Library 447, 10 Soldiers Field Rd., Boston,
in the future, the mean estimate was 38% whereas the ac-
MA 02163. Email: gbarron@hbs.edu. Eldad Yechiam is at the Max
tual probability is between 6% and 13% (Viscusi, 1992).
Wertheimer Minerva Center for Cognitive Studies Faculty of Industrial
Engineering and Management, Technion — Israel Institute of Technol-
2The model assumes that people ?rst assess the probability of an un-
ogy.
certain event and then transform the assessment using Prospect Theory’s
1Barron and Erev (2003) and Hertwig et al. (2004) clarify the differ-
weighting function. Fox & Hadar (2006) make the optimistic assertion
ence between this behavioral tendency that occurs when people decide
that the two-stage model can account for experience-based decisions.
based on personal experience, and the important tendency to overweight
3The “Choice-Judgement discrepency” in one-shot decisions is an
low probability outcomes in one-shot decisions based on a description
unrelated phenomona that refers to a different pattern of behavior, a
of the possible outcomes (see Tversky & Kahneman, 1992; Fox & Tver-
preference to bet on A rather than on B even though B is judged to be
sky, 1998).
at least as probable as A (Heath & Tversky, 1991).
447

---

Judgment and Decision Making, Vol. 4, No. 6, October 2009
Coexistence of overestimation and underweighting
448
feedback.
Table 1: Four repeated choice problems and aggregated
Although the results of Conditions 1 and 2 can be ac-
proportion of R choices reported in Barron and Erev
counted for by risk aversion in the gain domain and risk
(2003). The notation (v, p) denotes a gamble that pro-
seeking in the loss domain, Conditions 3 and 4 imply the
vides v with probability p and 0 otherwise.
opposite, that decision makers appear to take more risk
Problem
S
R
P(R)
in the gain than in the loss domain. All four results are
consistent with underweighting of small probabilities (of
1
(3, 1)
(32, 0.1)
0.28
receiving 32 in Conditions 1 and 2 and of receiving 0 in
2
(?3, 1)
(?32, 0.1)
0.60
conditions 3 and 4). Further research supports the con-
3
(9, 1)
(10, 0.9)
0.56
jecture that underweighting is, at least in part, the result
4
(?9, 1)
(?10, 0.9)
0.37
of a tendency to rely on small samples when making ex-
perience based decisions (Hertwig et al., 2004; Erev &
Barron, 2005; Erev, Ert, & Yechiam, 2008; Yechiam &
Busemeyer, 2006).4 Hertwig et al. (2004) showed for in-
Interestingly, smokers saw their choice to smoke as be-
stance that subjects’ choices were signi?cantly associated
ing consistent with their risk estimate (i.e., the pleasure
with their most recent outcomes, suggesting a reliance on
is worth the risk), a view consistent with the theory of
only part of the sampled choice outcomes. The tendency
utility maximization.
to rely on small samples is also consistent with existing
Overestimation of rare events in ?eld studies is typ-
research on information search and the perception of vari-
ically explained by invoking the availability heuristic.
ability (Kareev, 2000; Kareev, Arnon, & Horwitz-Zeliger,
Rare events (e.g., unique causes of death) that are more
2002).
salient are easier to retrieve from memory, hence they
Recent studies have attempted to measure both judg-
are overweighted (see Tversky & Kahneman, 1974). The
ment and choice at the same time in the context of rare
phenomenon is robust and is observed in controlled lab-
events (e.g., Fox & Hadar, 2006; Hau, Plescak, Kiefer
oratory experiments even when long-term memory is not
& Hertwig, 2008; Ungemach, Chater, & Stewart, 2009).
likely to play an important role. In one such study, Erev
In contrast to the current paper, these experiments stud-
and Wallsten (1993) had subjects estimate the probability
ied one-shot decisions based on repeatedly drawn sam-
of an icon on the computer screen making its way safely
ples. Overall, they reported evidence for underweighting
to the other side of a continuously opening and closing
in choice while subjects remained well calibrated in their
sliding door. The amount of time that the door remained
estimations, especially for larger samples of outcomes.
open (which determined the probability of success) was
Although statistically insigni?cant, a slight tendency to-
varied, and estimates were elicited based on the entire
wards overestimating small probabilities was also ob-
range of objective probabilities. The results indicated a
served. Because these studies lacked individual level
clear overestimation of small success and failure proba-
analyses, it is dif?cult to know if the subjects who over-
bilities. Erev et al. (1994) showed that a model assuming
estimated rare events were also those who underweighted
that error is added to subjective probabilities can capture
the events in choice.
the overestimation phenomena. Note that this assumption
The current paper’s main contributions are as follows.
is consistent with both the “regression effect” (Stevens &
First, we demonstrate simultaneous overestimation of
Greenbaum, 1966) and the “contraction bias” (Poulton,
probabilities and underweighting in choice at the indi-
1979) that describe shifts in responses towards the mid-
vidual subject level. As noted earlier, Hau et al. (2008)
dle of a range.
and Ungemach et al. (2009) did not demonstrate that in-
A very different effect of rare events was observed
dividual subjects displayed both biases at the same time
in studies of decisions from experience. These studies
and these could in fact be two separate groups of individ-
(e.g., Barron & Erev, 2003; Hertwig et al., 2004; We-
uals within the sample. Secondly, we provide evidence
ber et al., 2004; Erev & Barron, 2005; Yechiam & Buse-
for speci?c underlying mechanisms that can explain, at
meyer, 2006) re?ect underweighting of rare events. Ta-
least in part, the coexistence of overestimation and un-
ble 1 shows four conditions from Barron and Erev (2003)
derweighting within individual subjects. Finally we ex-
where subjects repeatedly chose between two unmarked
tend the ?ndings of Hau et al. (2008) and Ungemach et al.
buttons that provided outcomes sampled from two dis-
(2009) to a different experience-based paradigm, namely
tributions, “S” and “R”. Let (v, p) denote a distribution
4It is still debated whether such reliance constitutes completely ig-
where the outcome v occurs with probability p (other-
noring events that are not in the sample (e.g., Hertwig et al., 2004) or
wise zero). The right hand column shows the aggregated
just decreasing their relative weight in the decision (Yechiam & Buse-
meyer, 2006). The latter view is more consistent with traditional learn-
proportion of R choices over all trials (400 in Conditions
ing models such as those of Bush and Mosteller (1955) and March
1 and 2 and 200 in Conditions 3 and 4) with immediate
(1996).

---

Judgment and Decision Making, Vol. 4, No. 6, October 2009
Coexistence of overestimation and underweighting
449
repeated decisions with immediate feedback. This is dif-
tingent recency effect implies that these results will be
ferent from the sampling paradigm used in these earlier
robust to simultaneous choice and judgment in contexts
studies, where a single decision is made based on a sam-
where, outside of Las Vegas, people do not have precise
ple observed over time (with no monetary implications).
information regarding the dependency of outcomes.
1.2 The coexistence hypothesis
2 Study 1:
The Coemergence
Our interpretation of the results is referred to as the co-
of Overestimation and Under-
existence hypothesis. It assumes that there are qualita-
tive, yet simultaneous, differences in the effect of rare
weighting
events on judgment and decision processes. As noted
above, these differences have been well studied: Rare
To evaluate the three alternative explanations, Study 1 ex-
events are overestimated due to their increased availabil-
amined both judgments and choices in the same context.
ity in memory but are underweighted in choice due to the
Subjects performed a repeated choice task in which one
tendency to rely on small samples in experience based
of the alternatives included a rare (low probability) event
decisions. Although coexistence seems a reasonable hy-
of a negative payoff (loss of points). During the second
pothesis given the prior research, it has not been shown
half of the task they were asked to estimate the probabil-
within subjects in previous research and is inconsistent
ity of this event.
with the two-stage model of choice that predicts a cer-
tain consistency between estimates and choices. Speci?-
2.1 Method
cally, in assuming Prospect Theory’s weighting function,
the two-sage model predicts that choice will re?ect over-
2.1.1 Design
weighting of estimated small probabilities.
Another reason to predict coexistence pertains to re-
In a within-subject design, each subject performed a bi-
cency effects. Barron and Erev (2003) note that the ten-
nary choice task and a probability assessment task. The
dency to underweight rare events in choice can be a prod-
binary choice task was performed under uncertainty for
uct of a positive recency effect: Oversensitivity to recent
100 rounds, with immediate feedback. The probability
outcomes (i.e., one type of small sample as suggested
assessment task following each choice in rounds 51–100.
by Hertwig et al., 2004). This explanation implies that,
Upon completion, subjects performed a one-time retro-
since rare outcomes are less likely to occur recently or
spective probability assessment task.
in any cognitively limited small sample, on average they
In the binary choice task, subjects chose between
will be underweighted in choice. In contrast, judgment
two unmarked buttons presented on the screen (see Ap-
tasks typically produce evidence for negative recency (or
pendix). Each button was associated with one of two dis-
“gamblers fallacy”) in prediction tasks. (See the review
tributions referred to here as S (for safe) and R (for risky).
in Lee, 1971, and recent research by Sundali & Croson,
The S distribution provided a certain loss of 3 points
2006.) The above logic implies that overestimation can
while the R distribution provided a loss of 20 points with
be a product of a negative recency effect in estimation
probability 0.15 and zero otherwise. Thus, the two dis-
tasks. Negative recency (the expectation that the state of
tributions had equal expected value. To reduce noise and
the world will change between sequential trials) implies
sampling error, random sequences of 100 outcomes were
overestimation of the probability of the event that did not
produced repeatedly and the ?rst sequence with an ob-
occur in the last trial.
served probability of 0.15 for the ?20 outcome was used
This prediction is supported by Ayton and Fischer’s
for all subjects. The sequence provided the ?20 outcome
(2004) study. In their experiment subjects repeatedly
on rounds 12, 15, 19, 20, 21, 23, 25, 35, 40, 41, 60, 73, 80,
predicted the outcome of a roulette spin (red or blue
87, and 96. The position of the S and R buttons (right vs.
with equal probability) and indicated their con?dence
left) was randomly determined for each subject. At the
level (from “no con?dence” to “strong con?dence”) in
conclusion of the study, points were converted to mone-
the prediction. Although the results demonstrated neg-
tary payoffs according to the exchange rate: 100 points =
ative recency in the prediction task, simultaneous pos-
1 Shekel (about 18 US cents), and were subtracted from
itive recency was observed for subjects’ con?dence in
the show up fee.
their predictions. That paper concluded that sequences of
In the probability assessment task, performed af-
outcomes re?ecting human performance yield anticipa-
ter each binary choice in trials 51–100, subjects were
tions of positive recency, whereas outcomes due to inani-
prompted to estimate the chances (in terms of a percent-
mate chance mechanisms, such as coins, dice and roulette
age between 0 and 100) of ?20 appearing (on the R but-
wheels, yield anticipations of negative recency. The con-
ton) on the next round.

---

Judgment and Decision Making, Vol. 4, No. 6, October 2009
Coexistence of overestimation and underweighting
450

After completing 100 rounds subjects were asked to es-

Choice of R
Subj. Assessment
Obj. Probability
timate (“end-of-game estimates”), to the best of their rec-

ollection, two conditional probabilities: (1) the chances
1
of ?20 appearing after a previous round with a ?20
0.9
outcome [SP(?20 | ?20)] and (2) the chances of ?20
0.8
appearing after a previous round with a 0 outcome
0.7
[SP(?20 | 0)].
0.6
0.5
2.1.2 Subjects
0.4
0.3
Twenty-four Technion students served as paid subjects in
0.2
the study. Most of the subjects in this and the other stud-
0.1
ies described in this paper were second and third year
0
industrial engineering and economics majors who had
1
2
3
4
5
6
7
8
9
10
taken at least one probability or economics course. In
addition to the performance contingent payoff, described
Figure 1: Mean proportion of R choices in Study 1, mean
above, subjects received 28 Shekels for showing up. The
subjective assessments in trials 51–100, and observed ob-
?nal payoff was approximately 25 Shekels (about $5 US).
jective probability of the rare outcome in 10 blocks of 10
trials. Objective probability is the observed proportion of
trials where the rare outcome was observed, recalculated
2.1.3 Apparatus and procedure
after each trial.
Subjects were informed that they were operating a “com-
puterized money machine” (see a translation of the in-
2.2 Results
structions in the Appendix) but received no prior infor-
mation as to the game’s payoff structure. Their task was
2.2.1 Judgment and choice in the same context
to select one of the “machine’s” two unmarked buttons
The aggregated assessments and proportion of R (risky)
(see the ?gure in the Appendix) in each of the 100 trials.
choices are shown in Figure 1. The mean probability as-
In addition, they were told that they would be asked, at
sessment from trials 51–100, aggregated over trials and
times, to estimate the likelihood of a particular outcome
over subjects, was 0.27. This value is signi?cantly larger
appearing the following round. As noted above, this oc-
than 0.163, the mean running average of the observed
curred in trials 51–100.
probability of the ?20 outcome (t[23] = 3.11, p < 0.01).6
Subjects were aware of the expected length of the study
Thus, the results re?ect overestimation of the rare event.
(10–30 minutes), so they knew that it included many
As shown in Table 2, over all 100 trials, subjects’ ag-
rounds. To avoid an “end of task” effect (e.g., a change
gregate proportion of R choices was 0.74 (signi?cantly
in risk attitude), they were not informed that the study in-
larger than 0.5, t[23] = 7.47, p < 0.001). This result is
cluded exactly 100 trials.5 Payoffs were contingent upon
consistent with the assertion of underweighting of rare
the button chosen; they were produced from the predeter-
events in choice. The rate of R choice over trials 51–100
mined sequence drawn from the distribution associated
was 0.80 (signi?cantly larger than 0.5, t[23] = 6.78, p <
with the selected button, described above. Three types of
0.001).
feedback immediately followed each choice: (1) the pay-
The comparison of the judgment and choice data for
off for the choice, which appeared on the selected button
trials 51–100 supports the “coexistence” hypothesis. The
for the duration of 1 second, (2) payoff for the forgone
results demonstrated different reactions to rare events in
option, which appeared on the button not selected for the
judgment and in choice within the same context.
duration of 1 second and (3) an update of an accumulating
We next asked whether the different reactions occur
payoff counter, which was constantly displayed.
at the level of the individual subject. For 63% (15/24)
of the subjects, assessment and choice results were not
5Not knowing the length of the study also prevents subjects from us-
ing probability-based reasoning (the focus on the likelihood of achiev-
consistent in terms of the implied weighting of the ?20
ing a particular aspiration level) (Lopes, 1996). This type of reasoning
outcome aggregated over trials 51–100. Overestimation
bases choice on the probability of coming out ahead, which is a function
of the number of choices to be made. A second reason for not telling
6Although the rare events’ probability was 0.15, the mean running
subjects the game’s length is that this better approximates the real-world
average of its observed probability can be considerably higher if it oc-
small decisions that interest us. In such situations, the number of future
curs more often early on in the sequence. To see this, consider the sim-
choices to be made is often unknown and it is dif?cult to prescribe op-
ple sequence [1, 0]. While the mean is 0.5, the mean running average is
timal behavior.
(1 + 0.5)/2 = 0.75.

---

Judgment and Decision Making, Vol. 4, No. 6, October 2009
Coexistence of overestimation and underweighting
451
Table 2: A summary of the aggregate results of Study 1. The number in parenthesis denotes the null hypothesis for
the test reported in the text. All t-tests are one-sample tests unless otherwise noted.
Statistic
Trials 1–100
Trials 51–100
Retrospective
P(R): proportion of R choices
0.74(0.5**)
0.80(0.5**)
SP(?20): Mean subjective assessment of the probability
–
0.27(0.163**)
of a ?20 outcome
P(R | ?20): Prop. of R choices after a trial with a ?20
0.56
0.74
outcome
(paired 0.77**)
(paired 0.81?)
P(R | 0): Prop. of R choices after a trial with a 0 outcome
0.77
0.81
SP(?20 | ?20): Mean assessment of the probability of a
0.18
0.08
?20 outcome after a trial with a ?20 outcome
(paired 0.28*)
(paired 0.26**)
SP(?20 | 0): Mean assessment of the probability of a ?20
0.28
0.26
outcome after a trial with a 0 outcome
?p<0.1, *p<0.05, **p<0.01, ***p<0.001.
and underweighting of rare events was found to occur in
show a similar negative recency pattern. The estimated
100% of these 15 cases.
probabilities are 0.08 and 0.26 respectively. Thus, sub-
jects judged the ?20 outcome to be less likely after a
previous ?20 outcome (paired t-test, t[23] = 3.26, p <
2.2.2 The contingent recency effect
0.01).
The central column in Table 2 presents the mean judg-
The contingent recency effect described above cannot
ment and choice over trials 51–100 and presents the re-
by itself explain the observed overestimation and under-
sults conditional on the most recent outcome. Although
weighting in choice. As noted earlier, the mean estima-
the proportion of R choices in trials after an outcome of 0,
tion immediately following a rare event (SP(?20 | ?20)
aggregated for each subject over all 100 trials, was 0.77, it
= 0.18) was lower than the mean estimation following a
dropped signi?cantly to 0.56 for trials after an outcome of
frequent event, but still re?ected overestimation (of the
?20 appeared (paired t-test, t[23] = 5.66, p < 0.01). Sim-
objective probability). And the proportion of R choices
ilar, but slightly weaker evidence of positive recency was
was higher than 0.50 (0.74) even after the ?20 outcome.
observed in trials 51–100 (the same trials analyzed above)
A second relevant observation is the correlation across
with [P(R | 0)] = 0.81 and [P(R | ?20)] = 0.74, (paired t-
subjects between judgment, SP(?20), and choice of R
test, t[23] = 1.83, p = 0.08).
for each trial for which estimations were given (trials 52
In order to evaluate the recency effect on judgment
to 100).7 Computation of this correlation by experimental
we ?rst computed mean conditional subjective probabil-
trial reveals negative correlations in 36 of the 49 trials (p<
ity assessments, SP(?20 | ?20) and SP(?20 | 0), for each
0.001 in a sign-test). Thus, while the results supported the
subject by aggregating separately estimates from rounds
coexistence hypothesis, there remained a consistency be-
after a ?20 outcome, and the estimates from rounds af-
tween judgments and choices, as subjects tended to avoid
ter a 0 outcome. The results (see Table 2) revealed a
option R when they judged the probability of a loss to be
negative recency effect, with subjects judging the ?20
high.
outcome less likely after a previous outcome of ?20
(SP(?20 | ?20) = 0.18 and SP(?20 | 0) = 0.28, paired
t-test, t[23] = 1.99, p < 0.05). This result is interest-
3 Study 2: Generality over payoff
ing considering that the conditional objective probabil-
domain and payoff rule
ity OP(?20 | ?20) was larger than OP(?20 | 0). Note
also that even in trials that occur after an appearance of
Although Study 1’s results are consistent with the con-
the rare outcome, the subjective assessment (0.18) is still
tingent recency effect, an alternative explanation remains
overestimated.
Examination of the retrospective estimation of
7Assessments were elicited starting from trial 51 so that the ?rst
SP(?20 | ?20) and SP(?20 | 0) at the end of 100 rounds
choice following an estimate is on trial 52.

---

Judgment and Decision Making, Vol. 4, No. 6, October 2009
Coexistence of overestimation and underweighting
452

for the ?nding of positive recency for choice. In particu-
1
lar, subjects may place a different value on a certain loss

0.9
immediately following a preceding loss from the risky
option (a de-sensitization effect). In Study 2 we exam-
0.8

ined this possibility by paying subjects according to the
0.7

outcome of a single trial drawn at random at the end of
Choice of R (Gain)
Choice of R (Gain)
the game. By replicating Study 1 in both the gain and
0.6

Choice of R (Loss)
Choice of R (Loss)
loss domains, we also tested the hypothesis that the pref- 0.5
Est. (Gain)
Est. (Gain)
erence for option R in Study 1 might re?ect a tendency
Est. (Loss)
0.4
Est. (Loss)
to avoid alternatives with a larger proportion of losses (as
Obj. Probability
Obj. Probability
was observed in Erev & Barron, 2005).8 Additionally,
0.3
outcomes in Study 2 were randomly drawn from the dis-
0.2
tributions described next (without the pre-selection of a

single series that was employed in Study 1) and the study
0.1
was conducted for 400 trials. The distributions were also

0
changed so as not to include zero, as several studies have
1 3 5 7 9 11 13 15 17 19 21 23 25 27 29 31 33 35 37 39
demonstrated unique behavior related to zero outcomes
or costs (Ariely, Gneezy, & Haruvy 2005; Festinger &
Figure 2: Mean proportion of R choices in Study 2, and
Carlsmith, 1959).
subjective assessments of the rare outcome’s probability
in 40 blocks of 10 trials.
3.1 Method
3.1.1 Design
3.2 Results
The design was the same as for Study 1 with the excep-
3.2.1 Judgment and choice in the same context
tion that outcomes were randomly drawn in real-time, the
study was run for 400 trials (with assessments elicited
The results reveal the same pattern observed in Study 1.
on trials 201–400 and not at the end) and subjects were
Figure 2 presents the subjects’ aggregate proportion of
paid according to one randomly chosen trial. In the Loss
R choices and probability assessments in 40 blocks of
condition the S distribution provided a certain loss of 1.3
10 trials. Across all 400 trials and two conditions, sub-
points while the R distribution provided a loss of 3 points
jects’ mean proportion of R choices was 0.80 (signi?-
with probability 0.15 and a loss of 1 point otherwise.
cantly larger than 0.5, t[39] = 8.92, p < 0.001), consis-
Thus, the two distributions had equal expected value. For
tent with the underweighting of rare events in choice be-
the Gain condition, a constant of 4 was added to all pay-
havior. In trials 201–400 (see Table 3), when probability
offs so that S provided (2.7, 1) and R provided (3, 0.85;
assessments were also elicited, the mean proportion of
1).
R choices was 0.81 (greater than 0.5, t[39] = 7.17, p <
0.001) again consistent with the underweighting of rare
events in choice. Consistent with the visual impression in
3.1.2 Subjects
Figure 2, there was no signi?cant difference between the
Forty Technion students served as paid subjects in the
Gain and Loss conditions (t[38] = 0.21, ns).
study. In addition to the performance contingent pay-
The mean probability assessment from trials 201–400,
off, subjects in the Gain and Loss conditions received 25
aggregated over trials and conditions, was 0.22 (see the
Shekels or 29 Shekels for showing up. The conversion
third row of Table 3 and Figure 2). This is signi?cantly
rate for the one randomly chosen trial was 1 point = 1
larger than 0.15, the objective probability of the rare out-
Shekel. The ?nal average payoff was approximately 27
come (t[39] = 3.35, p < 0.01). This result is consistent
Shekels (about 5 US dollars).
with an overestimation of rare events in probability as-
sessments. There was no signi?cant difference in the
3.1.3 Apparatus and procedure
probability assessments between the Gain and Loss con-
ditions (t[38] = 1.15, ns).
The task and instructions were as in Study 1 except that
At the individual level, for 55% (22/40) of the subjects
the subjects were told that they would be paid according
assessment and choice results were inconsistent in terms
to one randomly sampled trial at the end of the experi-
of the implied weighting of the rare outcome (1 in the
ment.
Gain condition and ?3 in the Loss condition) aggregated
8Loss Aversion, as quanti?ed by Prospect Theory, would not imply
over trials 201–400. As can be seen in Table 4, overes-
a preference for R since both S and R are in the loss domain.
timation and underweighting of rare events was found to

---

Judgment and Decision Making, Vol. 4, No. 6, October 2009
Coexistence of overestimation and underweighting
453
Table 3: A summary of the results of Study 2 aggregated over Gain and Loss conditions. The number in parenthesis
denotes the null hypothesis for the test reported in the text. All t-tests are one-sample tests unless otherwise noted.
Statistic
Trials 1–200
Trials 201–400
P(R): proportion of R choices
0.79(0.5***)
0.81(0.5***)
SP(LowProb): Mean subjective assessment of the probability of the rare out-
–
?0.22(0.15***)
come
P(R | LowProb): Prop. of R choices after a trial with a rare outcome
0.71
0.77
(paired 0.82**)
P(R | HighProb): Prop. of R choices after a trial with the high probability
0.81
0.82
outcome
SP(LowProb | LowProb): Mean assessment of the probability of the rare out-
0.19
come after a trial with a rare outcome
(paired 0.23**)
SP(LowProb | HighProb): Mean assessment of the probability of a rare out-
0.23
come after a trial with a high probability outcome
*p<0.10, **p<0.05, ***p<0.01.
occur for 91% of these 22 subjects (p < 0.001, McNe-
mar’s test)
Table 4: Counts of individuals aggregated choice of R rel-
ative to 0.5 in trials 201–400, and subjective probability
estimation (SP), relative to 0.15 in Study 2.
3.2.2 The contingent recency effect
P(R) > 0.5 P(R) < 0.5
The second column of Table 3 (rows 3–6) presents the
SP(R) < 0.15
13
2
mean judgment and choice over trials 201–400 condi-
tional on the most recent outcome. For each subject we
SP(R) > 0.15
20
5
calculated two proportions, the proportion of R choices
following an observation of the rare event (aggregated
over trials 201–400) and the proportion of R choices fol-
lowing observations of the more common outcome. Ag-
R choices was 0.77. Additionally, while both overesti-
gregating over both conditions, a signi?cant amount of
mation and underweighting were concurrently observed
positive recency for choice was observed; subjects were
there was also an overall consistency between judgments
5% less likely to choose R on trials that immediately fol-
and choices. An examination of the association between
lowed an observation of the rare event (i.e., the bad out-
the mean choice rate of R and mean estimation (trials 201
come) (t[39] = 2.49, p < 0.05). On those same trials, the
to 400) over the 40 subjects reveals a correlation of r(38)
mean assessment of the rare event was 4% lower (i.e.,
= -0.48, p < 0.01.
they estimated them as less likely) than on trials not fol-
A within-person contingent recency effect (positive re-
lowing an observation of the rare event (t[39] = 2.29, p
cency in choice and negative recency in estimations) was
< 0.05), which is consistent with negative recency. No
found to occur for 11 of the 40 subjects. While only
signi?cant difference was found between the Gain and
5 subjects displayed the opposite tendency, negative re-
Loss conditions (t[38] = 1.40, n.s., for positive recency
cency in choice and positive in estimations, the difference
and (t[38] = 0.97, n.s., for negative recency). As was the
in counts was not signi?cant. Finally, a within-person
case in Experiment 1, the pattern of positive recency for
correlation between judgment and choices in trials 201 to
choices and negative recency for probability assessments
400 showed negative correlations for most of the subjects
was consistent with the contingent recency hypothesis.
(19 of 33)9, again re?ecting consistency between judg-
The contingent recency effect contributes to, but can-
ment and choice.
not explain by itself, the main results, since overesti-
mation and underweighting were observed even imme-
diately after observing the rare event. The average es-
9Correlations could not be computed for seven of the 40 subjects
timation in these trials was 0.19, and the proportion of
who chose R in every trial between 201–400.

---

Judgment and Decision Making, Vol. 4, No. 6, October 2009
Coexistence of overestimation and underweighting
454
3.2.3 Framing as an alternative mechanism for over-
1
estimation
0.9
Choice of R
Est.
In both Experiments 1 and 2 the rare event provided a
0.8
Obj. Probability
worse outcome than the more common result from the
0.7
risky distribution. These, comparatively bad, outcomes
0.6
Choice of R
may have been framed as losses by subjects. If “losses
loom larger than gains” (Kahneman & Tversky, 1979)
0.5
Est.
then these outcomes may have been more salient in mem-
Obj. Probability
0.4
ory than the relative gains, and subjects may have overes-
0.3
timated their probability for this reason. It is desirable
to differentiate between this mechanism, the increased
0.2
availability of losses, and the mechanism we assumed
0.1
based on previous research: the increased availability of
0
all rare events for probability assessments (and the addi-
1
4
7
10 13 16 19 22 25 28 31 34 37 40
tion of error to subjective judgments). Study 3 was de-
signed as a test of these two mechanisms.
Figure 3: Mean proportion of R choices in Study 3, and
subjective assessments of the rare outcome’s probability
in 40 blocks of 10 trials.
4 Study 3: Generality over framing
of rare events
4.2 Results
4.2.1 Judgment and choice in the same context
If loss aversion (relative to a reference point) is the prime
driver of the observed discrepancies in Studies 1 and 2,
The results revealed the same pattern observed in Stud-
the effect should diminish when the rare event is framed
ies 1 and 2, namely, a robust underweighting in choice
as a good outcome.
along with overestimation. Figure 3 presents subjects’
aggregate proportion of R choices and probability assess-
ments in 40 blocks of 10 trials. Over all 400 trials, sub-
4.1 Method
jects’ mean proportion of R choices was 0.19 (signi?-
cantly smaller than 0.5, t[19] = 32.32, p < 0.001), con-
4.1.1 Design
sistent with the underweighting of rare events in choice
behavior. In trials 201–400 (see Table 5), when probabil-
The design was the same as for Study 2 with the excep-
ity assessments were also elicited, the mean proportion of
tion that there was only one condition where the S distri-
R choices in these trials was 0.23 (less than 0.5, t[19] =
bution provided a certain gain of 2.7 points while the R
26.91, p < 0.001) again consistent with the underweight-
distribution provided a gain of 18 points with probability
ing of rare events in choice.
0.15 and 0 points otherwise. Thus, the expected values
The mean probability assessment from trials 201–400
and the S distribution were identical to those used in the
aggregated over trials and conditions was 0.21 (see the
Gain Condition of Study 2. The change is that the rare
second row of Table 5 and Figure 3). This is signi?cantly
event (18 points) was a relatively good outcome.
larger than 0.15, the objective probability of the rare out-
come (t[19] = 10.64, p < 0.001). This result is consistent
with an overestimation of rare events in probability as-
4.1.2 Subjects
sessments.
Twenty Technion students served as paid subjects in the
At the individual level, for all 20 subjects, assessment
study. In addition to the performance contingent payoff,
and choice results were not consistent in terms of the im-
subjects received 25 Shekels for showing up. The conver-
plied weighting of the 18 outcome, aggregated over tri-
sion rate for the one randomly chosen trial was 1 point =
als 201–400. Overestimation and underweighting of rare
1 Shekel. The ?nal average payoff was approximately 27
events was found to occur for every subject.
Shekels (about 5 US dollars).
In summary, even when the rare event is a relatively
good outcome, we found robust overestimation and un-
derweighting of rare events, as predicted by the coexis-
4.1.3 Apparatus and procedure
tence hypothesis. The result is consistent with the as-
sumption that overestimation re?ects the greater saliency
As in Study 2.
of rare events rather than the salience of negative events.

---

Judgment and Decision Making, Vol. 4, No. 6, October 2009
Coexistence of overestimation and underweighting
455
Table 5: A summary of the results of Study 3. The number in parenthesis denotes the null hypothesis for the test
reported in the text. All t-tests are one-sample tests unless otherwise noted.
Statistic
Trials 1–200
Trials 201–400
P(R): proportion of R choices
0.15(0.5***)
0.23(0.5***)
SP(LowProb): Mean subjective assessment of the probability of the rare out-
–
0.21(0.15***)
come
P(R | LowProb): Prop. of R choices after a trial with a rare outcome
0.27
0.29
(paired 0.13***)
(paired 0.22*)
P(R | HighProb): Prop. of R choices after a trial with the high probability
0.13
0.22
outcome
SP(LowProb | LowProb): Mean assessment of the probability of the rare out-
0.22
come after a trial with a rare outcome
SP(LowProb | HighProb): Mean assessment of the probability of a rare out-
0.21
come after a trial with a high probability outcome
*p<0.10, **p<0.05, ***p<0.01.
4.2.2 The contingent recency effect
ent days (Associated Press, 2002). Immediately follow-
ing this period, Israeli students were asked about their
The second column of Table 5 (rows 3–6) presents the
behavior and their probability assessments regarding the
mean judgment and choice over trials 201–400 condi-
threat of suicide bombings during the intifada. The hy-
tional on the most recent outcome. A signi?cant amount
pothesis was that, while students would assess the prob-
of positive recency for choice was observed; subjects
ability of an attack on the day after a previous attack to
were 7% more likely to choose R on trials that imme-
be lower than after an attack-free day (negative recency),
diately followed an observation of the rare event (i.e., the
they would choose to behave as if the probability had in-
good outcome) (t[19] = 1.65, p = 0.057). No signi?cant
creased (positive recency).
tendency towards negative recency was observed in this
condition, and the effect appears to be weaker when the
rare outcome is relatively favorable.
5.1 Method
5.1.1 Design
5 Study 4: The effect of rare terror- Subjects were randomly assigned to one of two condi-
ist suicide attacks
tions: Choice (43 subjects) or Probability (42 subjects).
The between subject design was chosen to eliminate
the possibility that questions regarding choice behavior
Studies 1 through 3 focused on abstract low-stake deci-
would affect probability assessments and vice-versa.
sions. They demonstrate that the well established mech-
anisms of judgment error and reliance on small samples
can lead to the coexistence of overestimation and under-
5.1.2 Subjects
weighting of rare events. The contingent recency effect
In the summer of 2002, following the intifada, Eighty-
contributes to this coexistence and was found in three out
?ve (46 males and 39 females) Technion students served
of four conditions tested, when the rare event was a rela-
as paid volunteers who came to ?ll out a number of un-
tively bad outcome. Study 4 was designed to evaluate the
related questionnaires. Subjects were paid 40 Shekels
generality of this effect to events outside the laboratory
(about 8 US Dollars) for their time.
in natural settings. It examines natural high-stake deci-
sions where the rare event is clearly disastrous: Human
5.1.3 Apparatus and procedure
reaction to suicide bombings in Israel.
During the al-aqsa intifada there was a period of 700
In both conditions subjects answered three questions on
days (September 30, 2000 to August 31, 2002) in which
5-point scales. Subjects were instructed that the ques-
suicide-bombing attacks were carried out on 71 differ-
tions pertained to the events of the (then) recent intifada.

---

Judgment and Decision Making, Vol. 4, No. 6, October 2009
Coexistence of overestimation and underweighting
456
5
20
Probability
17
Choice
16
4
15
3.56
3.52
2.93
3
2.58
10
9
2.21
2
5
1
After attack
After attack
No attack
0
with fatalities
no fatalities
After attack
After attack
No attack
with fatalities
no fatalities
Figure 4: Mean response (on a 1 to 5 scale) to the three
questions (on the x-axis) in conditions Choice and Proba-
Figure 5: Percentage of days where a suicide bombing
bility. In Condition Choice the responses re?ect the rela-
occurred according to the previous day for the period of
tive extent to which the responders felt cautious; in Con-
September 30, 2000 - August 31, 2002, during the al-aqsa
dition Probability the responses re?ect relative magnitude
intifada.
of estimated chance of attack.
subjects in the Choice condition reported more cautious-
The ?rst question asked about days on which there was
ness after an attack with fatalities than after a day with-
no attack on the previous day, the second question asked
out an attack (3.56 and 2.58 respectively, t[42] = 4.35, p <
about days on which there was an attack on the previ-
0.001). Yet, subjects in the Probability condition reported
ous day, but without fatalities. The third question asked
that they believe the chances of another suicide attack to
about days on which there was an attack with fatalities on
be smaller in the day after an attack with fatalities than
the previous day.10 In the Choice condition subjects were
after a day without an attack (2.21 and 3.52 respectively,
asked about their behavior while in the Probability con-
t[41] = 6.36, p < 0.001). In addition, these con?icting
dition they were asked about their estimate. For example,
positive and negative sequential dependencies were sig-
in the Choice condition, the third question was:
ni?cantly different (0.98 and ?1.3 respectively, t[83] =
7.5, p < 0.001). While seemingly paradoxical, these re-
“The day after a suicide bombing with fatalities, I am
sults are consistent with the results from Studies 1 and 2,
cautious about another suicide attack:”
with subjects exhibiting negative recency in their prob-
Much less
Much more
ability assessments while exhibiting positive recency in
than usual
than usual
choices.
The previous result is suf?cient to provide a demon-
1
2
3
4
5
stration of inconsistent choice and judgment in the con-
text of small probabilities. Nonetheless, we completed
The same question in the Probability condition was: “The
a brief analysis of the objective sequential dependencies
day after a suicide bombing with fatalities, the chance
in the bombing data. Figure 5 presents the percentage
of another suicide attack is:”. The same ?ve-point scale
of days where a suicide bombing occurred according to
accompanied all three questions in both conditions.
what happened the previous day for the period of Septem-
ber 30, 2000 to August 31, 2002, the period of al-aqsa
5.2 Results
intifada (Associated Press, 2002). While an attack was
almost twice as likely the day after a previous attack
Figure 4 presents the mean response to the three questions
(with or without casualties) than after a normal day, this
in conditions Choice and Probability. As can be seen,
difference is marginally signi?cant only after combin-
ing days after attacks with and without casualties (chi-
10The distinction between attacks with and without fatalities was in-
troduced to capture the intuition that the media’s differential coverage
squared(1)=3.54, p=0.06). This result suggests positive
of the two events might have a different effect on subjects.
recency in the series of suicide bombings for this period.

---