Cognitive abilities and superior decision making under risk: A protocol analysis and process model evaluation

Judgment and Decision Making, Vol. 4, No. 1, February 2009, pp. 20–33
Cognitive abilities and superior decision making under risk: A
protocol analysis and process model evaluation
Edward T. Cokely?
Colleen M. Kelley
Max Planck Institute for Human Development
Department of Psychology
Center for Adaptive Behavior and Cognition
Florida State University
Abstract
Individual differences in cognitive abilities and skills can predict normatively superior and logically consistent judg-
ments and decisions. The current experiment investigates the processes that mediate individual differences in risky
choices. We assessed working memory span, numeracy, and cognitive impulsivity and conducted a protocol analysis to
trace variations in conscious deliberative processes. People higher in cognitive abilities made more choices consistent
with expected values; however, expected-value choices rarely resulted from expected-value calculations. Instead, the
cognitive ability and choice relationship was mediated by the number of simple considerations made during decision
making — e.g., transforming probabilities and considering the relative size of gains. Results imply that, even in simple
lotteries, superior risky decisions associated with cognitive abilities and controlled cognition can re?ect metacognitive
dynamics and elaborative heuristic search processes, rather than normative calculations. Modes of cognitive control
(e.g., dual process dynamics) and implications for process models of risky decision-making (e.g., priority heuristic) are
discussed.
Keywords: Risky choice, intelligence, working memory, numeracy, cognitive control, dual process theory, information
search, rationality, expected value, protocol analysis, priority heuristic.
1 Introduction
2008). A variety of theories, such as dual-process the-
ories, attribute the individual differences to deliberative
Human decision-making is constrained by its bounded ra-
processes (Baron, 1985; De Neys, 2006; Evans, 2008;
tionality and does not always follow normative prescrip-
Frederick, 2005; Kahneman, 2003; Kahneman & Fred-
tions (Gigerenzer, Todd, & the ABC Research Group,
erick, 2007; Sloman, 1996; Stanovich & West, 1998;
1999; Kahneman, 2003; Payne, Bettman, & Johnson,
2000); however, the link between decision processes and
1993; Simon 1990). Nevertheless, individual differences
abilities is largely uninvestigated. What are the cognitive
in cognitive abilities and skills predict normatively supe-
processes that give rise to the relationship between cog-
rior judgment and decision-making (Frederick, 2005; Pe-
nitive abilities and superior decision making under risk?
ters & Levin, 2008; Peters, Vastfjall, Slovic, Mertz, Maz-
Previous research has examined individual differences
zocco, & Dickert, 2006; Stanovich & West, 1998; 2000;
in decision making under risk in lotteries with known
probabilities.
For low stakes lotteries normative ex-
?We thank the Society for Judgment and Decision Making for a stu-
pected utility processes are assumed to be approximated
dent research prize, awarded for a poster presentation based on this
research at the 21st annual meeting of the Society (2007). This re-
with calculations that multiply probabilities by potential
search was completed as part of Edward Cokely’s dissertation submit-
gains/losses, i.e., expected-value calculations (Frederick,
ted to The Florida State University. We would like to acknowledge the
2005; Payne, Samper, Bettman, & Luce, 2008). Fred-
other members of the committee including K. Anders Ericsson, Neil
Charness, Joyce Ehrlinger, and Michelle Bourgeios. We offer special
erick has demonstrated that expected-value choices are
thanks to Henrik Olsson and to Gerd Gigerenzer for extensive com-
associated with scores on the cognitive re?ection test,
ments on earlier versions of the manuscript. We also thank Tres Roring,
which is designed to measure one’s reliance on more con-
Ainsley Mitchum, Lael Schooler, Jeffery Stevens, Jörg Rieskamp, Lin-
sciously controlled processes rather than automatic ?rst
nea Karlsson, Mirta Galesic, Bettina Von Helversen, Cari Zimmerman,
Mark Fox, Katy Nandagopal, Mike Tuf?ash, and Steven Sloman for
impressions (e.g., Stanovich and West’s, 2000, delibera-
their comments. We are indebted to Amanda Walsh, Alicia Eddy, Ashly
tive System 2 rather than intuitive System 1). The three-
Baker, Carolina Avila, Tristan McCain, and Richard Molina for assis-
problem cognitive re?ection test, which is known to cor-
tance in testing subjects and transcribing protocols. Address: Edward
relate with other general cognitive ability measures, con-
Cokely, Center for Adaptive Behavior and Cognition, Max Planck Insti-
tute for Human Development, Lentzeallee 94, 14195 Berlin, Germany.
sists of mathematical problems for which an immediate
Email: cokely@mpib-berlin.mpg.de.
intuitive response is incorrect. Frederick demonstrated
20

---

Judgment and Decision Making, Vol. 4, No. 1, February 2009
Cognitive abilities and superior risky choice
21
that higher scoring individuals did not exhibit the clear
cording to the priority heuristic, decisions between sure
non-normative risk asymmetry for gains and losses pre-
versus risky options are the result of considering simple
dicted by prospect theory (Kahneman & Tversky, 1979).
reasons for a decision in a ?xed order, until a stopping
When given a choice between a gain of $100 versus a
rule is met. First, people consider minimum gains. If
75% chance of a $200 gain, prospect theory predicted risk
the minimum gains differ by 1/10 or more of the max-
aversion and the selection of the certain $100.1 However,
imum gain (1/10 of the maximum gain rounded to the
people with higher cognitive re?ection scores more of-
closest prominent number) consideration stops and peo-
ten selected options with the higher expected values (i.e.,
ple choose the option with the higher minimum gain. If
the probability multiplied by the potential risky gain —
necessary, they consider a second reason, the probabil-
$150) as compared to lower scoring individuals.
ity of the minimum gain. If the probabilities of the two
There are several candidate mechanisms that may ac-
options differ by 1/10 or more of the probability scale,
count for the link between cognitive abilities and supe-
consideration stops and people choose the option with the
rior decision making under risk. For example, one can
higher probability minimum gain. If necessary, they will
make expected-value choices by performing expected-
consider a third reason and choose the option with the
value calculations. Frederick (2005) suggests that com-
higher maximum gain. A similar set of reasons and stop-
putation of expected values may play a role, although
ping rules occur for choices between losses.
he notes that it is not likely the only factor. More gen-
The priority heuristic has accurately described ma-
erally, Stanovich and West (2000) suggest that individ-
jority choice-outcomes in several theoretically important
ual differences in normative judgments and decisions of-
datasets (but for critical reviews see Birnbaum, 2008;
ten arise from working memory capacity limitations on
Hilbig, 2008; Johnson et al., 2008). Some evidence also
computation, implying that high ability individuals may
supports the priority heuristic process model as laten-
make expected-value choices via expected-value calcu-
cies to choose between two options have been greater for
lations.2 Other research indicates that individual differ-
choices that require three considerations compared to one
ences in risky decision making may also arise from varia-
consideration (Brandstätter et al., 2006). However, the
tions in one’s general knowledge and understanding of
priority heuristic is silent on the potential cognitive pro-
probabilities — i.e., one’s numeracy (Peters & Levin,
cesses that may mediate the relationship between cogni-
2008; Peters et al., 2006). People high in numeracy, par-
tive abilities and superior decision-making. Given that
ticularly the ability to comprehend and transform prob-
the priority heuristic is designed to predict potentially
abilities, are less affected by attribute framing because
non-normative majority choices we hypothesized that it
they can readily transform items such as 74% correct into
may predict many participants’ choices and choice pro-
26% incorrect and translate percentages to frequencies
cesses, although it would be unlikely to predict behavior
and vice versa. Thus, numeracy may allow better risky
of high ability individuals.
choices as a result of a more accurate subjective sense of
the size of gains and losses or other probability related
trade-offs.
1.2 Heuristic search
Heuristics and simple considerations are common and of-
1.1 Process models of risky choice
ten effective bases for judgment and choice (Gigerenzer
et al., 1999; Payne, Bettman, Coupey, & Johnson, 1992;
Theories describing the actual cognitive processes com-
Payne et al., 1993; Tversky & Kahneman, 1974). We hy-
monly used for decision making under risk tend to be
pothesized that the relationship between cognitive abili-
imprecise (Brandstätter, Gigerenzer, & Hertwig, 2006;
ties and decision-making under risk would not necessar-
2008; Johnson, et al. 2008; Payne et al., 1993; Payne &
ily arise from expected-value calculations, but could re-
Braunstein, 1978; Selart et al., 2006). Risky choice mod-
sult from simple considerations of reasons as in the pri-
els are typically as-if models, as in the case of prospect
ority heuristic, and simple transformations of probability
theory, which does not describe the exact cognitive oper-
information as in the research by Peters et al. (2006; Pe-
ations of choice but holds only that people act as-if they
ters & Levin, 2008). Theory suggests that variation in
evaluate losses with a steeper utility curve (Johnson et
superior decision making does not necessarily need to
al., 2008). One exception to as-if modeling is the priority
rely on the exact use of calculations based on norma-
heuristic which is a parameter free choice-outcome and
tive models but can result from greater re?ectiveness or
cognitive process model (Brandstätter et al., 2006). Ac-
thoroughness in decision making (Baron, 1985; 1990).
Variation in risky choice performance has been linked to
1The exact predictions of prospect theory depend on the model pa-
differences in duration and type of information search
rameters used.
2Stanovich and West (1998) have also shown that metacognitive fac-
(Mann & Ball, 1994; Payne & Braunstein, 1978; Se-
tors account for unique variance in judgment and decision performance.
lart et al., 2006). Working memory measures are also

---

Judgment and Decision Making, Vol. 4, No. 1, February 2009
Cognitive abilities and superior risky choice
22
Table 1: Example of discrete model predictions and predicted verbal reports for a sample choice.
Sample options:
A: 100% chance to gain $150
B: 5% chance of gaining $2000
Decision Process and Rules
Potential Protocols
Choice
Expected value
Step 1:
Multiply probability by risky options
“that’s about 100 dollars” “$100
versus $150”
Step 2:
Select higher expected value
“5% of 2000 is less than $150”
A: Certain
choice
Priority heuristic
Step 1:
Is the difference in minimum gains larger
“150 is bigger than zero” “that’s
than 10% of the maximum gain? If not, go
less than $200” “$150 is less than
on. ($150 is less than $200 — so go on)
10% of 2000”
Step 2:
Is the difference in minimum gain
“$150 is a sure thing and 5%
probabilities larger than 10% of the
probably won’t happen” “that gain
probability scale? if not, go on. (95% and
is unlikely, but the other gain is
100% do not differ by 10% so go on)
certain”
Step 3:
Select the higher maximum gain.
“2000 is higher than 150”
B: Risky
choice
known to predict strategic differences in elaboration dur-
tial limitations see De Neys & Glumicic, 2008; Reisen,
ing encoding (Bailey, Dunlosky, & Kane, 2008; Cokely,
Hoffrage, & Mast, 2008). To illustrate this methodol-
Kelley, & Gilchrist, 2006; Guida, Tardieu, & Nicolas,
ogy, both an expected-value calculation and the priority
2008; McNamara & Scott, 2001) and differences in the
heuristic process predict that participants should consider
number of hypotheses generated during probability judg-
distinct types of information when making their choices.
ment (Dougherty & Hunter, 2003; Thomas, Dougherty,
Verbalization of an expected value or an attempt to es-
Sprenger, & Harbison, 2008). Therefore, we hypothe-
timate one (e.g., “75% of $200 is de?nitely more than
sized that elaborative heuristic search — i.e., more thor-
$100”) would provide evidence of expected-value type
ough exploration and representation of the problem space
processes. Similarly, the priority heuristic makes pre-
— would often be positively related to superior risky de-
dictions about what information will and won’t be con-
cision making. To test the elaborative heuristic search hy-
sidered for different lotteries, and in what order (Brand-
pothesis and to more precisely trace cognitive processes,
stätter et al., 2006) (Table 1). These predictions allowed
we conducted a protocol analysis.
us to develop a coding system to quantify the types and
amounts of considerations that were consistent and in-
consistent with processing products predicted by the pri-
2 Experiment
ority heuristic and expected-value calculations. Protocol
analysis codes were also derived from previous research
(Rettinger & Hastie, 2001; 2003) and a pilot study (Table
Our experiment was designed to examine individual dif-
2).
ferences in decision processes. Process tracing was per-
formed with retrospective verbal reports (Ericsson & Si-
We hypothesized that protocol analysis would re-
mon, 1980) in which participants verbally reported the
veal a positive relationship between expected-value type
exact thoughts they remembered in the order in which
choices and elaborative heuristic search (Baron, 1985;
they occurred, immediately following their choices.
Payne, 1976; Selart et al., 2006; Simon, 1990), opera-
When people consciously and deliberately consider in-
tionalized as the total number of different types of sim-
formation, such as comparing minimum gains or trans-
ple considerations verbalized (excluding expected-value
forming information into different probabilities, these
calculations and ambiguous codes), regardless of output
processes should be observable in participants’ protocols
order (Table 2).3 We also hypothesized that elaborative
(Evans, 2008; Sloman, 1996). Verbal reports have pre-
3Retrospective reports were selected as the concurrent reports used
viously been effectively used in related studies of choice
during pilot studies were often unrevealing. Because retrospect reports
(Rettinger & Hastie, 2003; Payne 1976; but for poten-
rely on memory they are not as reliable as concurrent reports concerning

---

Judgment and Decision Making, Vol. 4, No. 1, February 2009
Cognitive abilities and superior risky choice
23
Table 2: Coding system for protocol analysis including examples of each consideration, mean considerations per trial
(and standard deviations), and total observed considerations.
Protocol codes
Example Considerations:
Mean considerations Total observed
($125 or 30% of $900)
1. Minimum differences
$125 is more than nothing
.01 (.04)
26
2. Maximum differences $900 is a lot more than $125
.30 (.20)
712
3. Recode probability
30% chance is 70% to gain nothing
.13 (.14)
298
4. Probability low
30% just won’t happen
.41 (.18)
950
5. Probability high
30% will probably happen
.16 (.08)
383
6. 10% of maximum
10% of $900 is about 100
.00 (.00)
3
7. Avoid risks
I always want a sure thing
.13 (.11)
296
8. Avoid losses
I never want to lose anything
.03 (.04)
76
9. Maximum money
I want the most money
.08 (.10)
189
10. Value low
$125 is nothing
.11 (.10)
254
11. Value high
$900 is a lot of money
.17 (.13)
405
12. Expected value
30% of 900 is more than $125
.08 (.19)
198
13. Other-ambiguous
A is better than B
.22 (.20)
517
heuristic search would at least partially mediate the re-
reports were lost because of equipment failure. Seven
lationship between cognitive abilities and superior deci-
participants did not receive numeracy scores due to a pro-
sions.4 More elaborative and thorough search processes
cedural error.
were expected to include variations in the number of con-
siderations (e.g., consider maximum gains and probabil-
ities versus considering only maximum gains) as well as
2.2 Materials
explorations of different aspects of problems (e.g., inter-
Ability measures included: (1) the operation span —
pret the large difference between potential gains as a po-
a working memory capacity task that partially mediates
tential loss). Such variations could help some participants
relationships predicted by traditional intelligence instru-
avoid overlooking valuable information or oppose the in-
ments (Turner & Engle, 1989); (2) the cognitive re?ec-
?uence of framing effects (Peters et al, 2006).
tion test (CRT) which assesses differences in cognitive
impulsivity (System 1) versus more deliberative thinking
2.1 Participants
(System 2) (Frederick, 2005); (3) a numeracy scale mea-
suring understanding of numerical probabilities (Lipkus,
Eighty undergraduate students from introductory psy-
Samsa, & Rimer, 2001; see Peters and Levin, 2008 for
chology courses at Florida State University participated
the 11 item scale).
in partial ful?llment of course requirements and were
tested individually. Four cognitive re?ection scores were
excluded as participants had seen the test in another ex-
2.3 Decision making under risk
periment. Four working memory scores and two verbal
The stimuli included 40 choice problems with hypothet-
the ordering of cognitive events (Ericsson & Simon, 1980). Therefore,
we used a conservative data analysis approach focusing only on the type
ical gains/losses presented in US dollars. Each choice
and number of considerations verbalized, but not the order of output.
consisted of one certain option and one risky option,
The verbalized content of interest is likely to be at least moderately reli-
balanced such that expected value and priority heuristic
able as verbal protocols immediately followed decision making (which
models made unique predictions on exactly half of the
lasted only a few seconds) and the number of verbalized considerations
was found to correlate with participants’ overall decision latencies (see
trials. Expected-value ratios of lotteries were on average
results).
near the indifference point (M = 2.07, range = .15 to 5.3,
4In a pilot study, as part of the ?rst author’s doctoral dissertation,
relative to the certain option) a range in which the priority
we found that the majority of participants did not have suf?cient math
heuristic is expected to predict choices (for discussion see
skills to calculate expected values when explicitly instructed to do so.
Nonetheless, many of these participants still made many choices con-
Brandstätter et al., 2006; 2008). Expected-value calcula-
sistent with expected-value predictions.
tions predicted equal numbers of risky choices for gains

---

Judgment and Decision Making, Vol. 4, No. 1, February 2009
Cognitive abilities and superior risky choice
24
and losses; priority-heuristic predictions were asymmet-
Because abilities are known to in?uence choice, and
ric favoring risky choices for losses, but not gains. Pri-
given evidence on the limits of majority choice aggrega-
ority heuristic also predicted that 60% of choices would
tion analyses (Regenwetter, Grofman, Popova, Messner,
involve less search (i.e., a single consideration of the min-
Davis-Stober, & Cavagnaro, 2008), we examined individ-
imum possible gains/losses relative to 10% of the max-
ual model-prediction-accuracy scores. Subsequent analy-
imum gain/losses) while the other 40% of choices re-
ses compared the proportion of expected value consistent
quired the maximum number of considerations (i.e., all
choices averaged across all choices for each individual.
possible steps of the priority heuristic). Risky option
A one sample t test indicated that expected-value calcu-
probabilities ranged from 1%-80% (Appendix).
lations strongly predicted participant choices (M = .72,
SD = .12) above chance levels, t (79) = 16.02, p = .001,
2.4 Procedure
d = 1.9. The proportion of choices consistent with ex-
pected value was signi?cantly related to CRT, r (74) =
Participants were tested individually. Responses were
.27, p = .02, and numeracy, r (71) = .28, p = .02 (Table
recorded by a head-mounted microphone. Verbal report
6). A mixed model analysis of variance (ANOVA) with
instructions and warm up think-aloud problems were pro-
risk type (certain, risky) by choice type (gain, loss) by
vided by an experimenter seated behind the participant.
working memory span quartile (low, high) also indicated
The experiment began with the cognitive re?ection task
that working memory was associated with differences in
followed by an example lottery. Participants were told
choices, F (1, 36) = 7.70, p = .01, d = .8. High work-
that the experiment involved 40 such choices, all of which
ing memory span participants made signi?cantly more
were presented in the same randomized order. Choices
expected-value type choices (M = .79, SD = .13) as com-
were presented from the top to the bottom of the screen
pared to low span participants (M = .70, SD = .10).
with the ?rst option (e.g., “A. gain $50”) displayed for
two seconds before the second option appeared (e.g., “B.
50% to gain $400”). Choices remained on the screen un-
3.1 Protocol analysis
til the participant made a selection and was prompted for
a retrospective report. Lastly, participants completed the
Verbal reports were analyzed by two raters blind to
working memory span and numeracy measures, and were
model and judgment performance (Table 2). A randomly-
debriefed.
selected subset of verbal reports (13%) were scored by
both raters and indicated high inter-rater agreement on the
number of considerations, r (8) = .97, p = .01, and sub-
3 Results
stantial agreement on speci?c consideration codes (kappa
= .63). The total number of considerations verbalized was
Following Brandstätter et al. (2006), a model competition
also related to the mean choice reaction time, r (67) = .46,
was conducted. This analysis assessed the frequency with
p = .001,6 indicating that individuals who retrospectively
which each model predicted majority choices, across all
verbalized more considerations also took longer to make
choices. Binomial analysis indicated that expected-value
their judgments. Unless otherwise noted, seven partic-
calculations predicted majority choices signi?cantly bet-
ipants were excluded from subsequent analysis because
ter than chance (M = .83, p = .001). A non-parametric
more than 50% of their verbal protocols were unreveal-
test of equal proportions indicated that expected value
ing (e.g., “A is better; I like B”).7
also signi?cantly outperformed the priority heuristic, ?2
Three individuals verbalized expected-value calcula-
= 12.17, p = .001, d = 1.3, which predicted at chance
tions (or estimations) nearly exclusively (95–100% of all
levels (M = .45, p > .5). A variety of subsequent analy-
ses of the priority heuristic converged to suggest that in
three-reason choices (M = 11.79sec, SD = 9.16sec). Protocol analysis
the current task environment the priority heuristic was an
also revealed that key comparisons (i.e., considering the difference in
minimum gains relative to the maximum gain) predicted on 100% of
inaccurate process and choice-outcome model (see also
all trials were reported on fewer than 1% of trials, whereas processes
Birnbaum, 2008; Hilbig, 2008; Johnson et al., 2008).5
that were never predicted were among the most frequent verbalizations
(e.g., recoding probabilities, considering probabilities low or high; see
5Additional individual model-prediction-accuracy scores (i.e., anal-
Table 2). Priority heuristic choices were unrelated to all cognitive abil-
yses comparing the proportion of priority heuristic consistent choices
ity measures (p > .20).
averaged across all choices for each individual) indicated that priority
6Reaction time was related to number of verbalized considerations;
heuristic was less accurate (M = .42, SD = .09) compared to chance,
however, latencies showed only an unreliable trend in the expected di-
t (79) = -7.40, p = .001, d = .9, or as compared to expected value, F
rection with decision performance, F (1, 3151) = 3.11, p = .08. Non-
(1, 79) = 428.95, p < .001, ? 2
p
= .84. A univariate ANOVA indicated
expected-value choice latencies (M = 12.53 sec, SD = 9.16 sec) were
a signi?cant search difference (one reason, three reasons) in reaction
similar to expected-value choice latencies (M = 13.20 sec., SD = 9.85
times, F (1, 3151) = 19.63, p = .001. However, this difference was in
sec.).
the opposite direction of that predicted by the priority heuristic. One-
7Inclusion of these data in additional hierarchical regressions did not
reason choices tended to take longer (M = 13.3sec, SD = 9.82sec) than
signi?cantly change results.

---

Judgment and Decision Making, Vol. 4, No. 1, February 2009
Cognitive abilities and superior risky choice
25
1.0
tom quartile) working memory scores (M = 47.1, SD =
14.5), F (1, 30) = 4.51, p = .04, d = .08.
We next constructed a series of hierarchical linear re-
0.9
gression models (Table 7) with the most complex (full)
model using three predictors including (1) expected-value
0.8
verbalizations; (2) all three ability measures; and (3)
number of verbalized considerations. The full model was
0.7
a strong predictor of expected-value choices, F (5, 53) =
22.23, p = .001, R2 = .44. The number of verbalized con-
0.6
siderations accounted for a moderate amount of unique
variance, F (1, 53) = 8.15, p = .001, R2change = .24. The
0.5
number of verbalized considerations also fully mediated
Expected?value choices (proportion)
the relationships between all three cognitive ability mea-
0.4
sures and expected-value choices (ts < 1, see Table 7).
To what extent might these results re?ect the in?uence
0
20
40
60
80
100
of particular choices, such as choices on gains rather than
Number of considerations verbalized
losses or choices involving high versus lower monetary
values? To assess independent relationships controlling
Figure 1: A linear regression with elaborative heuristic
for these potentially in?uential factors we conducted a
search (i.e., the number of verbalized considerations) pre-
multilevel analysis. First, we constructed independent re-
dicting each participant’s overall proportion of expected-
gression equations for each participant, predicting each
value choices (ambiguous and expected-value verbaliza-
participant’s responses across all 40 choice trails. Indi-
tions are not included). The line is based on the regres-
vidual level regression equation coef?cients (i.e., unstan-
sion.
dardized ? coef?cients) were computed for each of the
following variables (1) expected-value model choice pre-
dictions; (2) priority heuristic model choice predictions;
trials; see Table 3 for examples). The frequency of ver-
(3) gain versus loss trails (to assess and control for poten-
balized expected-value calculations was signi?cantly re-
tial asymmetries in responding); and (4) the highest ab-
lated to expected-value type choices, r (69) = .25, p =
solute monetary value for each choice (to assess and con-
.03; however, expected-value calculations were unrelated
trol for potentially non-uniform in?uences of declining
to cognitive ability variables (Table 6). The remaining
marginal utility).10 Next, we examined the intercorrela-
participants exhibited a clear relationship between the
tions between the individual level regression coef?cients,
number of considerations verbalized and expected-value
all cognitive abilities, and the number of verbalized con-
choices excluding any ambiguous or expected-value ver-
siderations (Table 8).11
balizations, ? = .60, t = 15.90, p = .001, R2 = .36 (Fig-
As expected, results revealed reliable relationships be-
ure 1).8 Individuals who made the most expected-value
tween the expected-value choice coef?cients and all cog-
choices (top quartile) verbalized about twice as many
nitive ability measures including the cognitive re?ection
considerations per trial (M = 1.78, SD = .52) as did
test, r (66) = .29, p = .02; numeracy, r (62) = .29, p =
those who made fewer (bottom quartile) expected-value
.02; and working memory span, r (66) = .27, p = .03.
choices (M = .94, SD = .35). Across all participants,
The number of verbalized considerations was also sig-
the number of considerations verbalized was also signif-
ni?cantly related to the expected-value choice factor, r
icantly related to CRT, r (72) = .23, p = .05; numeracy,
(69) = .45, p = .001. Lastly, a hierarchical linear re-
r (69) = .36, p = .01; and working memory span, r (72)
gression was constructed, following the previous analy-
= .25, p = .04 (see Tables 4 and 5 for examples of ver-
ses but predicting the expected-value individual level co-
bal protocols; see Tables 6 and 8 for intercorrelations
ef?cients with (1) expected-value verbalizations, (2) all
among variables).9 For example, across 40 trials, ex-
three ability measures, and (3) number of verbalized con-
cluding expected-value or ambiguous verbalizations, in-
siderations. The full model was again a strong and sig-
dividuals with higher working memory span scores (top
ni?cant predictor, F (5, 53) = 4.22, p = .003, R2 = .29.
quartile) verbalized signi?cantly more considerations (M
= 60.8, SD = 20.8) as compared to those with lower (bot-
10Unreported analyses also investigated the in?uence of a ?fth factor,
an interaction between (3) & (4), which was found to be trivial and un-
8Verbalizations of expected-value calculations are never included in
related to all other variables including abilities and elaborative heuristic
estimates of elaborative heuristic search.
search.
9The correlations presented in Table 6 include the seven participants
11Each individual level regression coef?cient factor represents the es-
whose verbal reports were ambiguous and so the strength of some cor-
timated unique in?uence of that variable controlling for variance at-
relations may be underestimated.
tributable to all other individual level coef?cient variables.

---

Judgment and Decision Making, Vol. 4, No. 1, February 2009
Cognitive abilities and superior risky choice
26
Table 3: A sample of protocol analysis revealing expected-value calculation or estimation.
Choice options
Protocol analysis sample: Expected-value verbalizations
Loss: $50 or 5% chance to 5% times $4000 is $200 certain loss or is it $20. No $200 which is more than $40
lose $4000
certain loss.
Gain: $275 or 20% chance 10% chance of 900 is $90 which is $180. $180 or $275, $275 is more so yeah.
to win $900
275 certain gain is more than 20% of 900 which is $180 gain.
Loss: $120 or 5% chance uhh crap 5% chance- .05 times 1600 is pretty sure $30 or $300. uhh its $30 so
to lose $1600
$120 is more than $300, no, whatever, so $300 is more.
Gain: $80 or 3% chance to My ?rst thought, 3% of 5600 would be more than A
gain $5600
Loss: $275 or 20% chance 20% okay 50% of 900 is 450, 900 times .20 is $180. $275 certain loss is more
to lose $900
than $180 certain loss.
Gain: $150 or 30% chance Yeah its b. $150 is not or 30% of 1080 is more than $100
to gain $1080
Loss: $200 or 1% chance 10% of 3000 is 300. 200 certain loss is way more than 1% of $3000 and that’s
to lose $3000
how I came about that.
Loss: $50 or 50% chance uhh that’s easy 50% of- $200 is more than $50 certain loss.
to lose $400
Gain: $50 or 5% chance to My ?rst thought, 5% of 4000 is more than B.
win $4000
The number of verbalized considerations also accounted
tions, even when associated with normatively superior
for unique variance beyond other variables, F (1, 53) =
decision performance.
10.06, p = .003, R2change = .14, and again fully mediated
the in?uence of all three cognitive abilities (ts < 1).
4.1 Dual-process models and modes of cog-
nitive control
4 Discussion
The elaborative heuristic search captured by protocol
analysis in the current experiment may, in part, result
A very small minority of our sample (about 5%) consis-
from differences in top-down, early selection cognitive
tently verbalized expected-value processes during deci-
control mechanisms used during the task (Jacoby, Kel-
sion making (Payne & Braunstein, 1978). The vast ma-
ley, & McElree, 1999). The prevailing theoretical frame-
jority of expected-value choices were instead associated
work emphasizes a late correction cognitive control in-
with simple heuristic-type decision processes. These de-
terpretation of dual process dynamics. That is, when
cision processes were similar to the component consid-
controlled processes (System 2) do not compute an an-
erations in the priority heuristic (see Table 2), although
swer they are assumed to primarily operate by monitoring
the priority heuristic was otherwise an inaccurate process
and correcting the output of automatic processes (Kahne-
and choice-outcome model. Consistent with the elabo-
man, 2003). In contrast, early selection cognitive con-
rative heuristic search hypothesis we found a relation-
trol uses controlled processing (System 2) to generate
ship between the number of considerations verbalized
goals, strategies, and mental contexts that qualitatively
and expected-value choices. Elaborative heuristic search
alter the output of automatic processes (System 1) be-
also mediated the relationships between cognitive abili-
fore critical impressions are yielded (Jacoby, Shimizu,
ties and expected-value choices.12 These results demon-
Daniels, & Rhodes, 2005). For example, if some par-
strate that neither deliberative thinking nor cognitive abil-
ticipants approached the task with the mindset of play-
ities are necessarily associated with normative calcula-
ing a game (e.g., “I feel lucky”) they would likely gener-
12One reviewer suggested a potential concern that completing the
ate different search processes as compared to those con-
CRT before making choices might in?uence choice processes and out-
struing choices in terms of their actual spending power
comes (cf. Hsee & Rottenstreich, 2004). Although we suspect this is
(e.g., “the probability is low but I don’t even have $7000
unlikely the presence of this type of effect cannot be ruled out. Nonethe-
less, such an effect would not undercut the theoretical implications link-
dollars”). Spending-power type considerations (i.e., con-
ing abilities, performance, and elaborative heuristic search process.
sidering values small or large) were found to be signi?-

---

Judgment and Decision Making, Vol. 4, No. 1, February 2009
Cognitive abilities and superior risky choice
27
Table 4: A sample of coded protocol analysis from individuals with lower working memory, numeracy, and/or cogni-
tive re?ection scores (i.e., bottom quartile).
Choice Options
Protocol Analysis Sample: Lower Ability Verbalizations
Loss: $50 or 5% chance to 5 percent isn’t that big of a percentile (probability low).
lose $4000
Gain: $275 or 20% chance A [risky choice], because it’s more money (max difference).
to win $900
Loss: $120 or 5% chance 1600 is a lot more than 120 so A [120] (max difference).
to lose $1600
Gain: $80 or 3% chance to First thought was wow, only 3%? Lame. (probability low).
gain $5600
Loss: $275 or 20% chance My ?rst thought was that 275 certain loss was less so I chose that one (max dif-
to lose $900
ference).
Gain: $150 or 30% chance Um, my ?rst thought was to look at the percents and 30% of a $1080 gain is
to gain $1080
— those aren’t great chances (probability low) so I decided to pick the certain
amount of money.
Loss: $200 or 1% chance B is a lot more riskier than A so I chose A (avoid risks).
to lose $3000
Loss: $50 or 50% chance 50 percent is a lot (probability high).
to lose $400
Gain: $50 or 5% chance to My ?rst thought was the percent is really low (probability low) so I went with the
win $4000
sure gain.
cant predictors of expected-value choices [r (76) = .41, r
contextualization during reasoning — as opposed to ab-
(76) = .36, respectively] and were also strongly related
stract rule based expected-value calculations — which
to the overall number of considerations [r (76) = .62,
was evidenced by more elaborative heuristic search and
and r (76) = .68, respectively]. Moreover, related re-
consideration of more concrete real world implications
search indicates that other judgment and decision biases
of choices (for other links between context, abilities, and
— e.g., the endowment effect and non-rational discount-
performance see Delaney & Sahakyan, 2007; Morsanyi
ing in intertemporal choice — can be accounted for by
& Handley, 2008). Given that elaborative heuristic search
one’s initial memory query and the resulting constraints
accounted for unique variance beyond cognitive abilities,
on memory search and accessibility (see query theory and
bene?cial elaborative search processes may not require
the preferences-as-memory framework; Johnson, Haubl,
an exceptional cognitive capacity or skill. Instead, su-
& Keinan, 2007; Weber, Johnson, Milch, Chang, Brod-
perior risky decision performance may partially re?ect a
scholl, & Goldstein, 2007).
cognitive style that is typical of (but not necessarily lim-
A common assumption of dual process theories is that
ited to) individuals with higher working memory span.
controlled cognition (System 2) re?ects more rule-based,
Such a metacognitive style could generally bring more
abstract and decontextualized reasoning whereas more
world knowledge to bear on many problems and thus
automatic and impulsive cognition (System 1) is driven
may be less prone to compartmentalization and impulsive
by associations, personal relevance, and situational-
choice (Baron, 1985; Stanovich and West, 2000). Addi-
contextual information (cf. fundamental computational
tionally or alternatively, these cognitive style factors may
bias, Stanovich & West, 2000; but see also Evans,
be driven by more crystallized knowledge or skill mech-
2008).13 Interestingly, in the current experiment more de-
anisms. For example, more numerate individuals could
liberation was associated with more personalization and
derive more affective meaning from the consideration of
probabilities and the comparison of options (Peters et al.,
13Evans notes that “the notion that System 2 is in some sense rule-
2006, Experiment 4), which could motivate more elabo-
based is compatible with the proposals of most dual process theorists”
(p. 261, Evans 2008). However, Evans’ modi?cation for a general dual
rative search.
system theory (i.e., a dual type theoretical framework) notes that even if
Broadly, our results are consistent with general notions
abstract reasoning requires the use of System 2 it would be a mistake to
of re?ectiveness suggesting that cognitive abilities are
assume that concrete contexts preclude its application, as is apparent in
the current protocol data.
associated with more careful, thorough, and elaborative

---

Judgment and Decision Making, Vol. 4, No. 1, February 2009
Cognitive abilities and superior risky choice
28
Table 5: A sample of coded protocol analysis from individuals with higher working memory, numeracy, and/or cogni-
tive re?ection scores (i.e., top quartile).
Choice Options
Protocol Analysis Sample: Higher Ability Verbalizations
Loss: $50 or 5% chance to My ?rst thought was that 4000 is a lot to lose (value high) even though it’s only
lose $4000
5% chance (probability low) and only losing 50 compared to that (max differ-
ence) is not very bad at all.
Gain: $275 or 20% chance My ?rst thought was that 20% chance is not likely to happen (probability low)
to win $900
and it was only 900 compared to B (max difference) which was 275 certain, so
that’s why I chose B.
Loss: $120 or 5% chance My ?rst thought was it’s only 5% chance (probability low) to lose 1600 and I
to lose $1600
have a 95% chance (recode probability) of not having lost anything.
Gain: $80 or 3% chance to My ?rst thought was I saw the 5600 dollars and I saw the 80 dollars but the 5600
gain $5600
dollars — there is still a chance for me to gain, it was really small (probability
low), but you never know. That’s why I chose B because 5600 dollars is a lot of
money (value high).
Loss: $275 or 20% chance My ?rst thought was with 20% chance of owing 900 dollars, that gives me
to lose $900
80% chance (recode probability) to not owe 900 dollars and I have pretty good
chances (probability high).
Gain: $150 or 30% chance I chose the 30% chance of getting $1080 over certain chance er, certain that you’re
to gain $1080
getting 150. 150 is not a whole much (value low), you know? That’s not a whole
much a lot of money and 1080 is a good amount bigger than 150 (max differ-
ence) even though there is only, I think there was only 30% chance of getting it
(probability low).
Loss: $200 or 1% chance I chose the 1% of $3000 because that’s really small (probability low), it’s 1 in
to lose $3000
100 (recode probability) of you actually losing $3000 compared to certain losing
whatever—300.
Loss: $50 or 50% chance I chose the $50 certain loss because it’s not a whole lot of money (value low),
to lose $400
compare that to, I think it was, 50% chance of losing $400 so that’s a pretty big
difference (max difference).
Gain: $50 or 5% chance to Uh I took the 5% chance of getting 4000 compared to 50 ‘cause 50 is really, really
win $4000
small compared to 4000 (max difference) and you have a 5% chance which is
pretty small (probability low) but, uh, if you actually do gain that you gain a lot
more than if you take 50.
— but not necessarily normative — cognition (Baron,
biguously demonstrates that abilities and expertise are as-
1985). Our results further suggest that early selection
sociated with adaptive cognition, such that superior per-
cognitive control mechanisms may play a role in re?ec-
formers will tend to rely on less elaborative search when
tiveness and superior task performance. Indeed, individ-
it is advantageous (Bröder, 2003; Ericsson, Prietula, &
uals who score higher on cognitive ability measures are
Cokely, 2007; Fasolo, Misuraca, & McCelland, 2003,
known to spend more time preparing for tasks (Sternberg,
Mata, Schooler, & Rieskamp, 2007; Shanteau, 1992).
1977) and also more elaborately and strategically encode
information, deliberatively building cognitive represen-
tations that better support subsequent task performance
4.2 Models of risky choice
(Baron, 1978; Cokely et al., 2006; Ericsson & Kintsch,
Expected value was a reliable as-if choice outcome
1995; Hertzog & Robinson, 2005; McNamara & Scott,
model. Yet process data indicated that even in highly sim-
2001; Vigneau, Caissie, & Bors, 2005). However, we
pli?ed lotteries expected value was only an as-if model,
caution against an interpretation that higher performing
which showed little relation to actual cognitive processes
individuals (or better decision processes) always search
(Payne & Braunstein, 1978). The priority heuristic also
or re?ect more (for a discussion of “less is more” in deci-
proved to be an inaccurate process (and choice-outcome)
sion making see Gigerenzer et al., 1999). Research unam-
model. This limitation may re?ect the large individual

---

Judgment and Decision Making, Vol. 4, No. 1, February 2009
Cognitive abilities and superior risky choice
29
Table 6: Intercorellations for main variables.
1
2
3
4
5
1. Expected-value choices
.
.
.
.
.
2. Cognitive re?ection test
.27*
.
.
.
.
3. Numeracy
.28*
.31**
.
.
.
4. Working memory
.16
.31**
.37*
.
.
5. Expected-value verbalizations
.25*
.00
–.13
–.06
.
6. Elaborative heuristic search
.32**
.23*
.36*
.25*
–.40**
Notes: * p < .05; ** p < .01
Table 7: Hierarchical linear regression analysis explaining expected-value choices.
Models and variables
?
R
R2
?R2
F
Model 1.
Expected-value calculations
.21
0.21
.04
.04
2.57
Model 2. Ability variables added
Expected-value calculations
.27*
.
.
.
.
Working memory span
.02
0.44
.19
.15
3.46*
CRT
.24*
.
.
.
.
Numeracy
.24*
.
.
.
.
Model 3. Number of considerations added
Expected-value calculations
.53**
.
.
.
.
Working memory span
–.01
.
.
.
.
CRT
.09
.66
.44
.24
22.23**
Numeracy
.09
.
.
.
.
Elaborative heuristic search
.63**
.
.
.
.
Note: * p < .05; ** p < .01
differences in elaborative search elicited by the current
Schooler, & Mata, 2008; Schooler & Hertwig, 2005).
task environment. These data provide further evidence
on the limitations and boundary conditions of the priority
heuristic (Birnbaum, 2008; Hilbig, 2008; Johnson et al.,
5 Conclusions
2008). It should be noted that this limitation is apparent
only because the priority heuristic makes very exact pre-
People higher in working memory span, cognitive re?ec-
dictions at both the cognitive process and choice-outcome
tiveness, and those with greater skill in comprehending
levels, which is a useful and unique feature among risky
and transforming probabilities often made choices con-
choice models. Results indicate that more precise pro-
sistent with expected value; however, protocol analy-
cess modeling of risky choices with the priority heuris-
ses revealed that they did not commonly use expected-
tic or another computational model would require at least
value calculations to arrive at those choices (Payne &
one parameter that creates variation in search and stop-
Braunstein, 1978; Payne et al., 1993). Instead, cognitive
ping rules. However, accurate modeling of psychologi-
abilities were related to relatively simple yet elaborative
cally plausible mechanisms for the regulation of heuristic
heuristic search processes. The results accord with exam-
search will require greater speci?cation and research at
ples showing that good decisions can be made with sim-
the intersection of task environments and cognitive ca-
ple processes (Gigerenzer & Goldstein, 1996; Gigerenzer
pacities (Bröder, 2003; Gaissmaier & Schooler, 2008;
et al., 1999), although results also provide additional ev-
Gaissmaier, Schooler, & Rieskamp, 2006; Gaissmaier,
idence that even heuristic search processes can require

---