Challenging some common beliefs: Empirical work within the adaptive toolbox metaphor

Judgment and Decision Making, Vol. 3, No. 3, March 2008, pp. 205–214
Challenging some common beliefs: Empirical work within the
adaptive toolbox metaphor
Arndt Bröder?
University of Bonn and Max Planck Institute for Research on Collective Goods
Ben R. Newell
University of New South Wales
Abstract
The authors review their own empirical work inspired by the adaptive toolbox metaphor. The review examines factors
in?uencing strategy selection and execution in multi-attribute inference tasks (e.g., information costs, time pressure,
memory retrieval, dynamic environments, stimulus formats, intelligence). An emergent theme is the re-evaluation of
contingency model claims about the elevated cognitive costs of compensatory in comparison with non-compensatory
strategies. Contrary to common assertions about the impact of cognitive complexity, the empirical data suggest that
manipulated variables exert their in?uence at the meta-level of deciding how to decide (i.e., which strategy to select)
rather than at the level of strategy execution. An alternative conceptualisation of strategy selection, namely threshold
adjustment in an evidence accumulation model, is also discussed and the dif?culty in distinguishing empirically between
these metaphors is acknowledged.
Keywords: strategy selection, contingency model, cognitive costs
1 Introduction
serviceable solutions to individual problems.
To illustrate the basic idea we describe the operation
Over (2003) points out that many evolutionary psycholo-
of two of the heuristics contained in the toolbox. Imagine
gists have used tools as vivid metaphors for characteris-
you are facing a choice between two alternatives — such
ing the mind as comprising a range of speci?c modules.
as two companies to invest in — and your task is to pick
For example, Cosmides and Tooby (1994) suggested that
the one that is better with regard to some criterion (e.g.,
the mind be viewed like a Swiss army knife, with indi-
future returns on investments). “Take-the-Best” (TTB)
vidual blades specialised for particular “survival-related”
is designed for just such a situation. TTB operates ac-
tasks. In a similar vein, Gigerenzer, Todd and the ABC
cording to two principles. The ?rst — the recognition
Group (1999) proposed an “adaptive toolbox” containing
principle — states that for any decision made under un-
a variety of special tools for different tasks. Their idea
certainty, if only one amongst a range of alternatives is
is that the mind has evolved mechanisms or heuristics
recognised, then the recognised alternative will be cho-
that are suited to particular tasks, such as choosing be-
sen. When this ?rst principle can be relied on people
tween alternatives, categorising items, estimating quanti-
are said to be using the Recognition Heuristic (RH) —
ties, selecting a mate, judging habitat quality, even deter-
i.e., choosing objects that they recognise (Goldstein &
mining how much to invest in one’s children. Gigerenzer
Gigerenzer, 2002). The second principle is invoked when
and Todd argue that just as a car mechanic uses speci?c
more than one alternative is recognised, and the recogni-
wrenches, pliers and spanners in maintaining a car en-
tion principle cannot provide discriminatory information.
gine rather than hitting everything with a hammer, so too
In such cases, people are assumed to have access to a
the mind relies on unique one-function devices to provide
reference class of cues or features, which are searched
in descending order of feature validity (search rule) until
?Ben Newell acknowledges the support of the Australian Research
Council (Grant: DP 0558181) and the University of New South Wales
one that discriminates between alternatives is discovered.
for awarding him the John Yu Fellowship to Europe. Both authors
Search then stops (stopping rule) and this single best dis-
would also like to thank the Max Planck Institute for Research on
criminating feature is used to make the choice (decision
Collective Goods for hosting Ben Newell’s visit and the symposium.
rule). The algorithm is thus non-compensatory because,
Corresponding author: Arndt Bröder, Dept. of Psychology, Univer-
sity of Bonn, Kaiser-Karl-Ring 9, D-53111 Bonn, Germany. Email:
rather than using all discriminatory pieces of information
broeder@uni-bonn.de.
(as a compensatory model like linear regression would),
205

---

Judgment and Decision Making, Vol. 3, No. 3, March 2008
Common beliefs versus empirical results
206
it bases its choice on a single piece (Gigerenzer & Gold-
burden the limited capacity (Schneider & Shiffrin, 1977).
stein, 1996).
Metacognition — deciding how to decide — is concerned
These simple steps for searching, stopping and decid-
with allocating capacity to decision tasks and almost cer-
ing might seem rather trivial, but Gigerenzer and Gold-
tainly consumes cognitive capacity itself. However, this
stein (1996) showed convincingly that the TTB algorithm
latter aspect has hitherto been neglected in decision re-
is as accurate — and sometimes even slightly more ac-
search and the toolbox approach. The second dimension
curate — than more computationally complex and time
around which we examine the empirical evidence is the
consuming algorithms. These initial results, from a task
“target” of the respective studies: different studies fo-
in which the goal was to decide which of two cities had
cus on the search rule, the stopping rule, or the decision
a higher population, were replicated in a variety of real-
rule people use. Although these aspects are closely in-
world environments ranging from predicting professorial
tertwined empirically (e.g., Bröder, 2003), most studies
salaries, to the amount of sleep engaged in by different
focus on one or two aspects for methodological reasons.
mammals (Czerlinski, Gigerenzer, & Goldstein, 1999).
We will ?rst report studies concerning adaptivity and the
The toolbox, however, is one of different metaphors used
use of simple heuristics and then turn to results relevant
to characterize intelligent decision making. On one hand,
for the question of capacity limitations, automatization,
the toolbox with its incorporation of the modularity as-
and metacogntition.
sumption challenges the idea of the mind as containing
a “master tool” that comes as a general problem solver.
On the other hand, the toolbox idea itself has also been
3 Do people select simple and less
challenged by theoretical arguments. For example, some
simple heuristics adaptively?
authors claim that simple heuristics may not be so simple
in the ?rst place because they need a vast amount of pre-
Payne, Bettman, and Johnson (1993) report many results
computation (e.g., for constructing a cue-search hierar-
that suggest adaptive strategy changes contingent on task
chy, Juslin & Persson, 2002). Others conjecture that com-
demands. For example, time pressure or the dispersion
pensatory strategies may not be so costly as the toolbox
of attribute weights clearly in?uenced information search
and common wisdom in decision research presuppose
behavior in a preferential choice task (Payne, Bettman,
(e.g., Chater, Oaksford, Nakisa, & Reddington, 2003).
& Johnson, 1988). Rieskamp and Hoffrage (1999) con-
Theoretical objections to the toolbox are summarized and
?rmed these results in a mutli-attribute inference task.
discussed in Newell and Shanks (2007). Another chal-
Under time pressure, participants search for less infor-
lenge is empirical: Do people use different tools adap-
mation and do so more attribute-wise (rather than option-
tively, and more speci?cally, do they use simple heuristics
wise) which is similar to the search rule predicted by lex-
like RH and TTB? In this article, we will review empiri-
icographic heuristics like TTB. Being forced into simple
cal work from our labs that addresses this latter question
processing by time pressure may not be a strong argu-
and asks which factors affect the strategies people select.
ment in favour of adaptive strategy selection, however,
Whereas the goal is of course not new, we are convinced
so other investigators varied the nominal costs of infor-
that our results have some new implications for the tool-
mation purchases in a hypothetical stock market game
box metaphor as well as for multi-attribute decision re-
(Bröder, 2000; Newell & Shanks, 2003). In this task,
search in general.
participants make repeated stock purchase decisions be-
tween hypothetical companies that are described by four
binary cues (e.g., Turnover growth in last months —
2 Organization of the review
yes vs. no). Typically, cue values are hidden and have
to be actively uncovered by clicking the ?elds with the
Newell and Bröder (2008) mentioned several facts and
computer mouse. Participants are free to uncover as
topics about human cognition that have to be addressed
much information as they want in any sequence. This
by theories of decision making, namely (1) capacity
MouseLab-like procedure (see Payne et al., 1988) al-
limitation, (2) automaticity vs. controlled processing,
lows for outcome-based strategy assessment based on the
(3) learning, (4) categorization, and (5) metacognition.
choices as well as monitoring of the information acquisi-
These areas of interest and the question of whether peo-
tion process. Newell and Shanks (2003) found that rais-
ple adaptively choose strategies constitute one dimension
ing the costs for information search led to a lesser amount
of our review. Our empirical work predominantly cov-
of purchases, but still, participants on average bought
ers the question of adaptivity and areas (1), (2), and (5)
more cue information than “necessary” for performing a
which are closely interconnected. Whereas capacity lim-
simple lexicographic strategy. Hence, participants did not
itations mainly concern controlled, effortful, and perhaps
generally adhere to the stopping rule dictated by TTB. In
serial processes, any degree of automatization will un-
another study of Newell, Weston, and Shanks (2003, Exp.

---

Judgment and Decision Making, Vol. 3, No. 3, March 2008
Common beliefs versus empirical results
207
2), 38% of the participants even went on purchasing a
issue: How people learn cue validities and construct cue-
cue that was costly but objectively useless, hence using a
search hierarchies. As noted earlier, Juslin and Persson
clearly maladaptive stopping rule. These studies and ad-
(2002) argued that a good deal of the “simplicity” inher-
ditional asymmetries in favor of compensatory decision
ent in simple heuristics comes from the massive amounts
making (see below) suggest that there is an initial pref-
of precomputation required to construct cue hierarchies.
erence for being “well-informed” before making a deci-
Newell, Rakow, Weston and Shanks (2004) sought to
sion, at least as long as information is easy to obtain and
gain some insight into how people learned cue validi-
the task is not too complex with respect to the number of
ties and search rules by using experimental designs in
options and/or attributes. There is converging evidence
which participants could purchase cues in any, rather than
from studies testing the assumed noncompensatory na-
a ?xed order. Following Martignon and Hoffrage (1999)
ture of the RH which show that participants rarely ignore
we noted that the overall usefulness of a cue must take
information which is available in addition to the recogni-
account of both its validity and its redundancy — or abil-
tion cue (Bröder & Eichler, 2006; Newell & Fernandez
ity to discriminate between two options in two-alternative
2006; Newell & Shanks, 2004; Pohl, 2006; Richter &
forced choice task. More useful cues are those that can
Späth, 2006). The process model of the RH clearly states
frequently be used to make an inference (i.e., have a high
that “if one object is recognized and the other is not, the
discrimination rate); and, when used, usually point in the
inference is determined; no other information about the
correct direction (i.e., have a high validity).
recognized object is searched for and, therefore, no other
In support of this assertion, Newell et al. (2004) found
information can reverse the choice determined by recog-
that, in a simulated stock market environment involving
nition” (Goldstein & Gigerenzer 2002, p. 82); however,
a series of predictions about pairs of companies, partic-
even under conditions that are ideal for the RH (high
ipants’ pre-decisional search strategies conformed to a
recognition validity, natural recognition knowledge, in-
pattern that revealed sensitivity to both the validity and
ferences from memory), the decisions of 50% of the par-
discrimination rate of cues. Given suf?cient practice in
ticipants were affected by additional cue knowledge in a
the environment, participants searched through cues ac-
study by Pachur, Bröder, & Marewski (in press). These
cording to how “successful” they were for predicting the
results suggest that lexicographic stopping rules may be
correct outcome (see Martignon & Hoffrage, 1999, for a
the exception rather than the rule in decision making.
detailed discussion and de?nition of “success” — it is a
Bröder (2000) focused on the decision rule people used
function of the validity and discrimination rate of cues).
and also manipulated the nominal costs for information
Thus, rather than using a “validity” search rule — as pre-
purchase. An outcome-based classi?cation procedure
scribed by TTB and enforced in some experimental tests
suggested that the choices of about 65% of participants
— participants tended to use a “success” search rule. (See
were compatible with TTB under high search cost condi-
also Rakow, Newell, Fayers, & Hersby, 2005). This ini-
tions. A subsequent experiment con?rmed that this high
tial work on cue search needs to be supplemented by more
percentage (which contrasted with low TTB percentages
extensive explorations of potential mechanisms for learn-
in other studies) was in fact caused by the information
ing and implementing cue hierarchies.
costs, but not by other factors such as outcome feedback,
We noted earlier that in experiments with explicit
or successive information retrieval. In addition, search
costs participants might be deterred from further search
behavior corresponded well to the decision rule partici-
through and acquisition of information simply because of
pants used. Hence, both our labs showed that stopping
high nominal costs. To overcome this possibility Bröder
and/or decision rules were sensitive to search costs to a
and colleagues kept the nominal search costs identical
certain degree, probably re?ecting adaptivity. However,
in different conditions of their experiments but varied
several criticisms can be raised: First, there was no for-
the payoff functions to yield different expected payoffs
mal assessment of expected payoffs in these studies and
in different experimental conditions: some environments
hence, strategy changes might not have been “adaptive”
were compensatory, meaning that the costs spent on addi-
but rather caused by stinginess. That is, high nominal
tional cues were compensated by better accuracy and in-
costs of information may simply have deterred partici-
creased payoff; and some were noncompensatory, so that
pants from purchasing information despite its potential
the costs for additional cues would in the long run exceed
value for good decisions. This would demonstrate sen-
their utility for making better decisions. The empirical
sitivity to costs, but not necessarily adaptive behavior.
question of adaptivity was now whether people would be
Second, in Bröder’s (2000) study, information about cues
able to ?gure out the appropriate strategies in the respec-
could only be purchased in the order of their validities,
tive environments. The ?lled circles in Figure 1 sum-
probably boosting the use of TTB-like strategies.
marize the proportions of participants classi?ed as using
Limiting participants to searching information in one
TTB’s decision rule across a range of 11 experimental
particular order overlooks a crucial yet under-researched
conditions from several studies (Bröder, 2003; Bröder

---

Judgment and Decision Making, Vol. 3, No. 3, March 2008
Common beliefs versus empirical results
208
0
8
should be investigated as variables causing the individ-
?
0
ual differences, for example mind-sets or spillover effects
7
r
s
e
from routines established in similar tasks.
s
0
6

u
Recently, Bröder and Schiffer (2006a) reported results
?
?
B
0
T
?
which qualify the optimistic notion of adaptivity docu-
5
?
T
?
f
mented by the ?lled circles in Figure 1. Three of the open
0
?

o
4
?
squares in the ?gure do not ?t into the picture. These
e
g
?
0
experimental conditions have in common that the pay-
ta
3
n
?
off structure of the environment had changed after par-
e
0
?
2
r
c
ticipants had become used to another environment be-
?
Pe
0
fore. That means, the low percentage of TTB users in the
1
noncompensatory environment re?ects the fact that this
0
group had been exposed to a compensatory payoff struc-
0.4
0.6
0.8
1.0
1.2
1.4
1.6
ture before. Obviously, most participants adhered to a de-
cision strategy established as a routine before. These mal-
Ratio of expected Payoffs TTB vs. FR
adaptive routine effects were only marginally relieved by
a hint about the change or even by a switch to a similar but
Figure 1: Adaptive strategy selection demonstrated by the
different task. This observation contrasts with most par-
percentage of participants classi?ed as TTB users in the
ticipants’ obvious ability to adapt ?exibly to a new task.
stock market game as a function of on the expected payoff
We conclude that different mechanisms for strategy se-
of TTB relative to a compensatory strategy. Filled circles
lection may be at work when people are confronted with
are experimental conditions in which the task was new to
a new task than when they routinely use a strategy. Iner-
participants, and they show a clear adaptive trend. Open
tia effects like these are predicted by Rieskamp’s (2006)
squares depict the maladaptive routines after the environ-
reinforcement learning model.
mental payoff structure had changed (Bröder & Schiffer,
One additional observation was made repeatedly in the
2006a), and the triangle shows the high cognitive load
stock market paradigm: There was an initial preference
condition of Bröder and Schiffer (2003a).
for compensatory decision making and deep information
search (Bröder, 2000; 2003; Newell & Shanks, 2003;
Newell, Weston & Shanks, 2003; Rieskamp & Otto,
& Eichler, 2001; Bröder & Schiffer, 2003a; 2006a) as a
2006). Compensatory strategies were even somewhat
function of the expected payoff of TTB relative to that of
more subject to maladaptive routines than TTB (Bröder
a compensatory strategy known as “Franklin’s rule” (FR)
& Schiffer, 2006a). We conjecture that participants feel
which is a weighted additive rule. It is easy to see that
on the “safe” side if they use all information, and they
there is an adaptive trend (r = .83) which shows that the
have to learn actively whether information can safely be
majority of people tend to use appropriate strategies in
ignored. Many learn to adapt their stopping and/or deci-
compensatory (left of “1”) and noncompensatory (right
sion rule, others keep on buying information even when
of “1”) environments. However, adaptivity is not perfect
it is of no use (Newell, Weston & Shanks, 2003).
since in all cases, there is a signi?cant percentage of peo-
To summarize the adaptivity results: The toolbox idea
ple not using the appropriate strategy.
is corroborated in principle because many participants
Hence, the results of both labs converge on similar con-
adapt to payoff schemes. This supplements Payne et
clusions: There is a certain extent of adaptivity in strat-
al.’s (1993) work which showed that strategy selection is
egy choice concerning search, stopping as well as deci-
contingent on task demands in the domain of preferen-
sion rules. Participants are not only abhorred by costs,
tial choices. In addition to the formal similarity between
but they seem able to ?gure out payoff structures (even if
multi-attribute preferential choice and multiple-cue prob-
differences are subtle — see Figure 1) and select the strat-
abilistic inferences, these empirical similarities support
egy accordingly. However, there are large individual dif-
the idea of similar cognitive processes (or at least simi-
ferences in strategy selection. The attempt to ?nd person-
lar principles) in both domains. Note, however, that the
ality dimensions as correlates of strategy preferences has
observation that people appear to choose among heuris-
not been successful so far, even though we tried 15 plau-
tics of varying complexity could also be reinterpreted
sible dimensions (see Bröder, in press, for an overview).
as a threshold adjustment in an evidence accumulation
However, it is yet an open question whether the differ-
metaphor (e.g., Lee & Cummins, 2004; Newell, 2005).
ent strategy preferences diagnosed in a one-shot assess-
Evidence accumulation models assume individual deci-
ment of an experiment will turn out to be stable across
sion thresholds of evidence. Information search contin-
tasks and situations. If not, then states rather than traits
ues until a threshold in favour of one option has been

---

Judgment and Decision Making, Vol. 3, No. 3, March 2008
Common beliefs versus empirical results
209
crossed and a decision is made. Thresholds can be set
tive effort needed to reach a decision using a particular
at a continuum from strict to lenient. Lenient criteria im-
strategy is a function of the number and type of opera-
ply fast and frugal information searches, whereas strict
tors (productions) used by that strategy, with relative ef-
criteria demand more information before making a deci-
fort levels of various strategies contingent on task envi-
sion. Hence, “strategies” like TTB or WADD can also
ronments” (Payne et al., 1993, p. 14). Both Beach and
be viewed as endpoints of a continuum that de?nes one
Mitchell (1978) and Payne et al. (1993) admitted that the
general process of decision making. Rather than select-
exact nature of the deliberation process is unknown and
ing strategies, the decision maker might adjust thresh-
subject to further research, and the latter authors specu-
olds. At the moment, data do not allow for a clear de-
lated about different degrees of sophistication of this pro-
cision between the model classes because apparent strat-
cess. This reasoning about the apparent costs of compen-
egy switches can be reinterpreted as criterion shifts or
satory strategies is explicitly incorporated in the adaptive
vice versa (see Hausmann & Läge, 2008). Large indi-
toolbox metaphor and its rhetoric in which compensatory
vidual differences in adaptivity remain, and the general
strategies are associated with theories that “assume the
preference for compensatory deciding observed in studies
human mind has essentially unlimited demonic or super-
on TTB and the RH casts doubt on the assumption that
natural reasoning power” (Gigerenzer & Todd, 1999, p.
simple heuristics are the default mode of probabilistic in-
7). This image is contrasted against the fast and frugal
ferences — at least in tasks with cue information that is
heuristics.
easily accessible. Furthermore recent work suggests that
The emphasis on the execution costs of various deci-
a “uni?ed model” which treats TTB and more compen-
sion strategies promoted by the contingency model and
satory strategies as special cases of the same sequential
the adaptive toolbox leads to a simple and straightforward
sampling process provides an interpretable account of in-
prediction: These relative costs should decrease with in-
dividual differences in participants’ judgments. Although
creased cognitive capacity. Or in other words, greater
such a threshold model is more complex than “parameter-
cognitive capacity should reduce the pressure to use sim-
free” models like TTB, it is preferred to simpler models
plifying strategies like TTB (e.g., Beach & Mitchell,
on the grounds of model ?t criteria (e.g., minimum de-
1978; pp. 445–446). To our great surprise, in a ?rst
scription length) (Newell, Collins, & Lee, 2007).
study on that topic, our results were opposite to this pre-
diction and suggest a re-evaluation of the contingency
4 Capacity limitations, automatic- model. In the study of Bröder and Eichler (2001) par-
ticipants invested in the stock market game and subse-
ity, and metacognition
quently ?lled out an intelligence test. After classifying
participants’ decision strategies, results showed that TTB
In accordance with the multiple-strategy assumption in
users were slightly more intelligent than compensatory
decision research, Beach and Mitchell (1978) formulated
decision makers! This was opposite to the expectation
an early attempt to de?ne criteria that might govern strat-
from the contingency logic which predicts simpler strate-
egy selection. In their contingency model “strategy se-
gies will be associated with less capacity. Only after a
lection is viewed as a compromise between the press for
post-hoc analysis of the game’s payoff structure, we real-
more decision accuracy as the demands of the decision
ized that there had been a relatively subtle (10%) advan-
task increase and the decision maker’s resistance to the
tage in the expected payoff of TTB as compared to the
expenditure of his or her personal resources” (Beach &
compensatory strategy WADD in this task. In two sub-
Mitchell, 1978, p. 447). They classi?ed compensatory
sequent experiments, we replicated the small, but consis-
strategies as “analytic” and noncompensatory ones as less
tent superiority of TTB users with respect to intelligence
analytic and assumed that the “use of a less analytic strat-
in environments with noncompensatory payoff structures
egy requires, on the average, less expenditure of personal
(Bröder, 2003). This suggests that cognitive capacity —
resources than does use of a more analytic strategy” (p.
as indexed by intelligence — is not consumed by strat-
448). This intuitively plausible assumption has guided
egy execution, but rather by strategy selection. Since in-
a signi?cant part of research, for example Payne et al.’s
telligence can be related to many other causal variables,
(1993) systematic analysis of adaptive decision making.
we also manipulated cognitive capacity experimentally in
Christensen-Szalanski (1978; 1980) as well as Chu and
a subsequent experiment by imposing a very attention-
Spires (2003) supported the assumption by showing that
demanding secondary task on half of the participants dur-
it ?ts people’s intuitions. Payne et al. (1993) extended
ing their decisions (they had to count the occurrences of
and speci?ed the model further by deriving a measure for
the number “nine” in a stream of digits and were probed
the cognitive costs caused by strategies: They counted
in random intervals; Bröder & Schiffer, 2003a). In the
the elementary information processing steps necessary to
environment used, there was a very subtle payoff advan-
perform a decision rule and proposed that “the cogni-
tage for TTB, and results showed 60% TTB users in the

---

Judgment and Decision Making, Vol. 3, No. 3, March 2008
Common beliefs versus empirical results
210
condition without cognitive load, whereas only 26% used
that selects strategies. But note that these conclusions
TTB in the condition with heavy cognitive load. The
may be valid only for situations as complex as our exper-
others were classi?ed as using compensatory strategies.
iments, in which we only used up to three options with
This is again contrary to the expectation of the contin-
up to six cues at most. Furthermore, the cues used in
gency model and again supports another conclusion: At
our experiments were almost exclusively binary, and it is
least in our paradigm, the costs of strategy execution
conceivable that multi-valued cues are harder to process.
do not seem to differ much between TTB and compen-
Perhaps, processing capacity limits were not reached, and
satory strategies, and participants were able to use com-
strategy execution costs may become a severe factor only
pensatory strategies even under conditions of heavy cog-
with very complex decision situations.
nitive load. Rather, the cognitive load impaired partici-
However, after having developed this interpretation,
pants’ ability to ?gure out the payoff structure of the en-
one other result of Bröder and Schiffer (2006b) qual-
vironment and to choose the appropriate heuristic.
i?es this conclusion. In this study, participants again
This interpretation is also compatible with results re-
had to work on various capacity-demanding secondary
ported by Bröder and Schiffer (2006a) demonstrating
tasks during decision making. In contrast to the study
massive routine effects in the use of decision strategies.
mentioned before, this boosted the use of a TTB heuris-
Routine effects have been known as “Einstellung” effects
tic at the expense of compensatory strategies. The im-
for a long time in the psychology of thinking (Luchins &
portant difference here was that all cue information had
Luchins, 1959). Although Betsch and co-workers have
to be retrieved from memory rather than from the com-
demonstrated routines in repeated decisions before (see
puter screen. In a virtual criminal case, participants had
Betsch and Haberstroh, 2005, for a review), these demon-
learned various details about suspects which they later
strations concerned the choice of routine options rather
used for decisions about the probability of being the per-
than strategies. Bröder and Schiffer (2006a) based their
petrator. For example, they learned about aspects of the
research on these observations, but they demonstrated
suspects’ clothing and later received information about
that routines are retained also at the level of strategies,
witness reports that established a clear cue validity hierar-
even in a changing environment where they become mal-
chy. In a series of paired comparisons, they had to decide
adaptive (but see Rieskamp, 2008, for an alternative in-
which suspect was the perpetrator with a higher proba-
terpretation). The combination of a quick adaptation to
bility. Earlier studies in this memory-search paradigm
new environments, but slow adaptation to changing en-
had already shown that TTB is much more prevalent here
vironments suggests that strategy execution can become
than in screen-based information presentation. This sug-
routinized. Strategy selection, on the other hand, may re-
gests that memory retrieval is costly and promotes early
quire a costly re-examination of the environment in order
stopping rules in the same way as high explicit costs
to adjust the strategy accordingly. This selection process
promote early stopping of information search in screen-
cannot become routinized, but it always requires delib-
based tasks (Bröder, 2000; 2003; Newell & Shanks,
erate processes. The apparently routinized strategy ex-
2003). Furthermore, the costs of retrieval are apparently
ecution was re?ected in the time needed for each deci-
less severe when information is stored in a pictorial rather
sion which was much shorter for later trials in the task
than a verbal format (Bröder & Schiffer, 2003b; 2006b)
than for the ?rst 10 to 20 trials. In the ?rst phase of the
which is compatible with knowledge from cognitive psy-
experiments, most participants adaptively chose the ap-
chology (Paivio, 1991). However, recent work compar-
propriate strategy. When the environment changed after
ing judgments made on the basis of pictorial and verbal
80 trials, the reaction times did not increase again (even
information in screen-based tasks found no evidence for
after a hint about the change), and the result was a mal-
a difference in TTB use as a function of format. It ap-
adaptive trend to stick to one’s established strategy. This
pears then that the format effect is dependent on inducing
stickiness was even more pronounced for compensatory
memory retrieval costs (Newell, Collins, & Lee, 2007).
strategies. We hypothesize that the meta-decision how to
Bröder and Gaissmaier (2007) reanalyzed response
decide was only executed at the beginning of the exper-
times from published studies and found evidence that
iment (consuming time and capacity), whereas the exe-
people who were classi?ed as TTB users based on de-
cution was routinized after a few trials, probably without
cision outcomes apparently also used TTB’s stopping
consuming cognitive capacity further. This routinization,
rule: The response times increased monotonically with
however, happens at the expense of ?exibility (Schneider
the number of cues that had to be retrieved for perform-
& Shiffrin, 1977).
ing a lexicographic strategy. Other explanations (simi-
Hence, the contingency model’s and the toolbox’
larity of options, dif?culty of decisions) accounted for
rhetoric emphasis on processing costs of strategies and
less variance in the decision times than the assumption
heuristics may be mistaken, since the actual capacity-
of this simple stopping rule. In one experiment, there
consuming process is apparently the meta-decision rule
were apparently several participants with an even simpler

---

Judgment and Decision Making, Vol. 3, No. 3, March 2008
Common beliefs versus empirical results
211
strategy called “Take The First” who retrieved cues in the
pirically testing evidence accumulation thresholds are ar-
order of retrieval ease (as de?ned by the learning proce-
guably more advanced and established than are models of
dure) and showed response times consistent with a stop-
“strategy selection” (e.g., Vickers, 1979). Thus although
ping rule terminating search after one discriminating cue
the two model classes may currently be dif?cult to dis-
had been found.
tinguish at the data level, future investigations may deter-
To conclude: If information is available on the screen
mine the superior metaphor.
without burdening working memory too much, cogni-
One main result emerging from the synopsis of the
tive processing costs for strategies are not a serious fac-
work reported here is a fundamental re-evaluation of the
tor, the format of stimuli materials have little effect, and
contingency model and its successors (Beach & Mitchell,
cost differences between strategies like TTB and WADD
1978; Christensen-Szalanski, 1978; 1980; Chu & Spires,
are neglible. Only if search costs are explicit are stricter
2003; Payne et al., 1993). The wide-spread credo is
stopping (and decision) rules employed. Information in-
that compensatory strategies are cognitively more costly
tegration also does not seem to be costly, a conclusion
than noncompensatory ones. On the other hand, they
demonstrated by the performance of participants in the
are believed to be more accurate. Consequently, there
high cognitive load condition (counting nines) of Bröder
is a con?ict demanding a compromise between the two.
and Schiffer’s (2003a) experiment, described earlier, in
However, the second conviction (higher accuracy) has
which 60% of participants probably used a compensatory
been called into question by the toolbox proponents who
rule. Memory retrieval, on the other hand, appears to
showed via simulations that noncompensatory rules can
cause cognitive costs and promotes early stopping rules,
be as accurate as compensatory ones (Gigerenzer et al.,
but costs can be reduced by the use of integrated, pictorial
1999). This clearly came as a surprise and has been repli-
stimuli. Whereas the distinction between inference from
cated and investigated more thoroughly since then (Hog-
givens vs. inferences from memory is clear for controlled
arth & Kareleia, 2005, 2007). Interestingly, the toolbox
laboratory experiments, it may be less so in the applied
rhetoric on the other hand relies heavily on the assump-
everyday context. Here, we will often confront situations
tion that compensatory strategies are cognitively costly.
that involve both kinds of information retrieval. For in-
As all of our results suggest, this does not seem to be
stance, consumer choices may depend on an attribute ma-
the case. Compensatory strategies were performed under
trix provided in a “Consumer report” magazine as well
high cognitive load and they were subject to “thought-
as on facts we remember about the options. Hence, ac-
less” routines. Hence, multiple pieces of information can
tual decisions probably involve a mixture of information
be combined compensatorily without the “unlimited re-
sources and hence, a mixture of different cognitive costs.
sources” postulated for “rational demons” (see Gigeren-
zer & Todd, 1999). Whether this is done sequentially
in a simple random walk process (e.g., Lee & Cum-
5 Conclusions
mins, 2004) or by simultaneous constraint satisfaction
in a network model (e.g., Glöckner & Betsch, 2008) or
In this review, we focused on our own empirical work
some other way remains open to question. What can be
that was stimulated by and took place within the adaptive
costly is information search where costs are either deter-
toolbox metaphor. Since we did not report on numer-
mined by extrinsic (time pressure) or intrinsic (memory
ous other studies conducted within this framework, it is
retrieval) factors. We do not want to suggest that the exe-
fair to conclude that the toolbox has been extremely fruit-
cution of compensatory strategies is never costly: Several
ful in reanimating the interest in adaptive multi-attribute
studies have shown that the order of presentation, the pre-
decision making, supplementing Payne et al.’s (1993)
sentation format (numerical vs. verbal), or the similarity
work on preferences with work on inferences. Because
of alternatives have a strong in?uence on the way people
metaphors are not “correct” or “wrong” per se (they are
assess information (e.g. Schkade & Kleinmuntz, 1994;
all wrong, as Ebbinghaus [1885] already noted), they
Stone & Schkade, 1994), presumably re?ecting different
have to be evaluated by their fruitfulness. In this respect,
levels of processing ease. Furthermore, there will cer-
the toolbox fares quite well. Whether the box crammed
tainly be costs in very complex situations with many al-
with disparate tools is a more adequate metaphor than
ternatives and attributes. However, our results suggest
an “adjustable spanner” (Newell, 2005) remains to be
that the cognitive costs for strategy execution may have
seen. However, the success of evidence accumulation
been overestimated in relation to the costs for strategy se-
models in many other areas of cognition leads us to be
lection in moderately complex situations.
optimistic that they can perhaps also be fruitfully ap-
A closely related result is that enhanced capacity in-
plied to the more “controlled” processes in the decision
creased the proportion of people using simple heuristics
making domain (Busemeyer & Townsend, 1993: Wall-
— in environments in which they were appropriate! This
sten & Barton, 1982). Techniques for specifying and em-
was true for intelligence (Bröder, 2003) as well as for free

---

Judgment and Decision Making, Vol. 3, No. 3, March 2008
Common beliefs versus empirical results
212
working memory capacity (Bröder & Schiffer, 2003a).
sessment in multi-attribute decision research. Journal
Since these factors had no direct effects on strategy ex-
of Behavioral Decision Making, 16, 193–213.
ecution but rather on the adaptivity of the strategy use,
Bröder, A. & Schiffer, S. (2003b). “Take The Best” ver-
we conclude that the decision how to decide (or which
sus simultaneous feature matching: Probabilistic infer-
strategy to select) is the most demanding task in a new
ences from memory and effects of representation for-
decision situation. Although there has been some specu-
mat. Journal of Experimental Psychology: General,
lation about this deliberation process (Payne et al., 1993),
132, 277–293.
it has been neglected as a target of research. Probably,
Bröder, A. & Schiffer, S. (2006a). Adaptive ?exibility
the empirical investigation of the rules used is challeng-
and maladaptive routines in selecting fast and frugal
ing enough, and researchers have avoided adding another
decision strategies. Journal of Experimental Psychol-
level of complexity. We argue that the selection process
ogy: Learning, Memory, & Cognition, 32, 904–918.
is the crux of the matter since it consumes cognitive re-
Bröder, A. & Schiffer, S. (2006b). Stimulus format and
sources. Without modeling this demanding process, any
working memory in fast and frugal strategy selection.
theory of tool selection or threshold adjustment remains
Journal of Behavioral Decision Making, 19, 361–380.
incomplete.
Busemeyer, J. R., & Townsend, J. T. (1993). Decision
?eld theory: A dynamic-cognitive approach to deci-
sion making in an uncertain environment. Psychologi-
References
cal Review, 100, 432–459.
Chater, N., Oasksford, M., Nakisa, R., & Redington, M.
Beach, L. R., & Mitchell, T. R. (1978). A contingency
(2003). Fast, frugal, and rational: How rational norms
model for the selection of decision strategies. Academy
explain behavior. Organizational Behavior and Hu-
of Management Review, 3, 439–449.
man Decision Processes, 90, 63–86.
Betsch, T., & Haberstroh, S. (2005). Preface. In T. Betsch
Christensen-Szalanski, J. J. J. (1978). Problem solving
& S. Haberstroh, The routines of decision making (pp.
strategies: a selection mechanism, some implications
ix-xxv). Mahwah, NJ, US: Lawrence Erlbaum Asso-
and some data. Organizational Behavior and Human
ciates.
Performance, 22, 307–323.
Bröder, A. (2000). Assessing the empirical validity of
Christensen-Szalanski, J. J. J. (1980). A further exami-
the “Take The Best”-heuristic as a model of human
nation fo the selection of problem solving strategies:
probabilistic inference. Journal of Experimental Psy-
the effects of deadlines and analytic aptitudes. Organi-
chology: Learning, Memory, and Cognition, 26, 1332–
zational Behavior and Human Performance, 25, 107–
1346.
122.
Bröder, A. (2003). Decision making with the “adaptive
Chu, P. C., & Spires, E. E. (2003). Perceptions of accu-
toolbox”: In?uence of environmental structure, intelli-
racy and effort of decision strategies. Organizational
gence, and working memory load. Journal of Experi-
Behavior and Human Decision Processes, 91(2), 203–
mental Psychology:Learning Memory, and Cognition,
214.
29, 611–625.
Cosmides, L., & Tooby, J. (1994). Beyond intuition and
Bröder, A. (in press). The quest for take the best - Insights
instinct blindness: Toward an evolutionarily rigorous
and outlooks from experimental research. To appear in
cognitive science. Cognition, 50, 41–77.
P. Todd, G. Gigerenzer, & the ABC Research Group,
Czerlinski, J., Gigerenzer, G., & Goldstein, D. G. (1999).
Ecological rationality: Intelligence in the world, New
How good are simple heuristics? In G. Gigerenzer &
York: Oxford University Press.
P. M. Todd & The ABC Research Group (Eds), Simple
Bröder, A. & Eichler, A. (2001). Individuelle Unter-
heuristics that make us smart (pp. 97–118). Oxford:
schiede in bevorzugten Entscheidungsstrategien. [In-
Oxford University Press.
dividual differences of preferred decision strategies]
Ebbinghaus, H. (1885). Über das Gedächtnis. Unter-
In: A. C. Zimmer, K. Lange et al. (Hrsg.). Experi-
suchungen zur Experimentellen Psychologie [About
mentelle Psychologie im Spannungsfeld von Grundla-
memory. Investgations in experimental psychology].
genforschung und Anwendung (p. 68–75), [CD-ROM].
Leipzig: Duncker & Humblot [Reprint 1966, Amster-
Bröder, A. & Eichler, A. (2006). The use of recognition
dam: E.J.Bonset].
and additional cues in inferences from memory. Acta
Gigerenzer, G. & Goldstein, D. G. (1996). Reasoning the
Psychologica, 121, 275–284.
fast and frugal way: Models of bounded rationality.
Bröder, A. & Gaissmaier, W. (2007). Sequential process-
Psychological Review, 103, 650–669.
ing of cues in memory-based multi-attribute decisions.
Gigerenzer, G., & Todd, P. M. (1999). Fast and frugal
Psychonomic Bulletin and Review, 14, 895–900.
heuristics: the adaptive toolbox. In G. Gigerenzer, P.
Bröder, A. & Schiffer, S. (2003a). Bayesian strategy as-
M. Todd & the ABC Research Group, Simple heuris-

---

Judgment and Decision Making, Vol. 3, No. 3, March 2008
Common beliefs versus empirical results
213
tics that make us smart (pp. 3–34). New York: Oxford
quality of recognition and the inconsequentiality of
University Press.
further knowledge: Two critical tests of the recognition
Gigerenzer, G., Todd, P. M., & the ABC Research Group
heuristic. Journal of Behavioral Decision Making, 19,
(1999). Simple heuristics that make us smart. Oxford:
333–346.
Oxford University Press.
Newell, B. R., Rakow, T., Weston, N. J.,& Shanks, D.
Glöckner, A., & Betsch, T. (2008). Modelling option and
R. (2004). Search strategies in decision-making: The
strategy choices with connectionist networks: Towards
success of success. Journal of Behavioral Decision
an integrative model of automatic and controlled deci-
Making, 17, 117–137.
sion making. Judgment and Decision Making, 3, 215–
Newell, B. R., & Shanks, D. R. (2003). Take-the-best or
228.
look at the rest? Factors in?uencing ‘one-reason’ de-
Goldstein, D. G., & Gigerenzer, G. (2002). Models of
cision making. Journal of Experimental Psychology:
ecological rationality: The recognition heuristic. Psy-
Learning, Memory, and Cognition, 29, 53–65.
chological Review, 109, 75–90.
Newell, B. R., & Shanks, D. R. (2004). On the role of
Hausmann, D. & Läge, D. (2008). Sequential evidence
recognition in decision making. Journal of Experi-
accumulation in decision making: The individual de-
mental Psychology: Learning, Memory & Cognition,
sired level of con?dence can explain the extent of in-
30, 923–935.
formation acquisition. Judgment and Decision Mak-
Newell, B. R. & Shanks, D.R. (2007). Perspectives on
ing, 3, 229–243.
the tools of decision making. In Max Roberts (Ed.)
Hogarth, R. M., & Karelaia, N. (2005). Ignoring informa-
Integrating the mind (pp. 131–151). Hove, UK: Psy-
tion in binary choice with continuous variables: When
chology Press.
is less “more”? Journal of Mathematical Psychology,
Newell, B. R., Weston, N. J., & Shanks, D. R. (2003).
49, 115–124.
Empirical tests of a fast and frugal heuristic: Not ev-
Hogarth, R. M., & Karelaia, N. (2007). Heuristic and
eryone “takes-the-best”. Organizational Behavior and
linear models of judgment: Matching rules and envi-
Human Decision Processes, 91, 82–96.
ronments. Psychological Review, 114, 733–758.
Over, D. E. (2003).
From Massive Modularity to
Juslin, P., & Persson, M. (2002). PROBabilities from
Metarepresentation: The Evolution of Higher Cogni-
EXemplars (PROBEX): A “lazy” algorithm for prob-
tion. In D. E. Over, (Ed.) Evolution and the psychol-
abilistic inference from generic knowledge. Cognitive
ogy of thinking: The debate (pp. 121–144). Hove: Psy-
Science, 26, 563–607.
chology Press.
Lee, M. D., & Cummins, T. D. R. (2004). Evidence ac-
Pachur, T., Bröder, A. & Marewski, J. (in press). The
cumulation in decision making: Unifying the “take the
Recognition Heuristic in Memory-Based Inference: Is
best” and the “rational” models. Psychonomic Bulletin
Recognition a Non-Compensatory Cue? Journal of Be-
and Review, 11, 343–352.
havioral Decision Making.
Luchins, A. S. & Luchins, E. H. (1959). Rigidity of be-
Paivio, A. (1991). Dual code theory: Retrospect and cur-
havior: A variational approach to the effect of Einstel-
rent status. Canadian Journal of Psychology, 45, 255–
lung. Eugene: University of Oregon Press.
287.
Martignon, L., & Hoffrage, U. (1999). Why does one-
Payne, J. W., Bettman, J. R., & Johnson, E. J. (1988).
reason decision making work? A case study in eco-
Adaptive strategy selection in decision making. Jour-
logical rationality. In G. Gigerenzer, P. M. Todd &
nal of Experimental Psychology: Learning, Memory,
The ABC Research Group (Eds), Simple heuristics that
& Cognition, 14, 534–552.
make us smart (pp. 119–140). Oxford: Oxford Univer-
Payne, J. W., Bettman, J. R., & Johnson, E. J. (1993).
sity Press.
The adaptive decision maker. Cambridge: Cambridge
Newell, B. R. (2005). Re-visions of rationality? Trends
University Press.
in Cognitive Sciences, 9, 11–15.
Pohl, R. F. (2006). Empirical tests of the recognition
Newell, B. R. & Bröder, A. (2008). Cognitive processes,
heuristic. Journal of Behavioral Decision Making, 19,
models and metaphors in decision research. Judgment
251–271.
and Decision Making, 3, 195–204.
Rakow, T., Newell, B. R., Fayers, K., & Hersby, M.
Newell, B.R., Collins, P., & Lee, M.D. (2007). Adjusting
(2005). Evaluating three criteria for establishing cue-
the spanner: Testing an evidence accumulation model
search hierarchies in inferential judgment. Journal of
of decision making. In D. McNamara and G. Trafton
Experimental Psychology: Learning, Memory & Cog-
(Eds.), Proceedings of the 29th Annual Conference of
nition, 31, 1088–1104.
the Cognitive Science Society. (pp. 533–538). Austin,
Richter, T. & Späth, P. (2006). Recognition is used as
TX: Cognitive Science Society.
one cue among others in judgment and decision mak-
Newell, B.R. & Fernandez, D. (2006). On the binary
ing. Journal of Experimental Psychology: Learning,

---

Judgment and Decision Making, Vol. 3, No. 3, March 2008
Common beliefs versus empirical results
214
Memory & Cognition, 32, 150–162.
Schneider, W., & Shiffrin, R. M. (1977). Controlled and
Rieskamp, J. (2006). Perspectives of Probabilistic Infer-
automatic human information processing: I. Detection,
ences: Reinforcement Learning and an Adaptive Net-
search, and attention. Psychological Review, 84, 1–66.
work Compared. Journal of Experimental Psychology:
Stone, D. N., & Schkade, D. A. (1991). Numeric and
Learning, Memory, and Cognition, 32, 1355–1370.
linguistic information representation in multiattribute
Rieskamp, J. (2008). The importance of learning when
choice. Organizational Behavior and Human Decision
making inferences. Judgment and Decision Making, 3,
Processes, 49, 42–59.
261–277.
Vickers, D. (1979). Decision processes in visual percep-
Rieskamp, J., & Hoffrage, U. (1999). When do peo-
tion. New York: Academic Press.
ple use simple heuristics and how can we tell? In G.
Wallsten, T. S., & Barton, C. (1982). Processing prob-
Gigerenzer, P. M. Todd & the ABC Research Group,
abilistic multidimensional information for decisions.
Simple heuristics that make us smart (pp. 141–167).
Journal of Experimental Psychology: Learning, Mem-
New York: Oxford University Press.
ory, and Cognition, 8, 361–384.
Rieskamp, J., & Otto, P. E. (2006). SSL: A Theory of
How People Learn to Select Strategies. Journal of Ex-
perimental Psychology: General, 135, 207–236.
Schkade, D. A., & Kleinmuntz, D. N. (1994). Informa-
tion displays and choice processes: Differential effects
of organization, form, and sequence. Organizational
Behavior and Human Decision Processes, 57, 319–
337.

---