Theories of truth as assessment criteria in judgment and decision making

Judgment and Decision Making, Vol. 4, No. 2, March 2009, pp. 116–125
Theories of truth as assessment criteria in judgment and decision
making
Philip T. Dunwoody?
Juniata College
Abstract
Hammond (1996) argued that much of the research in the ?eld of judgment and decision making (JDM) can be
categorized as focused on either coherence or correspondence (C&C) and that, in order to understand the ?ndings of
the ?eld, one needs to understand the differences between these two criteria. Hammond’s claim is that conclusions
about the competence of judgments and decisions will depend upon the selection of coherence or correspondence as
the criterion (Hammond, 2008). First, I provide an overview of the terms coherence and correspondence (C&C) as
philosophical theories of truth and relate them to the ?eld of JDM. Second, I provide an example of Hammond’s claim
by examining literature on base rate neglect. Third, I examine Hammond’s claim as it applies to the broader ?eld of
JDM. Fourth, I critique Hammond’s claim and suggest that re?nements to the C&C distinction are needed. Speci?cally,
the C&C distinction 1) is more accurately applied to criteria than to researchers, 2) should be re?ned to include two
important types of coherence (inter and intrapersonal coherence) and 3) neglects the third philosophical theory of truth,
pragmatism. Pragmatism, as a class of criteria in JDM, is de?ned as goal attainment. In order to provide the most
complete assessment of human judgment possible, and understand different ?ndings in the ?eld of JDM, all three criteria
should be considered.
Keywords: coherence, correspondence, pragmatism, functionalism, Brunswik(ian), judgment and decision making, rep-
resentative design, heuristics and biases, fast and frugal.
1 Introduction
without our acknowledging the role of coherence and cor-
respondence” (Hammond, 2007, p. 225). This is a strong
Hammond (1996; 2007) argued that there are two main
claim and the focus of this paper is the evaluation of this
camps of researchers in the ?eld of judgment and deci-
claim.
sion making (JDM) who have each adopted different cri-
First, I provide an overview and brief history of the
teria for assessing the competence of human judgments
terms coherence and correspondence (C&C) as philo-
and decisions. According to Hammond (1996; 2007) re-
sophical theories of truth and as they relate to JDM.
searchers in the Brunswikian tradition tend to emphasize
Second, I provide an example of Hammond’s claim by
the correspondence of judgment with ecological criteria
examining some of the literature on base rate neglect.
(see Cooksey, 1996 for examples) while researchers in
Third, I examine Hammond’s claim as it applies to the
the Heuristics and Biases (H&B) program tend to assess
broader ?eld of JDM. Fourth, I critique Hammond’s
coherence (see Kahneman, Slovic, & Tversky, 1982 for
claim and suggest that re?nements to the C&C distinc-
examples). Hammond’s claim is that conclusions about
tion are needed. Speci?cally, I point out there are two
the competence of judgments and decisions will depend
important types of coherence and that a third philosophi-
upon the selection of coherence or correspondence as the
cal theory of truth, pragmatism, is needed to complete the
criterion (Hammond, 2008). Hammond stated that “un-
framework.
derstanding the important ?eld of human judgment can-
not go forward, cannot eliminate the current disarray,
?I would like to thank Mandeep Dhami, Michael Doherty, and
2 Overview of the terms coherence
Kenneth Hammond for thoughtful comments that helped improve the
and correspondence
manuscript. I am especially grateful to Xinli Wang, a professor of phi-
losophy with an of?ce close to my own. Not only did he offer a critique
of the manuscript, but also entertained my numerous questions about
The terms C&C have a long history in philosophy, and
philosophical issues while I worked on these ideas. Our impromptu dis-
Dawson and Gregory (2009) address some of their his-
cussions were invaluable to me in ?ushing out these ideas. Address:
Philip T. Dunwoody, Juniata College, 1700 Moore St, Huntingdon, PA,
toric roots. These terms stem from philosophy, where
16652. Email: dunwoody@juniata.edu
they are considered competing theories of knowledge at-
116

---

Judgment and Decision Making, Vol. 4, No. 2, March 2009
Theories of truth in JDM
117
tainment, or truth (Kirkham, 1992). Both of these theo-
truth normative standing. Hammond’s use of the term in-
ries of truth answer the same question; which is, how do
cludes this broader view of interpersonal coherence as he
we know that a statement/belief/judgment is true? The
includes the assessment of judgment against normative
different theories of truth offer different criteria for an-
standards, such as Bayes’s Theorem, under the coherence
swering this question and hence, assessing the truth value
metatheory (Hammond, 1996; 2007).
of a statement/belief/judgment.
Philosophers acknowledge that it is possible to have
The correspondence theory of truth is the oldest of the
two internally coherent belief systems, meaning that in-
philosophical theories of truth due to intuitive appeal.
dividually they are not self-contradictory, that may con-
Simply put, a belief can be said to be true if it corre-
tradict each other. The assumption of coherence philoso-
sponds with the facts. The philosopher Bertrand Rus-
phers is that reality cannot be self-contradicting. So,
sell advocates the correspondence approach stating that
while the coherence of a set of beliefs does not neces-
“although truth and falsehood are properties of beliefs,
sitate that the beliefs are true, coherence is a necessary
they are properties dependent upon the relations of the
feature of true beliefs. A related weakness of the coher-
beliefs to other things, not upon any internal quality of
ence theory of truth is the criticism that it “seems possible
the beliefs. This leads us to the view that truth consists
for a coherent system of beliefs, even an ideal coherent
in some form of correspondence between belief and fact”
system, to be false. We can imagine a system of beliefs
(quoted in Velasquez, 2005, p. 446). This view was dom-
that describes a fantasy world, one substantially differ-
inant with no real opposition until the nineteenth century
ent from the actual world” (Schmitt, 2004, p. 15). Like-
when it came under attack from pragmatists and ideal-
wise, Hammond (1996; 2007) has argued that coherence
ists (Schmitt, 2004). The idealists objected to the no-
of judgment does not guarantee correspondence of judg-
tion of fact. They argued that the apprehension of a fact
ment with facts. Baron (2004) uses the terms coherence
was itself an act of judgment and perception and con-
and calibration, rather than correspondence, to describe
sequently, a belief held by an individual. Because ob-
these approaches. Baron (2008) writes, “judgments can
jects in the world are not directly knowable, but mediated
be coherent without being calibrated. For example, I can
by our senses, they are only representations of objects
say that the probability of heads is .90 and the probability
in the world. Correspondence of beliefs with facts, they
of tails is .10. These two are consistent with each other,
argued, was impossible to assess since facts were also be-
but not with the facts” (p. 119).
liefs. This argument is what led to the coherence theory
Because the coherence of a set of beliefs does not guar-
of truth, the idea that truth is assessed via consistency of
antee truth, some correspondence of the beliefs with facts
belief (Schmitt, 2004).
is needed. Despite the limitations of C&C theories of
The coherence theory of truth states that “a set of two
truth, and their historic opposition, they are insepara-
or more beliefs are said to cohere if and only if (1) each
ble. If nature is necessarily lacking in self-contradictions,
member of the set is consistent with any subset of the oth-
as the coherence theorists argue, then beliefs that corre-
ers and (2) each is implied . . . by all of the others taken as
spond with nature should ultimately be coherent as well.
premises or, according to some coherence theories, each
Coherence is seen as a necessary, but insuf?cient con-
is implied by each of the others individually” (Kirkham,
dition for truth. Because coherent beliefs systems can
p. 104, 1992). In other words, internal consistency and
be widely accepted and false, such as the previously ac-
logical standards are the heart of the coherence theory of
cepted view that the earth was ?at, correspondence is in-
truth. This can be seen in Blanshard’s statement that, “It
escapable as a criterion of truth.
is perhaps in such systems as Euclidean geometry that
Modern scienti?c reasoning advocates using both co-
we get the most perfect examples of coherence that have
herence in the form of rationalism and correspondence
been constructed” (cited in Kirkham, p. 106, 1992).
in the form of empiricism. Coherence is used to orga-
The coherence theory of truth is not limited to beliefs
nize data through the creation and modi?cation of theo-
held by one person; what I will refer to as intrapersonal
ries, and in reasoning about speci?c hypotheses. Theo-
coherence. Intrapersonal coherence requires simply that
ries must be coherent. That is, theories cannot be self-
a statement or belief held by an individual be consistent
contradictory and generally, they must be consistent with
with other statements or beliefs held by the same individ-
other widely held beliefs within that scienti?c commu-
ual. Within the ?eld of JDM, the assessment of transi-
nity. It was for this very reason that Einstein never ac-
tivity is an assessment of intrapersonal coherence. The
cepted the probabilistic nature of quantum physics and
coherence theory of truth extends beyond intrapersonal
the notion of complementarity. Einstein rejected the idea
coherence to interpersonal coherence. That is, beliefs
that a quantum element could act as both a particle and
held by an individual are true if they are consistent with
a wave and assumed that there must be some coherent
widely accepted beliefs of other individuals. This inter-
explanation that has yet to be discovered. New theories
personal coherence is what gives the coherence theory of
that contradict widely held beliefs face an up-hill battle

---

Judgment and Decision Making, Vol. 4, No. 2, March 2009
Theories of truth in JDM
118
because they challenge normative beliefs. In time, such
problem, they had “a minimal effect on subjective proba-
theories can achieve normative standing as the ?eld reex-
bility” (p. 242).
amines its core beliefs and engages in a paradigm shift
Conclusions drawn from research on base-rate neglect
(Kuhn, 1962/1996; Ziman, 1984).
argued that base-rate neglect was an inevitable bias and
From theory, speci?c hypotheses are developed and
a result of either heuristics, incompetence, or both (see
then tested empirically. Hypotheses are evaluated for how
Koehler, 1996 for a review of this work). In an of-
well they correspond to empirical facts. If the hypothe-
ten quoted statement, Bar-Hillel (1980) wrote that, “The
sized data are found to correspond to the observed data,
genuineness, the robustness, and the generality of the
then we claim to have supported the coherent theory. If
base-rate fallacy are matters of established fact” (p. 215).
not, then we try and make sense of the data and the the-
Gigerenzer and Hoffrage (1995) would later show that
ory and engage in rationalism to modify the theory or ex-
base rate usage could be improved by changing the infor-
plain the discrepancy. In order for scienti?c theories to
mation format of the word problems from probability to a
be considered true, they must: 1) be internally consis-
frequency format. Although there has been debate about
tent, 2) be consistent with other widely held beliefs and
whether or not this result is due to the information format
3) correspond with empirical facts. Internal consistency
itself or a confound in the presentation of the summary
and consistency with other beliefs are examples of co-
statistics (see Neace, Michaud, Bolling, Deer, & Zecevic,
herence criteria while a match with empirical facts is an
2008 for a recent example), the approach is still funda-
example of a correspondence criterion. In the search for
mentally that of the coherence variety. A word problem
scienti?c truth, JDM researchers, embrace the strategies
is presented and judgment is assessed against the norma-
of both C&C.
tive standard of Bayes’s Theorem.
Despite the embrace of C&C concepts in our practice
In Koehler’s review of this literature, he points out that
as scientists, it is rare for researchers in JDM to explicitly
there are few instances in the real world where Bayes’s
use the terms C&C and even rarer for researchers to as-
Theorem can be unambiguously mapped to provide a
sess both the C&C of human judgment (for a notable ex-
clear criterion. He questions the conclusions from this
ception, see Adam & Reyna, 2005). Hammond’s claim is
literature by stating, “We have been oversold on the base-
that research on the competence of human judgment often
rate fallacy in probabilistic judgment from an empiri-
paints a contradictory picture because some researchers,
cal, normative, and methodological standpoint” (Koehler,
implicitly or explicitly, adopt one criterion while other
1996, p. 1). In other words, Koehler argues that the co-
researchers adopt the other. Without the recognition that
herence criterion of Bayes’s Theorem has few instances
they are using different kinds of criteria, the conclusions
where a clear correspondence criterion is available. In-
cannot be synthesized. Research on base rate neglect pro-
deed, base rate neglect has seen very little in the way of
vides an example of Hammond’s claim.
investigation using correspondence criteria.
Goodie and Fantino (1995; 1996; 1999a) introduced a
3 Base rate neglect: An example of correspondence criterion to the study of base rate neglect
when they provided subjects direct experience with base
Hammond’s claim
rates and individuating information and required subjects
to make predictions in simulated environments. Simula-
Research on our ability to use base rate information is a
tions such as these create micro-worlds where the corre-
good example of how dominant the coherence approach
spondence of judgments with an ecological criterion (that
is in evaluating judgment. The vast majority of evalua-
is, the ecology of the micro-world) can be assessed. Al-
tions of base rate usage utilize the criterion of Bayes’s
though Goodie and Fantino did observe greater sensitivity
Theorem.
Bayes’s Theorem is a normative standard
to individuating information than base rate information,
based on logic which makes it a coherence criterion.
they noted that there might be some environments where
Kahneman and Tversky published “On the Psychology
the strategy of base rate neglect would produce better cor-
of Prediction” in 1973, which included the now famous
respondence. They write, “. . . when base rates change
lawyer and engineer problem. In their experiments, sub-
relatively often and cue accuracy relatively seldom isn’t it
jects were presented with a variety of word problems and
just as well to underweight base rates, since they’re liable
asked to estimate probabilities and likelihoods. The cor-
to change at any moment?” (1999b, p. 327). In exam-
rect answer was established by the use of Bayes’s The-
ining base rate neglect with a correspondence criterion,
orem. When provided with word problems that include
they opened the door to considerations of how base rate
base rate information and speci?c information from a test,
usage might be shaped by correspondence.
“graduate students relied on a description derived from
Dunwoody, Goodie, and Mahan (2005) sought to ex-
such tests and ignored the base rates” (p. 239). When
amine how manipulating correspondence in?uenced base
base rates were manipulated in the lawyer and engineer
rate usage. Using a design based on that of Goodie and

---

Judgment and Decision Making, Vol. 4, No. 2, March 2009
Theories of truth in JDM
119
Fantino (1995; 1996) they created several different simu-
spondence that cause more short-term losses. These stud-
lated environments in which subjects had to learn to ac-
ies demonstrate that subjects have a sophisticated sensi-
cumulate points by accurately predicting binary events.
tivity to base rate information when it is experienced in
Subjects directly experienced base rates and the accuracy
a way that allows them to evaluate the correspondence
of individuating information over a series of learning tri-
value of using base rate information. This conclusion
als. Base rates were de?ned by the probability that a par-
is strikingly different from the more pessimistic conclu-
ticular event was the correct response and cue accuracy
sions drawn when base rate usage is evaluated only with
(the individuating information) was de?ned as the prob-
coherence criteria.
ability of the cue, given the correct response. Subjects
were randomly assigned to either a condition where cue
accuracies were stable and base rates varied or a condi-
4 Hammond’s claim applied to the
tion where cue accuracies varied and base rates were sta-
?eld of JDM
ble. Correspondence was assessed as a match between
the choice of the subject and the empirical outcome in the
Coherence based theories of judgment focus on evaluat-
micro-world. In this task, correspondence was equivalent
ing the competence of judgment using logic and math-
to reinforcement because subjects received a point for ev-
ematics as criteria. For example, do judgments follow
ery correct prediction. After 200 training trials with out-
the laws of probability? Do decisions violate transitiv-
come feedback, subjects completed 100 additional with-
ity? Are probability judgments consistent with Bayes’s
out feedback. Base rate usage was evaluated in these
Theorem? Do judges make conjunction probability er-
last 100 trials. Although the long-term reinforcement of
rors? Correspondence based theories of judgment fo-
base rates and cue accuracies was equal, correspondence
cus instead on evaluating the predictive ability of a judge
would be more stable if subjects relied on the information
based on some ecological criteria. For example, do judg-
source that was stable. Subjects who had experienced
ments accurately predict events in the world? How often
variable base rates chose a response that matched the base
is weather prediction accurate? How often are medical di-
rate 37% of the time while those that experienced a sta-
agnoses accurate? How well calibrated are judges? These
ble base rate chose a response that matched the base rate
two standards for evaluating judgment have produced dif-
56% of the time. Experienced correspondence had a sig-
ferent conclusions about the competence of human judg-
ni?cant impact on base rate usage in both studies 1 and 2
ment aside from the base rate example provided above.
despite the fact that long-term correspondence was iden-
Speci?cally, Hammond (1996; 2007) argues that the
tical between conditions, only short-term correspondence
Heuristics and Biases (H&B) program of Kahneman
varied.
and Tversky emphasizes coherence criteria and a pes-
In study 3, Dunwoody et al. expanded the inves-
simistic view of human judgment while the Fast and Fru-
tigation by manipulating long-term correspondence as
gal Heuristics (F&FH) program of Gigerenzer and col-
well. Short-term correspondence (information consis-
leagues emphasizes correspondence criteria and an opti-
tency) was manipulated as it was in studies 1 and 2 and
mistic view of human judgment. Although research in
orthogonally crossed with long-term correspondence (the
both traditions focuses on heuristics, their conclusions
reinforcement rate using a particular information source).
about the competence of human JDM differ.
Would subjects still prefer consistent information if it
Tversky and Kahneman write that “people rely on a
resulted in a 10% lower reinforcement rate? Subjects
limited number of heuristic principles which reduce the
switched from utilizing the consistent information source
complex tasks of assessing probabilities and predicting
in studies 1 and 2 to utilizing the information source with
values to simpler judgmental operations. In general, these
the highest long-term correspondence in study 3. When
heuristics are quite useful, but sometimes they lead to se-
base rates were inconsistent but resulted in 10% higher
vere and systematic errors” (1982, p. 3). Their program
correspondence, subjects chose base rate responses 69%
is widely known as the H&B program and this title em-
of the time. When base rates were consistent but resulted
phasizes their main ?ndings: heuristics produces biases,
in 10% lower correspondence, subjects chose base rates
or systematic errors, in judgment. Research in this tradi-
only 26% of the time.
tion regularly adopts coherence criteria in assessing com-
The combined results of the studies by Dunwoody et
petence and human judgment is often shown to deviate
al. (2005) show that base rate usage varies as a function
widely from these coherence criteria. According to much
of experienced correspondence. Subjects will use, or ne-
of the research in this tradition, human judgment fails the
glect, base rates depending on how base rates help them
coherence assessment and therefore humans are not ratio-
predict events. They are sensitive to both short-term ma-
nal. For example, Kahneman and Tversky (1972) stated,
nipulations of correspondence consistency that result in
“man is apparently not a conservative Bayesian: he is not
no long-term gains and long-term manipulations of corre-
Bayesian at all” (p. 450).

---

Judgment and Decision Making, Vol. 4, No. 2, March 2009
Theories of truth in JDM
120
Goldstein and Hogarth (1997) point out that the H&B
of correspondence. While this research has shown many
program has often been criticized. They state, “experi-
limitations on human judgment, it has often demonstrated
ments were conducted so that the word problems set up
that human judgment is capable of achieving the level of
a ‘trap’ that subjects would fall into if they were us-
accuracy allowed by the environment (Cooksey, 1996).
ing a particular heuristic. In this way, biased behavior
Gigerenzer and colleagues have taken a similar ap-
was merely a device used to provide (dramatic) evidence
proach to evaluating human judgment by emphasizing the
that hypothesized psychological processes were at work.”
correspondence of human judgment within a particular
This research agenda emphasizes, “that behavioral devia-
ecology. Gigerenzer and Todd (1999) explicitly state that
tions from a presumed standard of rationality are the ‘in-
their research agenda “dispenses with the focus on coher-
teresting’ phenomena requiring explanation” (p. 26). In
ence criteria . . . . Instead, we study correspondence-based
other words, the presumed standard of rationality from
performance of heuristics in real-world environments” (p.
which people deviate is typically a coherence criterion.
28). They introduce the term “ecological rationality” to
Much, but certainly not all, of the research in the
distinguish their correspondence-based rationality from
correspondence tradition is rooted in the work of Egon
the more traditional coherence-based rationality. Their
Brunswik.
Brunswik was a functionalist, in?uenced
?ndings are consistent with the title of their 1999 book,
heavily by the work of Darwin and Tolman and was con-
Simple heuristics that make us smart. In short, they argue
cerned primarily with how organisms adapt to their envi-
that F&FH are essential for adaptive human judgment.
ronments (Doherty & Kurz, 1996; Hammond & Stewart,
They argue that the H&B approach has focused on arti?-
2001; Tolman & Brunswik, 1935/2001). Brunswik was
cial environments where heuristics are likely to produce
interested in issues of perceptual constancy (Brunswik,
biases. However, their own approach is explicitly biased
1944/2001; Doherty & Kurz, 1996). Perceptual con-
in the opposite direction. They state that they “focus on
stancy is fundamentally focused on correspondence; the
the ways and settings in which simple heuristics lead to
correspondence between a percept and the distal object.
accurate and useful inferences” (p. 28).
Brunswik (1957/2001) believed that the proper focus for
Gigerenzer and colleagues’ research on F&FH has
psychology was the understanding of how the organ-
shown that given the right environment, intuitive use
ism achieves this correspondence. Hammond is cred-
of heuristics that may lack coherence can work well in
ited with demonstrating the relevance of Brunswik’s ar-
achieving correspondence. Gigerenzer and Kurz write:
guments for understanding human judgment (Cooksey,
1996; Goldstein & Hogarth, 1997). While Hammond
What we call ecological rationality is an elabo-
extended Brunswik’s initial ideas, he retained the strong
ration of the Brunswikian program of studying
focus on achievement, de?ned as the correspondence be-
the texture of environments. Heuristics are not
tween judgment and empirical event, as the central prob-
rational in the classical sense of coherence . . . .
lem of psychology. As such, most JDM researchers in
They derive their rationality through a match
this tradition focus on how well judgments correspond to
with the structure of the environment, not with
empirical events in the world.
the laws of logic or probability (2001, p. 346).
Correspondence based research also has a long his-
Although both the H&B program and the F&FH pro-
tory in what is known as multi-cue-probability-learning
gram study heuristics, they have reached different con-
(MCPL) (Cooksey, 1996; Hammond & Summers, 1965;
clusions about the competence of human JDM. While
Holzworth, 2001). Studies investigating MCPL often
much of the research in the H&B program has focused
create micro-worlds where empirical accuracy can be
on demonstrating a lack of coherence in human judgment,
measured as correct predictions within the micro-world.
and hence a lack of logic and rationality, the F&FH pro-
Cooksey (1996) states that:
gram of Gigerenzer has focused on demonstrating cor-
The central goal of MCPL research was to es-
respondence in human judgment, and hence ecological
tablish . . . the conditions under which a person
rationality. Much of the research in the H&B program ig-
could learn how to predict an ecological crite-
nores correspondence criteria in de?ning “rational judg-
rion of interest; that is, how achievement (in
ment” while the F&FH program explicitly rejects coher-
Lens Model terms) was maximized by a person
ence criteria in favor of correspondence criteria. Sim-
. . . . Such learning was demonstrated through
ilarly, research in the Brunswikian tradition, including
the person’s acquisition of the appropriate task
MCPL, typically ignores coherence criteria in assessing
properties . . . needed to predict the criterion (p.
human JDM.
63).
In the philosophy literature, the coherence of a judg-
ment is considered a necessary but insuf?cient feature
Achievement is a correlation between an individual’s
of truth. True beliefs are those that are both coherent
judgments and some ecological criterion and is a measure
and correspondent. It may be likewise true in the ?eld

---

Judgment and Decision Making, Vol. 4, No. 2, March 2009
Theories of truth in JDM
121
of JDM that increases in the coherence of judgment are
broad a brush and second, his framework is incomplete. I
(imperfectly) associated with increases in the correspon-
address each point below.
dence of judgment. Baron (2008), using the term cal-
While there is undeniable bene?t in examining schools
ibrated in place of correspondent, writes, “If my judg-
of thought, or programs of research, important connec-
ments are perfectly calibrated, however, they must also
tions between the programs, and subtleties within the pro-
be coherent” (p. 119). However, few studies have em-
grams, can be lost when painting with such a broad brush.
pirically examined the relationship between C&C. Dun-
Although in general Hammond is correct in his catego-
woody et al. (2005) found that, while manipulations of
rizations, one does not have to look far to ?nd notable ex-
experienced correspondence in?uenced base rate usage
ceptions. Calibration research, which focuses on the cor-
under conditions of direct experience, the same manipu-
respondence of judged and actual probabilities, is a main
lation failed to improve Bayesian responses to word prob-
area of research in the H&B program (see Lichtenstein,
lems in a related task. Adam and Reyna (2005) examined
Fischhoff & Phillips, 1982 for an early review of this lit-
the coherence and correspondence of experts’ judgments
erature). Similarly, a traditional measure in Brunswikian
about sexually transmitted infections and noted a discon-
research is cognitive control, written as Rs in the lens
nect between coherence and correspondence measures.
model equation (see Cooksey, 1996; Hammond & Sum-
They write:
mers, 1972), which measures the degree to which an in-
dividual judge consistently applies his or her policy. Re-
Coherence and correspondence are not com-
search in the Brunswikian tradition also examines judg-
peting criteria for rationality, as commonly as-
ments of repeated cases to examine the consistency of
sumed. Instead, each criterion captures a dis-
judgment. Both cognitive control and consistency fo-
tinct aspect of rationality: irrational judgments
cus on intrapersonal coherence, not correspondence (see
either con?ict with reality or with other judg-
Beckstead & Stamp, 2007 for a recent example utilizing
ments. The same experts received a positive ap-
measures of both cognitive control and consistency).
praisal based on correspondence criteria, but a
It is therefore an oversimpli?cation to discuss the en-
negative appraisal based on coherence criteria,
tirety of the H&B program as coherence oriented and the
with close correspondence for some judgments
entirety of the Brunswikian program as correspondence
and large coherence errors for other judgments.
oriented. While each program may emphasize one type
p. 183
of criterion over the other, C&C are used by both research
Weiss, Brennan, Thomas, Kirlik, and Miller (2009) ex-
traditions. It is more accurate to use the C&C distinc-
amine performance (not judgment) in a golf-putting task
tion to classify the criteria used to assess judgment, than
and found that a coherence based measure (CWS) cor-
to use it to classify research programs, which are much
related with a correspondence based measure (the mean
broader and more nebulous. Classifying assessment cri-
absolute distance between the golf ball and target) at .676.
teria, rather than research programs, may also make it
This is a strong correlation and indicates that at least un-
more likely that researchers examining a speci?c topic
der some conditions, coherence is correlated with corre-
do so via multiple criteria. To achieve a more complete
spondence. It is conceivable that task differences between
understanding of human competence, it is necessary to
studies might account for the differing relationships ob-
examine topics via multiple criteria, as demonstrated in
served between coherence and correspondence. The re-
the base rate neglect example above.
search ?ndings suggest that it might be best to conceive
My second critique, that the C&C framework is in-
of C&C assessments as orthogonal dimensions. When as-
complete, is based on two points. First, it ignores the
sessed jointly it is possible to be high on both dimensions,
distinction I introduced earlier in this paper between in-
only one dimension, or neither dimension.
trapersonal coherence and interpersonal coherence. In-
trapersonal coherence is the examination of consistency
within a person. Assessing judgments and decisions for
5 Critiquing Hammond’s claim
transitivity or consistency are good example of intraper-
sonal coherence. The phenomenon of cognitive disso-
Hammond’s claim is essentially correct; researchers in
nance shows that people are motivated to achieve intrap-
JDM either adopt coherence or correspondence criteria
ersonal coherence when inconsistencies are made salient.
and the adoption of different criteria has lead to different
Interpersonal coherence is the examination of consistency
conclusions about the competence of human JDM. Al-
in belief or judgments among people. Speci?cally, as-
though I support Hammond’s goal of greater recognition
sessing judgments and decisions of a subject against the
of C&C in the ?eld of JDM and am in agreement with
normatively held beliefs of others, such as Bayes’s The-
his main claim, I have two criticisms of his claim. First,
orem, is an assessment of interpersonal coherence. Al-
Hammond’s focus on research programs paints with too
though both appeal to logic, assessing judgments for in-

---

Judgment and Decision Making, Vol. 4, No. 2, March 2009
Theories of truth in JDM
122
ternal consistency is different from assessing judgments
thoughts means everywhere the possession of invaluable
against normative standards of which the judge may be
instruments of action” (quoted in Kirkham, 1992, p. 92).
unaware. It is easy to see that measures of interpersonal
Truths that correspond to reality are only useful in that
coherence need not also measure intrapersonal coherence
they help one achieve a goal.
James states, “Those
but the reverse is not true. Consistency within a person’s
thoughts are true which guide us to bene?cial interac-
beliefs/judgments is a normative standard of logic and
tion with sensible particulars as they occur, whether they
therefore assesses both intra and interpersonal coherence.
copy these in advance or not” (Kirkham, 1992, p. 92). For
The second aspect of Hammond’s C&C framework
pragmatism, the correspondence of a judgment is not as
that is incomplete is the absence of pragmatism. Co-
important as the utility.
herence and correspondence are two of the three main
Correspondence in and of itself, is not necessarily
philosophical theories of truth, pragmatism is the third. In
adaptive and it may even be adaptive to have beliefs that
adopting only two of the three main theories of truth for
lack correspondence. For example, some have argued
use as a categorization scheme, Hammond’s C&C frame-
that it is evolutionarily advantageous for our species to
work neglects an important class of assessment criteria,
have an overcon?dence bias in certain domains (Hasel-
goal-oriented assessment. I will ?rst address pragmatism
ton & Nettle, 2006). Haselton and Buss (2000) argue that
as a philosophical theory of truth and then explain why it
men are more likely to achieve their goals by overesti-
is a necessary criterion for human JDM.
mating the number of females interested in them sexually.
The costs of false-negatives (missing a sexual encounter)
5.1 Pragmatism and its necessity in JDM
outweigh the costs of false-positives (being turned down).
This bias, which is a lack of correspondence with our
Pragmatism is the most recent of the three main theories
actual abilities, leads to a certain degree of risk taking
of truth and attempts to deal with some of the limitations
that may be bene?cial for the organism, and in the long
in the correspondence and coherence views. These limi-
run, bene?cial for the species. These biases should be ex-
tations, addressed earlier in the paper, include the possi-
pected when the evolutionary costs of false-positive and
bility of a coherent set of beliefs that do not correspond
false-negatives are asymmetric. Haselton and Buss argue
with reality, and the dif?culty in selecting and identify-
that:
ing facts for correspondence. The pragmatic theory ar-
gues that the utility of a belief is a good measure of truth
. . . optimal designs are sometimes those that re-
value. False beliefs are not likely to be useful. This the-
sult in errors that historically minimized over-
ory of truth is primarily associated with Charles Pierce
all costs or maximized overall bene?ts. This
and William James.
rule of good design contrasts with the nearly
Charles Pierce argues that reality impinges itself upon
ubiquitous assumption in psychology that opti-
our senses and slowly forces beliefs to conform to real-
mal reasoning systems are those that best cor-
ity (Kirkham, 1992). At ?rst this may sound like corre-
respond to normative rules or that best produce
spondence, but the emphasis here is on the function, not
veridical inferences (2000, p. 90).
objective reality. Reality forces us to adopt pragmatic be-
They note that, “Heuristics and biases researchers have
liefs. One thing that may make them pragmatic is that
assumed that . . . errors reveal information-processing
they are true. William James shares this type of pragma-
shortcuts” but from their perspective, “some errors re-
tism with Peirce. Both argue that it is the functional value
veal the cost and bene?t asymmetries present over evo-
of the beliefs that makes them true (and that true beliefs
lutionary history. Errors may be evidence of evolved
are functional). James does not appear open to the idea
adaptive biases, not simplifying heuristics” (p. 90). To
of correspondence, what he refers to as “copying,” in the
reframe their argument, optimal decisions are not neces-
absence of functionalism. He writes:
sarily measured by a coherence criterion (“those that best
From the frequency of copying in the knowl-
correspond to normative rules”) or a correspondence cri-
edge of phenomenal fact, copying has been
terion (those that “produce veridical inferences,”) but by
supposed to be the essence of truth . . . the
a pragmatic criterion (utility/goal attainment).
whole notion of copying tends to evaporate . . . .
Pragmatism is indispensable as a criterion for JDM and
Their objects can be better interpreted as being
can be found in the major research traditions in JDM. Al-
created step by step by men, as fast as they suc-
though Brunswik focused on achievement, a correspon-
cessively conceive them. (Kirkham, 1992, p.
dence criterion, his emphasis was on practical achieve-
92).
ment rather than veridical correspondence. Brunswik
writes, “Constancy-research . . . is concerned with practi-
For James then, truth is not copying reality, which is
cal achievements of living beings. Therefore, it may not
a form of correspondence, but “The possession of true
be expected to ?nd as ideal results as have been assumed

---

Judgment and Decision Making, Vol. 4, No. 2, March 2009
Theories of truth in JDM
123
in the previous chapters” [italics added] (1937/2001, p.
truth, pragmatism is the most closely linked with func-
45).
Likewise, Gigerenzer and colleagues’ focus on
tionalism. As such, pragmatism appears to be the most
F&FH highlights their functional value in speci?c en-
appropriate class of criteria for Brunswik’s Probabilistic
vironments.
They do not argue that F&FH produce
Functionalism and expected utility theory. Even Gigeren-
veridical correspondence, but rather that F&FH are func-
zer, who explicitly rejects coherence standards in favor of
tional because they exploit a match between the heuristic
correspondence standards, is more appropriately catego-
and the environment. Gigerenzer and Todd emphasize
rized as a pragmatist since he is ultimately interested in
this functional perspective in explaining their research
the adaptive value of judgments. While Hammond (1996)
agenda. They state, “What works to make quick and
has pushed for recognition of the C&C theories of truth
accurate inference in one domain may well not work in
in JDM, the criterion of pragmatism, and the similarities
another. Thus, different environments can have different
it shares with functionalism, has been overlooked.
speci?c fast and frugal heuristics that exploit their par-
ticular information structure to make adaptive decisions”
(1999, p.18).
6 Conclusion
Pragmatism in the ?eld of JDM is not limited to the
correspondence research, but also clearly seen in the nor-
In philosophy, coherence is a necessary but insuf?cient
mative expected utility framework. In his introduction to
feature of truth. Philosophical theories of truth have high-
the idea of utility, Baron (2004) equates utility with good
lighted the need to consider both coherence and corre-
and states that, “good is the extent to which we achieve
spondence when evaluating a judgment. A full view of
our goals” (2004, p. 24). Baron (2008) writes, “utility is
the competence of human JDM should assess with a mul-
supposed to be a summary measure of how consequences
titude of criteria. It is not always possible to study out-
realize our ultimate values or goals” (p. 234). The ex-
come achievement and therefore, not always possible to
tent to which our decisions bring us closer to our goals
evaluate correspondence (see Dhami, Hertwig, & Hof-
is an assessment based on the criterion of pragmatism for
frage, 2004 for more on this point). Representative De-
pragmatism is fundamentally about the utility of beliefs.
sign might be useful in this regard as Representative De-
In discussing William James’s pragmatic theory of truth,
sign explicitly focuses on including the conditions one
Schmitt (2004) writes “that a belief is true just in case
wishes to generalize to in the research study (Brunswik,
it has practical utility in life (or belongs to a system of
1956; Dhami, Hertwig, & Hoffrage, 2004; Dunwoody
beliefs that has practical utility)” (p. 9). In these state-
2006/2007). Understanding the conditions one is gener-
ments we see that pragmatism is closely associated with
alizing to will help clarify the relative roles of coherence,
the notion of utility, a major focus in the ?eld of JDM.
correspondence, and pragmatism. For this reason and
The connections between pragmatism and both C&C
others, coherence, correspondence, and pragmatism cri-
can be seen in the writings of James. James indicates
teria have a central place in the evaluation of human JDM.
three main types of utility. First, “a belief can be use-
It remains to be seen under what conditions increases in
ful . . . if it helps us to manipulate the objects in the
coherence leads to increases in correspondence. The lim-
world” (Kirkham, 1992, p. 93). Beliefs that meet co-
ited research evaluating the relationship between C&C
herence criteria appear to be particularly useful in this
paints an inconclusive picture. The precise relationship
regard. Coherent theories by their very nature involve
between these criteria may depend on task conditions and
causal narratives. Coherent theories allow us to manipu-
is an important research question that deserves more at-
late the environment because of their causal nature. Sec-
tention.
ond, “beliefs are useful when they allow successful com-
Goal-attainment is one of the most basic aspects of hu-
munication with our fellows” (Kirkham, 1992, p. 93).
man behavior. It is a main focus of Brunswik’s proba-
Again, the coherence based truths seem well ?tted to this
bilistic functionalism and the foundation of utility theory.
purpose in that coherence is a consistency in belief ex-
Any framework for classifying criteria in the ?eld of JDM
pressed through language. A lack of coherence in belief,
would be negligent to exclude the organism’s own goals
thought, and language is a major barrier to communica-
as a major class of criteria. Necessarily, the inclusion of
tion (see Tetlock, 1997 for a similar argument). Third, a
pragmatism as a classi?cation criteria means that many
“belief can be useful if it leads to accurate predictions”
criteria used to assess JDM will fall under more than one
(Kirkham, 1992, p. 93). Here the correspondence-based
of the three theories of truth. For example, achievement is
beliefs are most functional as they are excellently suited
a measure of correspondence but if the organism’s goals
to the task of prediction. Pragmatic-based criteria allow
include achievement, or if achievement facilitates goal at-
for the use of C&C criteria as long as their use is func-
tainment, then achievement can be considered a criterion
tionally adaptive.
of both correspondence and pragmatism. Likewise, util-
We can see above that of the three main theories of
ity theory has normative standing and as such, is an exam-

---

Judgment and Decision Making, Vol. 4, No. 2, March 2009
Theories of truth in JDM
124
ple of intrapersonal coherence. Since utility is also about
Brunswik, E. (1957/2001), Scope and aspects of the cog-
goal attainment, it is also a criterion based on pragma-
nitive problem. In K. R. Hammond & T. R. Stewart
tism. Other normative criteria, such as Bayes’s theorem,
(Eds.), The essential Brunswik: Beginnings, explica-
may have little to do with the organism’s goals.
tions, applications (pp. 300–312). New York: Oxford
I have argued in this paper that Hammond’s claim has
University Press.
merit and I support greater recognition of these terms
Cooksey, R. W. (1996). Judgment analysis: Theory,
within the ?eld of JDM. I have also argued that the C&C
methods, and applications. San Diego, CA: Academic
distinction should be re?ned to include intra and inter-
Press.
personal coherence, and expanded to explicitly include
Dawson, N. V., & Gregory, F. (2009). Correspondence
a category of assessment criteria based on goal attain-
and coherence in science: A brief historical perspec-
ment; that is, pragmatism. Interpreting and generaliz-
tive. Judgment and Decision Making, 4, 126–133.
ing the ?ndings of research in JDM will bene?t from ex-
Dhami, M. K., Hertwig, R., & Hoffrage, U. (2004). The
plicit recognition of the three main philosophical theo-
role of representative design in an ecological approach
ries of truth as categories of assessment criteria. This is
to cognition. Psychological Bulletin, 130 (6), 959–
not only true for understanding past research in the ?eld
988.
of JDM, as shown in this paper, but for understanding
Doherty, M., & Kurz, E. M. (1996). Social judgment the-
engineering decisions (Katsikopoulos, 2009), medical re-
ory. Thinking and Reasoning, 2 (2/3), 109–140.
search (Adam & Reyna, 2005; Tape, 2009), and aviation
Dunwoody, P. T. (2006/2007). The neglect of the environ-
research (Mosier, 2009).
ment by cognitive psychology. Journal of Theoretical
and Philosophical Psychology, 26, 139–153.
Dunwoody, P. T., Goodie, A. S., & Mahan, R. P. (2005).
7 References
The use of base rate information as a function of ex-
perienced consistency. Theory and Decision, 59, 307–
Adam, M. B., & Reyna, V. F. (2005). Coherence and cor-
344.
respondence criteria for rationality: Experts’ estima-
Gigerenzer, G., & Hoffrage, U. (1995). How to im-
tion of risks of sexually transmitted infections. Journal
prove Bayesian reasoning without instruction: Fre-
of Behavioral Decision Making, 18, 169–186.
quency formats. Psychological Review, 102,684–704.
Bar-Hillel, M. (1980). The base-rate fallacy in probabil-
Gigerenzer, G., & Kurz, E. M. (2001). Vicarious func-
ity judgments. Acta Psychologica, 44, 211–233.
tioning reconsidered: A fast and frugal lens model. In
Baron, J. (2004). Normative models of judgment and de-
K. R. Hammond & T. R. Stewart (Eds.), The essential
cision making. In D. J. Koehler & N. Harvey (Eds.),
Brunswik: Beginnings, explications, applications (pp.
Blackwell handbook of judgment and decision making
342–347). Oxford, England: Oxford University Press.
(pp. 19–36). Malden, MA: Blackwell Publishing Ltd.
Gigerenzer, G., & Todd, P. M. (1999). The research
Baron, J. (2008). Thinking and deciding (4th ed.). Cam-
agenda. In G. Gigerenzer, P. M. Todd & the ABC
bridge: Cambridge University Press.
Research Group (Eds.), Simple heuristics that make us
Beckstead, J. W., & Stamp, K. D. (2007). Understand-
smart (pp. 1–36). Oxford, England: Oxford University
ing how nurse practitioners estimate patient’s risk for
Press.
coronary heart disease: a judgment analysis. Journal
Goldstein, W. M., & Hogarth, R. M. (1997). Judg-
of Advanced Nursing, 60(4), 436–446.
ment and decision research. In W. M. Goldstein & R.
Brunswik, E. (1937/2001). Psychology as a science of
M. Hogarth (Eds.), Research on Judgment and Deci-
objective relations. In K. R. Hammond & T. R. Stew-
sion Making: Currents, connections, and Controver-
art (Eds.), The essential Brunswik: Beginnings, expli-
sies (pp. 3–68). Cambridge, United Kingdom: Cam-
cations, applications (pp. 38–54). New York: Oxford
bridge University Press.
University Press.
Goodie, A. S., & Fantino, E. (1995). An experientially
Brunswik, E. (1944/2001). Distal focusing of perception:
derived base-rate error in humans. Psychological Sci-
Size constancy in a representative sample of situations.
ence, 6(2), 101–106.
In K. R. Hammond & T. R. Stewart (Eds.), The essen-
Goodie, A. S., & Fantino, E. (1996). Learning to commit
tial Brunswik: Beginnings, explications, applications
or avoid the base-rate error, Nature 380, 247–249.
(pp. 68–105). New York: Oxford University Press.
Goodie, A. S., & Fantino, E. (1999a). Base rates ver-
Brunswik, E. (1956/2001), Perception and the represen-
sus sample accuracy: Competition for control in hu-
tative design of psychological experiments.
In K.
man matching to sample. Journal of the Experimental
R. Hammond & T. R. Stewart (Eds.), The essential
Analysis of Behavior, 71(2), 155–169.
Brunswik: Beginnings, explications, applications (pp.
Goodie, A. S., & Fantino, E. (1999b). What does and
260–264). New York: Oxford University Press.
does not alleviate base-rate neglect under direct expe-

---

Judgment and Decision Making, Vol. 4, No. 2, March 2009
Theories of truth in JDM
125
rience. Journal of Behavioral Decision Making, 12,
Kuhn, T. S. (1962/1996). The structure of scienti?c revo-
307–335.
lutions (3rd ed.). Chicago: The University of Chicago
Hammond, K. R. (1996). Human judgment and social
Press.
policy: Irreducible uncertainty, inevitable error, un-
Lichtenstein, S., Fischhoff, B., & Phillips, L. D. (1982).
available injustice. New York: Oxford University
Calibration of probabilities: The state of the art to
Press.
1980. In D. Kahneman, P. Slovic, & A. Tversky (Eds.),
Hammond, K. R. (2007). Beyond rationality: The search
Judgment under uncertainty: Heuristics and biases
for wisdom in a troubled time. New York: Oxford Uni-
(pp. 306–334). Cambridge, England: Cambridge Uni-
versity Press.
versity Press.
Hammond, K. R. (2008, November 14). Reconciliation
Mosier, K. L. (2009). Searching for coherence in a corre-
of opposing conclusions regarding rationality. Paper
spondence world. Judgment and Decision Making, 4,
presented at the 24th Annual International Meeting of
154-163.
the Brunswik society.
Neace, W. P., Michaud, S., Bolling, L., Deer, K., & Zece-
Hammond, K. R., & Stewart, T. R. (Eds.). (2001), The
vic, L. (2008), Frequency formats, probability formats,
essential Brunswik: Beginnings, explications, applica-
or problem structure? A test of the nested-sets hypoth-
tions. Oxford, England: Oxford University Press.
esis in an extensional reasoning task. Judgment and
Hammond, K. R., & Summers, D. A. (1965). Cognitive
Decision Making, 3, 140–152.
dependence on linear and nonliear cues. Psychological
Schmitt, F. F. (2004). Truth: An introduction. In F. F.
Review, 72, 215–224.
Schmitt (Ed.), Theories of truth (pp. 1–38). Oxford:
Hammond, K. R., & Summers, D. A. (1972). Cognitive
Blackwell Publishing Ltd.
control. Psychological Review, 79, 58–67.
Tape, T. G. (2009), Correspondence and coherence in
Haselton, M. G., & Buss, D. M. (2000). Error manage-
medicine. Judgment and Decision Making, 4, 130–
ment theory: A new perspective on biases in cross-sex
140.
mind reading. Journal of Personality and Social Psy-
Tetlock, P. E. (1997). An alternative metaphor in the
chology, 78, 81–91.
study of judgment and choice: People as politicians.
Haselton, M. G., & Nettle, D. (2006). The paranoid op-
In W. M. Goldstein & R. M. Hogarth (Eds.), Research
timist: An integrative evolutionary model of cognitive
on Judgment and Decision Making: Currents, connec-
biases. Personality and Social Psychology Review, 10
tions, and Controversies (pp. 657–680). Cambridge,
(1) 47–66.
United Kingdom: Cambridge University Press.
Holzworth, R. J. (2001). Multiple cue probability learn-
Tolman, E. C., & Brunswik, E. (1935/2001). The or-
ing. In K. R. Hammond & T. R. Stewart (Eds.), The
ganism and the causal texture of the environment. In
essential Brunswik: Beginnings, explications, applica-
K. R. Hammond & T. R. Stewart (Eds.), The essential
tions (pp. 348–350). Oxford, England: Oxford Univer-
Brunswik: Beginnings, explications, applications (pp.
sity Press.
17–34). New York: Oxford University Press.
Kahneman, D., Slovic, P., & Tversky, A. (Eds.). (1982).
Tversky, A., & Kahneman, D. (1982). Judgment under
Judgment under uncertainty: Heuristics and biases.
uncertainty: Heuristics and biases. In D. Kahneman,
Cambridge: Cambridge University Press.
P. Slovic, & A. Tversky (Eds.), Judgment under uncer-
Kahneman, D., & Tversky, A. (1972). Subjective prob-
tainty: Heuristics and biases (pp. 3–22). Cambridge:
ability: A judgment of representativeness. Cognitive
Cambridge University Press
Psychology, 3, 430–454.
Velasquez, M. (2005). Philosophy (9th ed.). Belmont,
Kahneman, D., & Tversky, A. (1973). On the psychology
CA: Thomson Wadsworth.
of prediction. Psychological Review, 80 (4), 237–251.
Weiss, D. J., Brennan, K., Thomas, R., Kirlik, A., &
Katsikopoulos, K. V. (2009). Coherence and correspon-
Miller, S. (2009). Criteria for performance evaluation.
dence in engineering design: Informing the conversa-
Judgment and Decision Making, 4, 164–174.
tion and meeting with JDM research. Judgment and
Ziman, J. (1984). An introduction to science studies: The
Decision Making, 4, 147–153.
philosophical and social aspects of science and tech-
Kirkham, R. L. (1992). Theories of truth: A critical in-
nology. Cambridge: Cambridge University Press.
troduction. Cambridge, MA: The MIT Press.
Koehler, J. J. (1996).
The base rate fallacy recon-
sidered: Descriptive, normative, and methodological
challenges. Behavioral & Brain Sciences, 19, 1–53.

---