Modeling sequential context effects in judgment analysis: A time series approach

Judgment and Decision Making, Vol. 3, No. 7, October 2008, pp. 570–584
Modeling sequential context effects in judgment analysis:
A time series approach
Jason W. Beckstead?
College of Nursing
University of South Florida
Abstract
In this article a broad perspective incorporating elements of time series theory is presented for conceptualizing the
data obtained in multi-trial judgment experiments. Recent evidence suggests that sequential context effects, assimilation
and contrast, commonly found in psychophysical judgment tasks, may be present in judgments of abstract magnitudes.
A time series approach for analyzing single-subject data is developed and applied to expert prognostic judgments of
risk for heart disease with an emphasis on detecting possible sequential context effects. The results demonstrate that
sequential context effects do exist in such expert prognostic judgments. Contrast and assimilation were produced by cue
series; the latter occurring more frequently. Experts also showed assimilation of prior responses that was independent
of the cue series input. Time series analysis also revealed that abrupt or large trial-by-trial changes in the value of cues
that receive the most attention in prognostic judgment tasks can disrupt the accuracy of these judgments. These ?ndings
suggest that a time series approach is a useful alternative to ordinary least squares regression, providing additional
insights into the cognitive processes operating during multi-cue judgment experiments.
Keywords: expert judgment, time series, contrast and assimilation, single-subject analysis.
Psychological data are segments of life histories: as
and recording a series of responses. Such idiographic de-
such they are ordered sequences of observations and by
signs (and analysis of their ensuant data) are the focus of
de?nition time series. — Robert A. M. Gregson (1983).
this article.
Contrast and assimilation are psychological processes
1 Introduction
involving the sequential context in which judgments
are made.
In a variety of psychophysical judgment
Many judgment experiments may be viewed as involv-
paradigms employing large numbers of trials (e.g., ab-
ing two time series: a series of stimuli presented by
solute and relative magnitude scaling tasks, absolute and
the experimenter, and a series of responses provided
relative identi?cation tasks) sequential context effects are
by the subject. Over the years various theories of hu-
frequently observed (for reviews see DeCarlo & Cross,
man judgment have been proposed. One thing all these
1990; Stewart, Brown, & Chater, 2005). Assimilation
theories have in common is that they have been, and
occurs when the response to a given stimulus intensity
are being, developed using single-subject repeated mea-
tends to be larger when the immediately preceding stim-
sures experiments. Whether judgment data are mod-
ulus is of greater intensity than the current stimulus, and
eled using multiple regression, as is typical in the judg-
tends to be smaller when the preceding stimulus inten-
ment analysis paradigm often associated with social judg-
sity is less than that of the current stimulus. Contrast
ment theory (Hammond, et al., 1975), by single-subject
occurs when the response to a given stimulus intensity
ANOVA which forms the foundation of information inte-
tends to be smaller when the immediately preceding stim-
gration theory (Anderson, 1981; 1982), by conjoint anal-
ulus is of greater intensity than the current stimulus, and
ysis (Luce & Tukey, 1964; see also Krantz & Tversky,
tends to be larger when the preceding stimulus intensity is
1971), or by fast and frugal heuristics, such as Take the
less than that of the current stimulus. DeCarlo and Cross
Best (Gigerenzer et al., 1991), or the Matching Heuristic
(1990) discuss various theoretical models of psychophys-
(Dhami & Ayton, 1998; 2001), the data are obtained by
ical judgment that have been proposed to explain sequen-
presenting the subject with a series of stimuli to be judged
tial context effects in magnitude scaling experiments and
show how these can be evaluated using time series regres-
?Address: Jason W. Beckstead, University of South Florida College
of Nursing, 12901 Bruce B. Downs Boulevard, MDC22, Tampa, Florida
sion. One class of models is referred to as relative judg-
33612. Email. jbeckste@health.usf.edu.
ment models in which the subject is portrayed as com-
570

---

Judgment and Decision Making, Vol. 3, No. 7, October 2008
Time series and judgment analysis
571
paring the value of the current stimulus to the value of
the stimulus on the preceding trial rather than to some
?xed internal reference. Stewart et al. (2005) discuss
relative judgment models in absolute identi?cation tasks.
Another class of models suggests that sequential context
effects result from a response heuristic, or the tendency
of the subject, in the face of uncertainty, to guess in the
direction of his or her previous response (see DeCarlo &
Cross, 1990 for discussion). The point for consideration
is that sequential context effects may arise because inter-
nal representation of the current stimulus is affected by
the previous stimulus, and/or because of a tendency un-
der uncertainty to provide a response based on the previ-
Figure 1: Sequential context effects produced by the dia-
ous response. The question is whether similar sequential
betes cue in a multi-cue judgment task where clinicians
context effects operate in expert judgment tasks.
judged patient’s risk for coronary heart disease. Re-
Expert prognostic judgments, such as a clinician’s es-
sponses to the current patient (trial) are categorized ac-
timate of the likelihood that a patient will suffer a heart
cording to consecutive values of the diabetes cue. Subject
attack in the future based on current signs and symptoms,
63 shows assimilation, Subject 59 shows contrast. Plotted
have been studied by judgment researchers employing
values are adjusted for category differences on the eight
multiple regression analysis (e.g., Beckstead & Stamp,
cues.
2007; Harries, 1995; Tape, Kripal & Wigton, 1992). To
illustrate how sequential context effects might manifest
themselves, let us examine what could happen with a sin-
(i.e., prisoner’s dilemma games). In their experiment,
gle dichotomous cue. For example, consider the clinician
playing a random sequence of 96 cooperative and un-
faced with the prognostic task of estimating a patient’s
cooperative games produced greater mean differences in
risk for coronary heart disease (CHD) and say that the cue
cooperation rates (71% vs. 18%) when compared to con-
in question is whether or not the patient has diabetes. The
ditions composed of 48 cooperative games followed by
situation can be described by assimilation if the judgment
48 uncooperative games (33% vs. 18%), and vice versa
of CHD risk for a patient with diabetes tends to be lower
(18% vs. 50%). These differences were analyzed us-
when the immediately preceding patient does not have di-
ing ANOVA on aggregated responses. The authors in-
abetes, and, the judgment of CHD risk for a patient with-
terpreted the signi?cant interaction as support for trial-
out diabetes tends to be higher when the preceding patient
by-trial (local) contrast effects. The current article inves-
has diabetes. Alternatively, the situation can be described
tigates whether such sequential context effects operate in
by contrast if the judgment of CHD risk for a patient with
expert prognostic tasks but takes a different theoretical
diabetes tends to be higher when the immediately preced-
and analytical approach.
ing patient does not have diabetes, and, the judgment of
When sequential context effects associated with a cue
CHD risk for a patient without diabetes tends to be lower
in a multi-cue judgment task are observed, they are here
when the preceding patient has diabetes.
interpreted to mean that a cue’s in?uence (as represented
In psychophysical judgment tasks, such as magnitude
by its ? weight) is altered by the values that the cue takes
scaling, the observed response provided by the subject is
over consecutive trials. Assimilation means that when the
interpreted to be an estimate of the sensory magnitude
cue values on trials t and t ? 1 are different, the cue’s
associated with a given, unidimensional, stimulus inten-
in?uence is smaller than when the cue values on these
sity. In multi-cue judgment tasks where there is not nec-
trials are the same. Contrast means that when the cue
essarily an objective stimulus intensity to be scaled, the
values on trials t and t?1 are different, the cue’s in?uence
response provided by the subject may be interpreted dif-
is larger than when the cue values on these trials are the
ferently. In such judgment tasks, the observed response
same. Although this interpretation may sound odd to a
may be interpreted as an estimate of an integrated judg-
psychophysicist, it is consistent with traditional methods
ment along a more abstract subjective continuum, such as
of demonstrating sequential context effects. For example,
patient-risk for CHD.
one way of demonstrating these effects is to plot the mean
Vlaev and Chater (2007) asked whether contrast and
response to the stimulus value on the current trial as a
assimilation, as observed in psychophysical judgments,
function of the differences between the stimulus value on
would operate when people make estimates of more ab-
the current and immediately preceding trial.
stract magnitudes. They examined estimates of cooper-
Figure 1 is an example of such a plot using data
ativeness made in a series of strategic choice decisions
from two clinicians who participated in a judgment task

---

Judgment and Decision Making, Vol. 3, No. 7, October 2008
Time series and judgment analysis
572
Table 1: Changes in in?uence weight, ?, for the diabetes cue as a function of the cue’s values on consecutive trials.
Subject
Effect
All trials
cue(t) = cue(t-1)
cue(t) = cue(t-1)
63
assimilation
.493
.575
.223
59
contrast
.404
.281
.788
Note: Number of trials total is 80, number of trials where consecutive cue values were equal is 54, number of
trials where cue values were not equal is 25.
wherein they estimated 80 patients’ risk for CHD based
2 Psychology from a time series
on eight cues (the task will be described in detail be-
perspective
low). Here we focus on the diabetes cue for illustration.
Each subject’s responses were analyzed separately and
converted to standardized scores to allow for direct com-
2.1 An introduction to time series
parison. Subject 63 (squares and solid line) shows as-
A time series is a realization of a data-generating process,
similation; the mean rating of risk for the current patient
where observations are equally spaced across time. Fa-
is biased toward the diabetes status of the prior patient.
miliar examples in econometrics include a stock’s daily
Subject 59 (triangles and dashed line) shows contrast; the
price, or quarterly sales ?gures (Yaffee, 2000). In terms
mean ratings are biased away from the diabetes status of
more familiar to psychologists, Gregson (1983) de?ned
the prior patient. Each subject’s responses were also ana-
a time series as a sequence of events ordered in time,
lyzed separately using multiple regression (diabetes was
which we may have good reason to believe is gener-
coded 1 if present, 0 if absent). Table 1 shows some of the
ated by some lawful underlying process that itself persists
results. First, when responses from all 80 trials were an-
throughout the whole duration of the observations made.
alyzed, both subjects shows roughly the same size partial
In psychology the series may be the responses a subject
? weights for the diabetes cue. When the data are catego-
gives on successive trials of an experiment or the amount
rized according to the values of the diabetes cue on con-
of some behavior a client undergoing psychotherapy ex-
secutive trials and re-analyzed, we see how the ? weight
hibits daily over several weeks. The data collected in the
for each subject changes when assimilation or contrast
psychological laboratory, or in ?eld studies, are consid-
takes place.
ered sampled segments from ongoing processes that are
Although these regression analyses illustrate that the
amenable to representation by univariate stochastic dif-
cue’s in?uence changes when assimilation and contrast
ference equations. Most measurements taken in psychol-
take place, this approach is ?awed because of its piece-
ogy may be regarded as discrete realizations of contin-
meal nature; data from two subsets of trials have been an-
uous processes. The trial in a judgment experiment is
alyzed separately. A better approach would be to analyze
conceptually taken as the unit throughout this article and
the data from all the trials simultaneously.
the series of cue values and judgments are considered a
Time series analysis can be used to test hypotheses that
discretely sampled data system.
sequential context effects are operating in the series of re-
Time series analysis is a set of regression-based meth-
sponses obtained from a single subject. Speci?c models
ods for analyzing data ordered sequentially in time. The
can be constructed to isolate sequential context effects
goal of the analysis is to identify patterns in the sequence
produced by the cue (stimulus) series and those operat-
of values, that is, to identify how the values are corre-
ing independently in the judgment (response) series. In
lated with themselves but offset in time, in order to gain
the present article a time series approach is developed by
some insight into the underlying process(es) that gener-
extending ideas discussed in the context of psychophys-
ated the data. A series is decomposed into numerous po-
ical research to cover multi-cue judgment tasks. Before
tential components. One of these is a random process,
preceding to discuss the application of time series anal-
referred to in the parlance of time series theory as a se-
ysis, a broad perspective in which to position time series
ries of “shocks.” Overlaid on these shocks are various
theory and methods in psychological research is needed.
possible patterns. Most obvious of these are trends over
The next section is an attempt to provide such a perspec-
time (including linear and quadratically increasing and
tive. Following this discourse, an illustrative applica-
decreasing means). A second pattern is the lingering ef-
tion is presented with an emphasis on detecting possible
fects of earlier values in the series (i.e., an autoregres-
sequential context effects operating in expert prognostic
sive or AR process), and a third is the lingering effects
judgments of risk for heart disease.
of earlier shocks (i.e., a moving-average or MA process).

---

Judgment and Decision Making, Vol. 3, No. 7, October 2008
Time series and judgment analysis
573
These patterns are not mutually exclusive and all three
may be found in a given time series. Readable introduc-
tions in econometrics are available (see Ostrom, 1990;
Yaffee, 2000) and more rigorous mathematical treatment
may be found in Hamilton (1994). A thought-provoking
monograph, surveying the elementary theory of time se-
ries and indicating where and how its use can increase
insight into psychological processes that extend through
time, has been written by Gregson (1983). His treatise
focuses on data obtained in the psychological laboratory
and will be relied upon heavily throughout this article.
2.1.1 A psychology of organism-environment inter-
actions in time
Figure 2:
Dynamic structure of the organism-
Brunswik (1952) advocated for a psychology of systems,
environment system.
and suggested that the proper subject matter for study
by psychologists is the organism as it interacts with, and
adapts to, objects in its changing environment. Adapta-
2.1.2 Gregson’s dynamic structure of the organism-
tion is a sequential process, and as such, those seeking to
environment system
understand it can bene?t by applying time series theory
and analysis. In tracing the history and thematic rela-
Gregson (1983) offered a framework for considering,
tions of psychology to other sciences, Brunswik (1956)
and identifying, the dynamic structure of the organism-
pressed for aligning the methods and explicit theoriz-
environment system using time series. He realized that
ing of psychology with those of other disciplines he de-
the responses of an organism to its environment are not
scribed as “already recognized as statistical on charac-
static and that adaptation may exhibit natural periodicity.
ter.” Arguably, Brunswik may have recognized the im-
He also recognized that the very act of doing an experi-
portance of sequential context effects in adaptation and
ment in which responses are elicited to a series of stim-
foreseen the relevance of their study for psychology. In-
uli can induce sequential dependency in responses. Like
deed, his schematic representation of history concludes
Brunswik, he recognized that the organism and the en-
with the (at the time of his writing, unrealized) contribu-
vironment form a dynamic system. Gregson’s concep-
tions of time series theory and analysis, citing the work
tual framework may be illustrated graphically (see Fig-
of Wiener (1949). When discussing probabilistic predic-
ure 2). The ?gure highlights the point that the study
tion he mentioned autocorrelation as being useful. Major
of organism-environment relationships is limited to mea-
advances in the theory and mathematics of time series,
sured stimulus-response relationships. Gregson treats the
now taken for granted, occurred after Brunswik’s death,
portion inside the double dashed line as a closed subsys-
notably the work of Kalman (1960) and Box and Jenkins
tem and regards it as the total scope of time series anal-
(1970).
ysis. This closed subsystem contains all the quantitative
Researchers (Hammond, Hursch, & Todd, 1964;
data available to the researcher who wants to investigate
Tucker; 1964), working with Brunswik’s lens model, de-
organism-environment relationships.
veloped the lens model equation which quanti?es and re-
Within this closed subsystem the various structures are
lates the cue-criterion relationships in the environment,
considered in terms of their functional linkages, each of
the cue-judgment relationships, and the correspondence
which may be the focus of one or more time series mod-
between judgments and criterion. As useful as the lens
els. Using Gregson’s notation, we refer to the complete
model equation has proved to be as a framework for con-
set of linkages {l}, and the component linkages are then:
ceptualizing the expert judgment process (see Stewart,
lc = current stimulus-response linkage.
2001), in its current form it does not accommodate au-
ls = linkage within the stimulus series.
tocorrelation that may be present in the judgments and
lr = linkage within the response series.
environment. Although beyond the scope of this article, it
lsr = linkage from previous stimuli to the current re-
may be possible to modify the lens model equation to ac-
sponse bypassing the current stimulus.
commodate sequential effects by incorporating concepts
lrs = linkage from previous responses to current stim-
from time series theory.
ulus (this will be absent unless the stimuli are contingent

---

Judgment and Decision Making, Vol. 3, No. 7, October 2008
Time series and judgment analysis
574
autocorrelation to zero. (Throughout the remainder of
this discussion ls will be assumed absent by design.) In
most psychological research the lr and lsr linkages are as-
sumed (usually implicitly) to be absent. In many operant
studies (and some judgment studies assessing the impact
of feedback on accuracy) the investigator may be inter-
ested in lrs. In Figure 2 lrs is shown as a dotted line be-
cause, while the organism may derive feedback from the
environment, the in?uence of such feedback cannot be
investigated unless the value of the feedback stimulus is
recorded from trial to trial. The {lp} represents what the
organism has learned through past interactions with its
environment. For those readers familiar with Brunswik’s
writings, {lp} may be analogous to what he called the
texture of the environment.
The various linkages in Figure 2 correspond to basic
time series models that may be applied to psychological
data (i.e., to stimulus-response relationships) in general.
If all linkages with the exception of lc are assumed ab-
sent, the model is considered to be outside time and in-
volves no time series analysis. If responses are hypoth-
esized to be generated by an autonomous process (i.e.,
a process that operates independently of the stimulus se-
ries), then only lr is assumed extant and the process is
identi?ed by an autoregressive (AR) structure. When the
current response is hypothesized to be a function of cur-
rent and previous stimuli, (lc and lsr are assumed extant;
lr is assumed absent) the response-generating process is
Figure 3: Schematic representations of multi-cue multi-
identi?ed by a moving-average (MA) structure. More
trial judgment task. (a) Traditional view of judgment task
generally, when the current response is hypothesized to
(outside time). (b) Judgment task viewed from time series
be a function of current and previous stimuli, as well
perspective. Note that the in?uence lines within previous
as previous responses, all three types of linkage (lc, lsr,
trials are not shown in b for visual clarity.
and lr) are assumed extant and the process is identi?ed
by an autoregressive-moving-average (ARMA) structure.
“The general identi?cation problem may be productively
upon previous responses; such linkage can exist if feed-
approached by assuming an ARMA structure and esti-
back has been introduced by the environment, which in-
mating the parameters within it or by seeking directly
cludes the actions of the experimenter).
for a MA or AR solution; as the latter two are restricted
{lp} = past set of linkages, stimuli to responses and
forms of ARMA, this can eventually give the same result”
responses to stimuli.
(Gregson, 1983, p.27). The time series approach outlined
Each of these linkages may be extant, or absent, in a
by Gregson thus incorporates the principles from relative
time series model of current response (e.g., judgment)
judgment models and response heuristic models; DeCarlo
generation. Given the set of linkages {l}, the general
and Cross (1990) develop this idea in detail although they
problem of identi?cation is to decide, using input-output
do not refer to Gregson’s work.
records and notions of causal relations provided by psy-
chological theory, which links are extant and which are
2.1.3 Incorporating time series into judgment anal-
absent. The speci?c problem of identi?cation is one of
ysis
deciding on the details of the algebraic structure and pa-
rameter values that most accurately represent what the
Consider a judgment experiment in which the subject is
links do, given that it is known which are extant.
presented with a series of m pro?les, each composed of k
Various experimental designs may be represented us-
cues, and makes a series of m judgments. Each pro?le-
ing {l}. In most psychological experiments the focus in
judgment is here referred to as a trial, ranging from 1
on lc while ls is absent by design; the stimuli are pre-
to m. On the current trial t, judgment Y(t) is a func-
sented in random order with the intent of reducing their
tion of the current cue values (X1 (t) . . . Xk (t)) and e(t),

---

Judgment and Decision Making, Vol. 3, No. 7, October 2008
Time series and judgment analysis
575
representing residual or unmodeled sources of in?uence
with the output series. For example, over the series of tri-
(see Figure 3a). Following Gregson, the solid lines are
als the correlation between the judgments and a cue’s val-
here referred to as in?uence lines representing the im-
ues, where the cue series is lagged by one trial, de?nes a
pact of the cues and that of the amalgamation of un-
?rst-order cross-correlation; when the cue series is lagged
modeled sources. In regression-based judgment models
by two trials we have a second-order cross-correlation,
(e.g., Hammond, Stewart, Brehmer & Steinmann, 1975)
and so on. For completion, the correlation between cues
the strengths of the cue in?uences are often estimated us-
and judgments concurrent on the same trial is referred to
ing ordinary least squares multiple regression (OLSMR)
as a zero-order cross-correlation. Linear transfer function
coef?cients and are assumed constant across trials; the in-
models consist of two parts; the ?rst part describes the re-
?uence of e(t) may be expressed as 1?R2. In other mod-
lationships among the input and output series and the sec-
els of judgment proposed by investigators working within
ond part depicts the autoregressive structure of the resid-
the Brunswikian tradition (e.g., Dhami & Ayton, 1998,
uals after cross-correlations have been ?tted. Gregson’s
2001; Gigerenzer, Hoffrage & Kleinbölting, 1991), the
linkages (lc, lsr, and lr) may be elegantly represented in
strength of a cue’s in?uence can vary from trial to trial.
this class of time series models.
An assumption shared by all these models is that the cue
I propose that a linear transfer function autoregressive-
values from previous trials do not in?uence judgments on
moving-average model, incorporating general principles
the current trial. This assumption is represented by the
from psychophysical models is the best way to depict se-
vertical lines demarcating the trials in Figure3a. In other
quential context effects that may be operating in multi-
words, these judgment models are outside time, limited to
cue judgment experiments involving several trials. Judg-
lc linkage; the lr and lsr are assumed absent. Subsequent
ment is modeled as a function of the current cue values
discussion will be limited to regression-based models be-
and the immediately preceding values of each cue. This
cause they assume (at least initially) a constant cue in-
relationship corresponds to a relative judgment model
?uence throughout the series of trials and because they
in psychophysics and is identi?ed as a MA1 structure,
include a residual term (e(t)) that is conveniently de?ned
where 1 refers to a ?rst-order cross-correlation. The MA1
mathematically. These two qualities are important in the
structure of the model uses the values of each cue on
proposed time-series-analytic approach developed below.
the current and immediately preceding trial throughout
When considered inside time, the same judgment ex-
the series to provide parameter estimates of the extent to
periment may be depicted as in Figure 3b. The lc, lsr
which the in?uence of the cue is modi?ed by changes in
and lr linkages are assumed extant; ls is absent by de-
its consecutive values.
sign. The lightening of the in?uence lines, from the cur-
The linear transfer function model also includes an
rent trial through the second previous trial, represents the
AR1 error term to represent the portion of the judg-
weakening impact of prior cues and judgments occurring
ment process that cannot be predicted from the cue val-
more distant in the series. Note that in regression-based
ues and their MA1 parameters. Inclusion of this AR1
models of judgment Y(t) = Y
+ e
is the
structure accommodates the gist of the response heuristic
(t)
(t), where Y(t)
portion of the judgment that can be predicted from the cue
model which suggests that people exhibit sequential con-
values (X1 (t) . . . Xk (t)) and their regression coef?cients,
text effects originating in their responses, independently
and e(t) = Y(t) - Y
is the portion of the judgment that
of those operating in their perceptions of stimuli; or in
(t)
cannot be so predicted. As such, the series e(t), e(t?1),
Gregson’s framework the lr linkage is assumed extant.
e(t?2), . . . corresponds to lr and represents the in?uence
The model is speci?ed as:
of prior judgments with the effects of the cues partialed
out via lc. This source of in?uence captures the response
k
heuristic described above. To represent all these linkages
Y(t) = µ +
?0 iXi(t) ? ?1 iXi(t?1) +
and their relationships mathematically a type of time se-
i=1
ries model known as a linear transfer function model may
e(t)
(1)
be used.
1 ? ?e(t?1)
A linear transfer function (LTF) model depicts the re-
lationship between an output series and one or more input
where Y(t) is the value of the judgment on the current
series. This class of time series models characterizes the
trial, µ is the mean of the judgment series, Xi (t) is the
autocorrelation function of the output series and the auto-
value of the ith cue on trial t, ?0 i is a weighting coef?-
correlation function of each input series (each of which is
cient for the ith cue on trial t, ?1 i is a weighting coef?-
zero by design in most judgment experiments), as well as
cient for the ith cue on trial t - 1, e(t) is the residual on the
the cross-correlation functions between each input series
current trial, e(t?1) is the residual on trial t ? 1, and ? is
and the output series. A cross-correlation function de-
an autoregressive weighting coef?cient (limited to range
scribes how lagged values of an input series are correlated
from –1 to 1) for e(t?1).

---

Judgment and Decision Making, Vol. 3, No. 7, October 2008
Time series and judgment analysis
576
In Equation 1 the in?uence the ith cue is conveyed us-
CHD for 80 patient pro?les. Four of the nurse practition-
ing two parameters in order to represent possible sequen-
ers were male. The average age was 48.2 (SD = 6.8).
tial context effects. If the cue is used when forming a
Most (81.3%) worked in primary care settings. On aver-
judgment then the sum of the absolute values of its two
age, subjects had 8.9 years of practice experience (SD =
parameter estimates (?0 i and ?1 i) will be greater than
6.3).
zero. If the cue produces contrast effects during the judg-
ment process these two parameter estimates will have the
same valence (note that ?1 i has a negative sign in Equa-
3.1.2 Materials
tion 1). If the cue produces assimilation, the two param-
eters will have opposite signs. In DeCarlo and Cross’s
Selection of Cues and Outcome Measure. The optimal
(1990) time series model, assimilation versus contrast is
set of risk factors for predicting CHD were identi?ed by
conveyed solely by the sign of ?1 i because ?0 i is always
Anderson, Odell, Wilson, and Kannel (1991a) using a
positive owing to the fact that sensory magnitude is pos-
sample of 5,573 patients followed for over 12 years as
itively correlated with stimulus intensity. In many multi-
part of the ongoing Framingham study of heart disease.
cue judgment tasks some cues will have negative correla-
The equation of Anderson et al. provided regression co-
tions with values on the judgment dimension and so the
ef?cients for eight patient characteristics: gender, age,
signs of both ?0 i and ?1 i are necessary to distinguish
smoking status, total cholesterol level, high-density lipid
contrast from assimilation effects. The magnitude of ?1 i
level (HDL), systolic blood pressure (SBP), and whether
(positive or negative) estimates the extent to which the
or not the patient has been diagnosed with diabetes or
cue’s in?uence changes due to differences in the cue’s
left ventricular hypertrophy (LVH). This “gold standard”
values on consecutive trials. This method of estimating
for predicting CHD was published in the form of a clin-
the change in a cue’s in?uence is more reliable than the
ical worksheet later that same year (Anderson, Wilson,
piecemeal approach used in Table 1 because the estimate
Odell, & Kannel, 1991b). In the current study, judgments
is based on data from all trials rather than subsets of trials.
of patient risk for CHD were made using a 0% to 100%
Equation 1 also accommodates sequential context ef-
response scale.
fects that may be operating in the response series inde-
Choice of Cue Values. A representative design was
pendently of the cues. If the judge has a tendency, in the
used to construct patient pro?les for the judgment task.
face of uncertainty, to guess in the direction of his or her
The risk-factor distributions reported by Anderson et al.
previous response, this form of assimilation will result in
(1991a; 1991b) were used to generate a population of
a positive value of ?. If the judge tends to guess in the op-
cases with similar means, variances, and correlations
posite direction (i.e., contrast) ? will be negative. Thus,
among the eight risk-factor cues. Eighty cases were ran-
the approach assumes lc, lsr, and lr linkages are extant
domly sampled from this population and randomly or-
(ls is absent by design) and the model provides the means
dered for presentation in the judgment task.
for quantifying contrast and assimilation operating within
the cue and judgment series.
As a proof of concept, Equation 1 was ?tted to data
3.1.3 Procedure
from a sample of nurse practitioners who performed
a prognostic judgment task, estimating patient risk for
The materials were presented to each subject in a booklet.
CHD. The goals of this application are (1) to estab-
Booklets contained a cover page describing the purpose
lish the utility of Gregson’s framework for studying
of the study (“to understand how nurse practitioners form
the dynamic structure of the organism-environment sys-
judgments of patient risk for CHD”), instructions for the
tem for examining prognostic judgments, (2) to show
judgment task, the series of patient pro?les presented sep-
that the linear-transfer-function autoregressive-moving-
arately in tabular format, and a brief section requesting
average (LTF ARMA(1,1)) model can ?t such data bet-
basic demographic information. Nurse practitioners were
ter than OLSMR, and (3) to demonstrate that sequential
instructed to “Please read each pro?le carefully and make
context effects exist in prognostic judgment tasks.
an assessment of the patient’s risk for CHD within the
next 10 years on a scale of 0% to 100%.” Nurse prac-
3 An illustrative application
titioners were tested individually and in small groups in
of?ce and classroom settings. After obtaining informed
3.1 Method
consent, instructions describing the judgment task and ac-
3.1.1 Subjects
companying materials were read aloud to subjects. The
procedure took an average of 32 minutes (SD = 11.6) to
Seventy-?ve nurse practitioners completed a prognostic
complete. All subjects received the pro?les in the same
judgment task in which they made estimates of risk for
order.

---

Judgment and Decision Making, Vol. 3, No. 7, October 2008
Time series and judgment analysis
577
3.1.4 Preliminary analyses
OLSMR residuals from a single subject, typical in judg-
ment analysis, the LB test may be more sensitive for de-
Multiple-cue judgment analyses typically use standard-
tecting serial dependence.
ized coef?cients as estimates of a subject’s cue weight-
ing strategies. Prior to ?tting the model in Equation
3.2 Results
1, all variables (cues and judgments) were standardized
for each subject individually to have means of 0 and
3.2.1 Goodness of ?t: Comparing OLSMR and LTF
variances of 1. This was done as a matter of interpre-
ARMA(1,1) on the basis of their R2 values
tational convenience providing a common metric upon
The LTF ARMA(1,1) model was successfully ?tted to 68
which to compare parameters across individuals and also
of the 75 subjects. The other seven required higher-order
because SAS’s PROC ARIMA does not provide standard-
autoregressive terms to identify the AR portion of their
ized parameter estimates. Second, for comparative pur-
judgment models and render the residuals as white noise.
poses, each individual’s judgments were regressed onto
These will be discussed later. Except where mentioned
the cues using ordinary least squares multiple regression
explicitly, the remainder of this section focuses on the
(OLSMR) and residuals were examined for serial depen-
analyses from 68 individuals. The model was ?tted using
dence using two tests described in the next paragraph.
SAS’s PROC ARIMA; parameters were estimated using
Third, for each individual OLSMR was used to con-
conditional least squares rather than maximum-likelihood
?rm that the functional relationships of all cues to judg-
because this method has been shown to perform better
ments were linear (i.e., there were no quadratic trends in
when the number of trials is less than 100 (Yaffee, 2000,
cue-judgment relationships). Fourth, the autocorrelation
pp. 192–204).
function for each cue was assessed to con?rm that ls link-
R2 values ranged from .552 to .928 with an average of
ages were absent by design. Fifth, Dickey-Fuller tests
.800. F tests revealed that the R2 value for each subject
(Dickey & Fuller, 1979) were used to con?rm that all
was signi?cantly larger (p < .05) than his or her OLSMR
judgment series were stationary prior to ?tting the time
R2 value (these ranged from .490 to .908 with an aver-
series model. Stationarity refers to a series having a con-
age of .750). These tests took into account the differing
stant mean and variance.
number of parameters between the two models. A second
There are various methods for assessing serial depen-
method for comparing the ?t of these two models focuses
dence. One of the more well known methods is the
on serial dependence in their residuals. The results of
Durbin-Watson test (Durbin & Watson, 1950; 1951).
these analyses are reported below when discussing tests
Cooksey (1996) discusses how the Durbin-Watson (DW)
for serial dependence.
test may be applied in judgment analysis. An advantage
of the DW test is that it is commonly available as an op-
3.2.2 On determining whether a cue is used when
tional test of residuals in OLSMR procedures of most
forming a judgment
statistical packages (e.g., SAS and SPSS). A disadvan-
tage is that it does not assess serial dependence beyond
Reliance on tests of signi?cance when determining
?rst-order autocorrelation. A second disadvantage of the
whether a cue is being used by an individual in a judg-
DW test is that the derivation of its standard errors (and
ment task has been recently called into question (Beck-
hence, critical values) is not straightforward. Cohen et al.
stead, 2007). An alternative to signi?cance tests for de-
(2003) discuss the DW test in detail. An alternative, and
termining whether or not a cue is in?uential is to focus
in the present application more useful, test is the Ljung-
on effect sizes. In the current application if a change of
Box statistic (Ljung & Box, 1978) that can be used to
one standard deviation in a cue’s value produced at least
assess a series for departures from “white noise” by si-
a .333 standard deviation change on the judgment scale,
multaneously examining autocorrelations over a range of
the cue was considered to have been used by the subject.
predetermined orders. The LB test is a weighted sum of
Although arbitrary, this de?nition is somewhat conserva-
squared autocorrelations. One criterion for identifying a
tive when considered in the context of traditional notions
correct time series model is that serial dependence in the
of effect size (see Cohen, 1988). As each cue was rep-
residuals is reduced to zero (i.e., a white noise process).
resented by two parameters (?0 i and ?1 i) the sum of the
The LB test was developed as a means to make such as-
absolute values of its two parameter estimates had to be
sessments. The LB statistic is distributed as ?2 where a
? .333. For purposes of illustration, in order to have been
nonsigni?cant result indicates the series is free from se-
considered as exhibiting contrast or assimilation the ab-
rial dependence or does not differ signi?cantly from a
solute value of each parameter estimate had to contribute
white noise process. Based on the LB test, 26 individ-
at least .111 to this sum. Using these operational de?ni-
uals exhibited serial dependence; the DW test identi?ed
tions, the distribution of cue utilization was as follows:
only 15 of these. Thus, it appears that when applied to
?ve individuals used only one cue, 19 used two, 26 used

---

Judgment and Decision Making, Vol. 3, No. 7, October 2008
Time series and judgment analysis
578
be the case is not clear. Although purely speculative, it is
Table 2: Number of Individuals Exhibiting Sequential
possible that nurse practitioners were more familiar with
Context Effects on Each Patient Characteristic (Cue) used
the range of values of this cue in relation to heart disease
in Prognostic Judgments of Coronary Heart Disease.
(perhaps it represents a de?ning characteristic) and that
Assimi-
No
this induced stronger memory traces or increased pro-
Cue
Used Contrast
Total
lation
SCEs
cessing of the information provided by the cue which
ultimately manifested as assimilation from one trial to
Gender
no
0
0
64
64
the next. Inspection of cue in?uence rankings revealed
yes
1
0
3
4
that sequential context effects tended to be more com-
mon with higher ranks, that is, they occurred more often
Age
no
0
0
50
50
for cues that carried more weight in the judgments. (See
yes
2
1
15
18
Table 3.) In total there were 38 instances of sequential
SBP
no
0
0
40
40
context effects produced by the cue series (26 instances
of assimilation and 12 instances of contrast). These in-
yes
1
13
14
28
stances were distributed among 30 individuals. Eighteen
LVH
no
0
0
43
43
subjects displayed evidence of assimilation effects only,
yes
0
4
21
25
eight exhibited evidence of contrast effects only, and four
Choles-
showed both assimilation and contrast produced by dif-
no
0
0
48
48
terol
ferent cues.
yes
2
3
15
20
HDL
no
0
0
61
61
3.2.4 Sequential context effects in judgments inde-
yes
1
2
4
7
pendent of cue series
Smoking no
0
0
28
28
The LTF ARMA(1,1) model incorporated a parameter for
yes
1
2
37
40
quantifying the autoregressive structure of the residuals,
Diabetes no
0
0
11
11
that is, the degree of serial dependence in the judgment
yes
4
1
52
57
series that was independent of the cue in?uences. The
results from 19 subjects included signi?cant ? parame-
Note: To be counted as being used in judgment, a one
ters (p < .05). These parameter estimates ranged from
standard deviation change in the cue’s value had to pro-
.264 to .766 with a mean of .433 indicating that assim-
duce a .333 standard deviation change on the judgment
ilation (not contrast) was operating autonomously in the
scale. SCE = sequential context effect, SBP = systolic
responses. When both ? and ? parameters were consid-
blood pressure, LVH = left ventricular hypertrophy, HDL
ered, 39 subjects exhibited evidence of sequential context
= high-density lipids.
effects in the judgment task. For nine of these individu-
als this was limited solely to assimilation effects in the
response series. Twenty exhibited sequential context ef-
three, 12 used four, and six used ?ve. For each subject
fects produced by only the cue series, and 10 displayed
who used multiple cues, the cues used were rank ordered
evidence of sequential context effects operating in both
according to their in?uence (1 being assigned to the most
cue and response series.
in?uential cue). This was done in order to ascertain if se-
quential context effects occurred more frequently as cues
carried more in?uence.
3.2.5 Comparing OLSMR and LTF ARMA(1,1) on
the basis of serial dependence in their residuals
3.2.3 Sequential context effects produced by cue se-
Using the DW test, only 12 of the 39 subjects who
ries
showed any form of sequential context effects in the time
series analysis screened positive for serial dependence in
Each cue was used by at least four of the subjects (see
their OLSMR residuals. The test missed 24 of the 30
Table 2). The most frequently used cue was whether or
who exhibited sequential context effects produced by the
not the patient had diabetes, and the least frequently used
cues series and missed seven of the 19 who showed as-
was patient gender. Each of the cues produced assimila-
similation to previous responses. The DW test produced
tion, contrast, or both, although with varying frequency
no false positives. As noted above, the LB test appears
among subjects. The blood pressure (SBP) cue produced
more sensitive than the DW test for detecting serial de-
sequential context effects, notably assimilation, for the
pendence in OLSMR residuals. Sixteen of the 39 subjects
majority of subjects who used the cue. Why this should
who showed sequential context screened positive for se-

---

Judgment and Decision Making, Vol. 3, No. 7, October 2008
Time series and judgment analysis
579
Table 3: Frequencies of sequential context effects produced by cue series according to in?uence rank.
Cue
Rank
Gender
Age
SBP
LVH
Cholesterol HDL
Smoking
Diabetes
1
0/0
2/7
3/3
2/5
1/6
2/2
1/8
4/37
2
0/1
0/6
6/11
0/8
3/8
1/2
1/17
1/10
3
1/2
0/2
3/10
0/7
0/2
0/2
1/12
0/7
4
0/1
0/2
2/4
1/3
0/3
0/1
0/1
0/3
5
0/0
1/1
0/0
1/2
1/1
0/0
0/2
0/0
Note: Cues were rank ordered according to the size of their parameter estimates (largest assigned rank
of 1). Denominator is number of times cue appeared at each rank. Numerator is number of times cue
exhibited sequential context effect. SBP = systolic blood pressure, LVH = left ventricular hypertrophy,
HDL = high-density lipids.
rial dependence in OLSMR residuals using the LB test.
spending patterns over several years (i.e., December data
The test missed 21 of the 30 exhibiting sequential con-
from year t are correlated with December data from year
text effects produced by the cues series and four of the 19
t ? 1, January data from year t with January data from
who showed assimilation to previous responses. The LB
year t ? 1, etc.). Although purely speculative, in self-
test produced no false positives.
paced judgment tasks, like the one examined here, some
Given the higher sensitivity of the LB test, it was
individuals may experience waxing and waning atten-
used to make comparisons between the two analytic ap-
tion/concentration on the task from trial to trial, or they
proaches. Applying the LB test to the residuals from
may engage in self-monitoring efforts producing alternat-
the LTF ARMA(1,1) analyses for the 16 subjects with
ing response set bias. These cognitive processes might
serial dependence in their OLSMR residuals revealed
possibly manifest as higher-order periodic AR structures.
that LTF ARMA(1,1) residuals were rendered as white
Of course, such higher-order structures might also be the
noise for all these subjects. Thus using serial depen-
result of unknown in?uences associated with conditions
dence in residuals as the criterion, it appears that the LTF
of the experimental context.
ARMA(1,1) model ?t the judgment data better than the
OLSMR model did.
Identifying such higher-order autoregressive structures
is largely an exploratory process. One exploratory ap-
3.2.6 Identifying higher-order autocorrelated struc-
proach uses SAS’s PROC AUTOREG employing its
tures in judgments
backstep option to test the effects of including higher-
order autoregressive parameters on reducing serial depen-
The remaining seven of 75 individuals exhibited per-
dence in the residuals. This option removes nonsignif-
sistent serial dependence in their data after ?tting the
icant autoregressive parameters (analogous to backward
LTF ARMA(1,1) model based on LB tests. Although
elimination in multiple regression) using Yule-Walker
atypical, such ?ndings are not without precedent. Early
equations. (See Brocklebank & Dickey, 2003 for math-
empirical evidence (Holland & Lockhead, 1968) in the
ematical details.) What remains in the model is the
psychophysical realm suggested that autocorrelations up
most parsimonious autoregressive structure that accu-
to eighth-order may be operating in some serial judg-
rately (within prede?ned limits) ?ts the data. This ap-
ments. Later computer simulations by Gregson (1976)
proach was used on the data from these seven individ-
suggested, however, that ?rst- and second-order pro-
uals, testing for ?rst- through eighth-order autoregres-
cesses are more psychologically plausible and that such
sive parameters. This exploratory process identi?ed id-
higher-order ?ndings were likely the result of model mis-
iosyncratic higher-order AR structures (Table 4). Three
speci?cation.
subjects (10, 12, and 42) showed evidence of pure pe-
Higher-order AR structures in the absence of all in-
riodicity; the others displayed more complex structures.
tervening lower-order ones are known as periodic, sea-
Despite these atypical and atheoretical AR structures,
sonal, or cyclic. For example, in economic models of
the majority of these individuals showed evidence of se-
data recorded monthly, it is common to observe AR(12)
quential context effects produced by the cue series; three
structures that re?ect the monthly cyclicity in sales or
showed only cue assimilation effects, two showed only

---