Scientific and Professional Foundations of Clinical Psychology

PART I
Clinical Psychology Research

---

---

CHAPTER ONE
Research Methodology and
Clinical Psychology: An Overview
Michael C. Roberts and Stephen S. Ilardi
Scienti?c and Professional Foundations of Clinical Psychology
The ?eld of clinical psychology has a rich history of empirical research across a number
of domains: assessment, diagnosis, psychotherapy, experimental psychopathology, and
many others (Reisman, 1981; Routh, 1994; Routh and DeRubeis, 1998; Walker, 1991).
In fact, an emphasis on the generation of clinically relevant knowledge through rigorous
research has been a hallmark of clinical psychology from its inception as a distinct ?eld.
Many of the earliest clinical psychologists came to the ?eld with a background in the
natural sciences, integrating their scienti?c proclivities with an interest in generating
practical knowledge as a means of addressing an array of clinical problems. Such a
foundational merging of science and clinical practice was fortuitous, and it has resulted
in a robust empirical foundation for the ?eld. In fact, we would argue that the continued
existence of clinical psychology as a vital discipline is contingent upon both the enduring
soundness of the ?eld’s scienti?c framework and the demonstrable application of its
scienti?c knowledge as a means of improving human lives.
The founder of clinical psychology, Lightner Witmer, established the ?rst psycho-
logy clinic and training program in 1896. Later, Witmer founded and edited the ?rst
scienti?c and professional journal for the developing ?eld, Psychological Clinic. Thus,
even at the outset, there was an implicit recognition of the value of integrated science
and practice. Nevertheless, the research methodologies which characterized most early
clinical psychology investigations (and many of the conclusions derived therefrom)
are generally regarded as ?awed, even primitive, by today’s standards. Clinical psycho-
logy has bene?ted from an ongoing process of scienti?c development and advance-
ment, a process which has tended over time to correct for many methodological and

---

4
Roberts, Ilardi
conceptual foibles (even those vigorously embraced, at one time or another, by most
of the ?eld). In fact, the sensibility of employing scienti?c scrutiny to critically
evaluate and re?ne existing concepts and practices has permeated the history of clinical
psychology.
There are three principal professional roles which have emerged in clinical psycho-
logy – that of clinical scientist (with a primary emphasis on conducting clinical research),
that of scientist–practitioner (re?ecting an equal emphasis on science and clinical prac-
tice), and that of applied clinical scientist (with a preeminent focus on the application
of existing scienti?c knowledge) – and despite their differing emphases, each role
re?ects the ?eld’s intrinsic balance between the scienti?c generation of knowledge
and the applied aspects of clinical assessment and intervention. Clinical science and
practice are inextricably interwoven and reciprocally inform one another, and (for-
tunately) many areas of clinical psychology emphasize their integration rather than
bifurcation.
The need for extensive research training of aspiring clinical psychologists is a point
repeatedly stressed in the ?eld’s historic training conferences (e.g., the famous Boulder
Conference of 1949: Raimy, 1950) and the ensuing reports which have come to de?ne
clinical psychology as a discipline (American Psychological Association Committee
on Accreditation, 2002; Belar and Perry, 1992; Korman, 1976; Roberts et al., 1998;
Trierweiler and Stricker, 1998). This sensibility is also re?ected in the stated program
philosophies, goals, and educational curricula of master’s-level and doctoral programs
in clinical psychology and allied ?elds. For example, the Clinical Child Psychology
Program at the University of Kansas (which one of us, MCR, directs) af?rms in its
philosophy statement that graduates should be “ready for future changes and needs, to
produce original contributions to clinical child psychology, and to evaluate their own
work and others . . . Equally important in the program is the preparation of students
to contribute to and evaluate the scienti?c knowledge base guiding psychological prac-
tice” (www.ku.edu/~clchild). Variations on this and related themes are endorsed by
clinical psychology programs of many different orientations and foci. The consensus
view is that all clinical psychology graduates should be the bene?ciaries of research
training suf?cient to enable them – at a minimum – to critically evaluate the existing
research literature and to engage in informed applications thereof in an array of practice
activities.
Today’s clinical psychologist likely will have more formal training than his or her
predecessors, inasmuch as the amount of material to be mastered has grown com-
mensurate with growth in the ?eld’s scienti?c underpinnings. Due in large part to the
increasingly rigorous research methodology which has come to characterize clinical psy-
chology, the ?eld has witnessed many important advances in recent decades, including
the introduction of novel interventions of high demonstrated ef?cacy, concurrent with
the occasional identi?cation of less effective or even detrimental clinical procedures.
Consequently, professionals in the ?eld – regardless of their level of experience – are wise
to remain abreast of all new developments in the discipline’s science and practice, and
continually to evaluate their own work and that of others in light of relevant scienti?c
advances.

---

Overview
5
Professional and Research Challenges for Clinical Psychology
Numerous challenges confront today’s clinical psychologist, regardless of his or her
theoretical orientation or area of activity, and it is our view that such challenges can
be met successfully only in tandem with a clear research emphasis. Because the full
delineation of all such challenges would be formidable, we will brie?y highlight several
which appear especially noteworthy. First, research is needed to facilitate a deeper under-
standing of the fundamental processes of psychological development (normal and abnor-
mal; prenatal to senescence), as an essential precursor to the ?eld’s development of more
comprehensive models of human behavior. Such enhanced understanding, we believe,
will lead to improved preventive and therapeutic interventions on the part of psycho-
logists and other healthcare professionals. While developmental considerations might
naturally seem most applicable to clinical child practice, adult clinical psychologists
are increasingly recognizing that the process of psychological development continues
throughout adulthood. Thus, improved models of psychological change for adult
clinical psychology are also needed. Moreover, just as child-oriented researchers and
practitioners have long recognized the complexity of families and peers in in?uencing
the process of change over time – as observed, for example, in psychotherapy outcomes
– so too will adult-oriented clinical psychologists need to develop such comprehensive
multi-person systemic conceptualizations. Second (but relatedly), there remains a need
for greater emphasis upon examination of the mediators and moderators of psycho-
logical change (including, of course, therapeutic change) as a means of advancing the
?eld beyond overly simplistic understandings (e.g., this therapy somehow seems to lead to
some improvement for some individuals) toward increasingly sophisticated models which
re?ect more adequately the full complexity of human functioning.
A third contemporary challenge to clinical psychology to be met by research is to
develop clinical assessment devices and methods of greater reliability and validity. Cor-
respondingly, existing diagnostic schemes and taxonomies of psychological disorder are
in considerable need of re?nement on the basis of applied scienti?c investigation. Fourth,
research can help identify valid and invalid psychotherapies, psychological interventions,
and prevention efforts. Improvements in therapy techniques, and in the more precise
identi?cation of the processes by which psychotherapies exert their effects, can be accom-
plished through targeted research informed by the methodologies outlined in this hand-
book. Measurement of treatment procedures, treatment integrity, behavioral changes,
functional performance, objective measurements, perceptions of change, and satisfaction
from a variety of sources, follow-up assessment, etc., are needed to establish the “scienti?c
credentials” of each therapeutic approach. Fifth, measurement of the range of outcomes
following psychotherapies and preventive interventions can help establish the associated
costs and bene?ts associated with each. Relevant outcomes can include all aspects of a
patient’s life, such as personal perceptions and functioning, work, and signi?cant rela-
tionships (parents, spouses, friends, siblings, offspring). Additionally, research is required
to determine the costs, bene?ts, and harm of clinical psychology activities (e.g., assess-
ment, prevention, therapy) – both with respect to direct as well as indirect effects of such

---

6
Roberts, Ilardi
activities (e.g., practice patterns and charges for psychologist’s time; medical cost offsets,
insurance reimbursement patterns). The effects of psychological practice (and research)
on society in general stand in great need of more rigorous investigation.
A sixth domain of professional challenge and research effort concerns evaluation of
the organization and delivery of a variety of clinical services through program evaluation.
There is an ongoing need within the ?eld for evaluative frameworks, methodologies, and
instruments that may be applied across the wide variety of settings (e.g., inpatient/
outpatient units; clinics and hospitals; private practice) and problems faced by clinical
psychology (e.g., different sets of psychologically related symptoms and diagnoses). At
this time, clinical psychology is no longer a single specialty, but is now an amalgam
of more specialized substantive foci: clinical child, pediatric, adult clinical, clinical
neuropsychology, geropsychology, health, and others. The varieties of these foci require
development and acceptance of a multitude of approaches within the scienti?c traditions
of the overarching ?eld of clinical psychology.
A seventh challenging issue, as noted by the Clinical Treatment and Services Research
Workgroup (1998) of the National Institute of Mental Health, is re?ected in the fact
that improvement in research and clinical practice requires an iterative investigational
process across a continuum of treatment research emphases: ef?cacy (i.e., demonstrated
treatment-related improvements as observed in controlled research studies), effectiveness
(i.e., the degree to which the treatment is ef?cacious across the wide array of individuals
and therapists found in real-world settings), practice (i.e., how services are delivered), and
service systems (i.e., how mental health services are structured). The translation of re-
search to applied clinical settings with the aim of improving practice is clearly important;
equally important, however, is the principle that the research itself be informed by
psychology practice. Finding the appropriate mechanisms by which to accomplish such
translating/informing actions poses an ongoing challenge for clinical researchers. Finally,
informing each of the aforementioned current and future challenges is the fact that
clinical psychologists conduct research and practice in an increasingly diverse society,
especially in the United States. Populations underserved by mental health service provid-
ers are typically those which have been under-researched as well. Finding ways to in-
crease the representativeness of participants in clinical research will enhance the ?eld’s
ability to respond effectively to each of its principal challenges.
Numerous commentators have highlighted these and other complex challenges facing
clinical psychology at present (e.g., Compas and Gotlib, 2002). For example, similar
issues have been articulated speci?cally for the area of pediatric psychology (e.g., Brown
and Roberts, 2000; Roberts, Brown, and Puddy, 2002) and clinical neuroscience (Ilardi,
2002), areas in which we have personal interests. We encourage readers of this handbook
to remain alert both to the delineation of such challenges as they are outlined in detail in
the chapters to follow, and to the many exciting future research opportunities discussed
in the book’s ?nal chapter. It is our hope that the highlighting of such challenges and
opportunities will serve to help catalyze research in such areas for decades to come. We
recognize, however, that some of the ?eld’s current assumptions and enthusiasms – even
some of those emphasized in this text! – will likely be replaced over time as the evidence
mounts (as it inevitably does). Indeed, new and completely unanticipated questions will
doubtless arrive at the of?ces, clinics, and laboratories of clinical researchers and practi-

---

Overview
7
tioners. Nevertheless, the research methods and principles outlined in this handbook, we
believe, will remain important to the ?eld’s advancement in the years ahead.
Purpose and Overview of this Handbook
Some students (and even some graduated professionals) approach the general topic of
“research” with a groan, a dread of boredom, or even with unmitigated fear and loathing
– this despite perhaps a grudging recognition of the necessity of research training as a
means of ful?lling requirements of courses and/or theses and dissertation projects. Still
others view research and the scienti?c process as interesting detective work, a means of
solving important problems and resolving questions tinged with the thrill of discovery. It
is this latter sense of excitement at the prospects of discovery which we seek to emphasize
in this handbook, though with a clear recognition that individual results may vary.
The organization of this handbook re?ects the editors’ attempt to be comprehensive
in coverage, i.e., not providing merely a collection of essays related to research, but an
integrated framework allowing the reader to see a broad range of methodologies and
their respective applications in advancing the science and practice of clinical psychology.
In developing this book we wanted the contributors to convey the excitement of con-
ducting empirical research, utilizing a variety of methodologies, to answer a broad range
of enormously important questions facing clinical psychology at present. As noted, such
questions may be regarded as challenges to be met through the use of evidence-based
approaches outlined herein.
We hope that this book meets the needs for a concise textbook for students, instructors,
professionals, and scientists interested in expanding their base of knowledge regarding
research methods in clinical psychology. The chapters cover the major approaches
to research and design for clinical psychology, with attention to both child and adult
populations. In addition, brief research vignettes describe examples of projects with
exemplary designs and methodologies as a means of illustrating the essential elements of
many of the research topics covered herein. This handbook consists of twenty chapters,
each covering a different facet of clinical research. The ?rst two parts of the text examine
important issues which affect all clinical researchers – areas such as ethics, research
validity, research designs, methodology, and data analysis; the third part focuses on
speci?c topical areas of application in clinical psychology. For many of the latter topics,
separate discussions are provided for research with adult and child populations, inas-
much as the research with these populations has become increasingly specialized and
independent (although common questions and methods are highlighted as well).
Part one on Clinical Psychology Research covers topics of important relevance to all
aspects of scienti?c work in the ?eld. In fact, these are areas which require the researcher’s
continual attention when applying the content of later chapters on methodology and
focal research topics. In a foundational chapter, Michael S. Finger and Kevin L. Rand
describe the manner in which con?dence in the professional psychologist’s ?ndings (and
clinical activities) is contingent upon careful attention to numerous validity issues. The
authors de?ne and illustrate four principal types of research validity concerns (internal,

---

8
Roberts, Ilardi
external, construct, and statistical conclusion) and illustrate ways of addressing them.
They also elucidate many common potential threats to validity in clinical psychology
research, and discuss strategies for addressing in simultaneous fashion internal and ex-
ternal validity concerns in research projects. In chapter 3, Michael C. Roberts, Keri J.
Brown, and Julianne M. Smith-Boydston outline issues germane to moving research
through the review process to the publication end stage. They discuss how to determine
what is publishable, how to select a publication outlet, how to prepare a manuscript, and
many possible outcomes of the editorial review process. In chapter 4, William A. Rae
and Jeremy R. Sullivan elucidate ethical considerations in clinical psychology research.
These authors articulate important ethical concerns that may arise in each of four phases
of the research process: research planning, institutional review boards, informed consent,
and analysis and write-up for publication. They focus special attention on issues of
con?dentiality, research with vulnerable populations (including children), and use of
deception and recording (e.g., audio/video).
In part two of this handbook the focus shifts to the foundational research designs and
statistical approaches requisite to conducting appropriate research on the central ques-
tions posed in clinical psychology. In chapter 5, Cynthia M. Anderson and Christine
Kim describe speci?c strategies for examining data obtained from the individual psycho-
therapy client, as opposed to larger groups of participants. Derived from applied behavior
analysis, these single-case techniques are particularly applicable to heuristic, exploratory
investigations in the early stages of intervention research, as well as for practicing
clinicians attempting to evaluate the effects of their therapeutic activities. Anderson and
Kim note that single-case approaches are widely applicable to clinical psychology prac-
tice, regardless of the theoretical orientation of the practitioner. Next, in chapter 6,
Andrea Follmer Greenhoot discusses the design and analysis of experimental and quasi-
experimental investigations. She presents the principal types of experimental designs and
the set of related statistical techniques commonly used to investigate between-group
differences on key variables (e.g., to evaluate the effects of a psychotherapy intervention
versus a control condition). In chapter 7, Charles M. Judd and Melody S. Sadler focus
attention on the analysis of datasets in which the variables of interest are measured as
they are found (observational data); i.e., the key variables are not manipulated in an
experiment. These authors address the conceptualization of correlational research, the
pragmatic concerns of correlational data analysis, and strategies for the resolution of
interpretational dif?culties. In chapter 8, Samuel B. Green and Marilyn S. Thompson
describe a speci?c form of statistical analysis which has become widely used by psycho-
logical scientists over the past two decades: structural equation modeling. Clinical psycho-
logy research involves the examination of human behavior and change via increasingly
complex theoretical models capable of representing causal interrelationships among a
large number of variables over time; structural equation modeling provides one such
useful modeling approach. In chapter 9, Gloria L. Krahn and Michelle Putnam describe
the applicability of qualitative research in clinical psychology. They demonstrate how
qualitative research, if undertaken systematically and with proper training, may consti-
tute a useful scienti?c approach. They outline principles involved in selecting qualitative
techniques, the practical applications of the various qualitative methods, and optimal
ways to resolve challenges of sampling, data collection techniques, and analyses. Part two

---

Overview
9
concludes with chapter 10, Joseph A. Durlak’s treatment of the basic principles of meta-
analysis as applied to clinical psychology topics. He notes that meta-analytic techniques
are useful statistical methods of reviewing and summarizing clinical psychology research
that may be dispersed across many studies. Durlak describes the basic methodology of
meta-analysis and provides examples to illustrate his points. He also notes that meta-
analytic studies help elucidate problems with extant research studies and indicate where
further work is needed.
In the third and ?nal part of this handbook, a wide range of more focal topics of
research is considered. Many of these topics are covered across two separate chapters,
with emphases on child and adolescent versus adult populations, respectively. In chap-
ter 11, Eric M. Vernberg and Edward J. Dill outline developmentally oriented research
frameworks for examining the manner in which psychological problems emerge, inten-
sify, and remit. Although the term developmental psychopathology is often thought to refer
exclusively to child/adolescent disorders, developmental approaches are those based on
consideration of change over time (and thus applicable to adults as well). Vernberg and
Dill present the core research issues in this area by means of a series of “research tasks”
for research in developmental psychopathology. Chapter 12 has a parallel focus on
psychopathology research among adult populations. Written by John P. Kline, Steven
D. LaRowe, Keith F. Donohue, Jennifer Minnix, and Ginette C. Blackhart, this chapter
describes the manner in which experimental psychopathology encompasses the invest-
igation of causal mechanisms associated with psychological disorders across multiple
intersecting levels of analysis (e.g., neurophysiological, cognitive, affective, interpersonal,
etc.). As the term implies, experimental psychopathology derives from the tradition of
lab-based experimental psychology, and involves the application of experimental principles
and methods to the study of psychological disorders. Both psychopathology chapters
demonstrate the importance to clinical psychology of the ongoing development of a
scienti?c knowledge-base regarding the processes through which psychological problems
develop and progress.
In the book’s next two chapters, the emphasis shifts to the assessment and diagnosis of
children and adults, respectively, with extensive coverage given to research methodolo-
gies used to develop assessment instruments and to conduct empirical evaluations thereof.
Diagnostic assessment has always been an important aspect of clinical psychology, and
the ?eld continues to witness important new conceptualizations and evaluative approaches
in this area. In chapter 13 on child and adolescent assessment and diagnosis research,
Paul J. Frick and Amy H. Cornell demonstrate the techniques of psychological assess-
ment with children and the applicability of scienti?c research techniques in evaluating
the instruments used in assessment. Throughout their chapter, Frick and Cornell indi-
cate that, all too often, instruments used in psychopathology research are different from
those which are useful in applied clinical assessment settings with children and adoles-
cents. In chapter 14, Thomas E. Joiner, Jr., and Jeremy W. Pettit describe the primary
conceptual issues germane to research in the area of clinical assessment and diagnosis,
and they suggest several strategies for implementing research with the existing array of
clinical assessment techniques. In particular, they highlight three common approaches
used in this work – structured clinical interviews, symptom scales, and projective tests
– and discuss the degree to which the extant empirical literature which supports (or fails

---

10
Roberts, Ilardi
to support) major assessment instruments within each of these domains. The authors
also highlight limitations associated with the ?eld’s DSM-based diagnostic classi?cation
system, and suggest ways of facilitating research progress in assessing and diagnosing
psychopathology.
Another signi?cant area of activity for clinical psychologists has been the develop-
ment, evaluation, and application of psychotherapeutic interventions for the various
clinical concerns. In chapter 15, Ric G. Steele and Michael C. Roberts detail therapy
and interventions research with children, youths, and families. These authors emphasize
empirically supported treatment approaches and discuss such issues as ef?cacy, effect-
iveness, treatment selection, study participant selection, internal and external validity,
and treatment integrity. In chapter 16 on therapy and interventions research with
adults, Anne D. Simons and Jennifer E. Wildes provide an overview of issues central to
conducting psychotherapy research with adults. They explain that such research exam-
ines whether an intervention works, how and why it might work, factors which might
affect its ef?cacy, and how long the effects might last. The authors also provide an
overview of the methods and current trends in research regarding the effects of adult
psychotherapy.
An important aspect of clinical psychology, sometimes neglected, is the fact that
often the most ef?cient means of alleviating distress is to intervene before any problems
are evident – for example, by creating healthier psychological environments for at-risk
individuals, especially during temporal windows of vulnerability at key stages of devel-
opment. Consequently, in chapter 17 on research in prevention and promotion, George
C. Tremblay and Barbara Landon emphasize that a developmental perspective underlies
most effective prevention approaches. They detail the salient issues facing prevention
research in clinical psychology, and describe the prevailing methodologies for conduct-
ing scienti?cally sound research on prevention programs. In an overview of material
germane to each of the aforementioned topics in part three, in chapter 18 Yo Jackson
explicates research in ethnic minority communities. She calls for greater multicultural
competence among clinical psychology researchers, and describes the research challenges
raised by an ethnically diverse population in the need for more research with different
groups. She attends to the conceptual and pragmatic issues of conducting such research
in order to generate useful ?ndings, while remaining attentive to the importance of
accounting for cultural differences.
As clinical psychology has developed as a profession, it has increasingly examined a
range of professional issues, such as training and education, ethics, licensing and cred-
entialing, practice, and service activities. The methodologies requisite for the empirical
investigation of such issues are described in chapter 19 by Michael C. Roberts, Jodi L.
Kamps, and Ephi J. Betan. The authors report on existing research covering a range of
topics and methodologies, such as surveys regarding outcomes of training (e.g., student
placement) and attitudes about various issues affecting the ?eld (e.g., managed care,
ethics), clinical case analysis and practice pattern studies, and even research on the research
activities of clinical psychologists.
Finally, in chapter 20, Stephen S. Ilardi and Michael C. Roberts focus attention on a
number of important windows of opportunity for scienti?c discovery in the discipline of
clinical psychology in the years immediately ahead. They give primary coverage to areas

---