Why do students fail social statistics?
Paul T. von Hippel
Unpublished manuscript. Correspondence: Paul T. von Hippel, Department of Sociology and
Center for Population Research, Ohio State University, 300 Bricker Hall, 190 N. Oval Mall,
Columbus OH 43210, von-hippel.1@osu.edu.
I thank Shelley Pacholok for keeping the careful and detailed gradebook on which this research is
based.

---

ABSTRACT
Social statistics is often the most failed course in the undergraduate sociology
major. In this paper we consider three theories of sociology undergraduates’ weak
performance in statistics courses. The first theory ascribes students’ difficulties to poor
math and logic skills. The second theory implicates lack of effort. The third theory points
to distractions such as heavy courseloads and full- or part-time jobs.
Using data from a course of 45 students, we regressed midterm and final exam
scores on measures of skill, effort, and distraction. The results suggest that low exam
scores were strongly related to weak math skills and to neglect of assignments;
distractions were relatively weak predictors of exam scores. We recommend that statistics
courses include appropriate prerequisites and review materials in mathematics. We also
discourage policies that encourage reduced effort—e.g., the policy of dropping the lowest
assignment grade.

Why Students Fail—2

---

INTRODUCTION
Sociology is a largely quantitative discipline, yet this is rarely why
undergraduates choose the sociology major. To the contrary, statistics is often the most
feared course in the sociology curriculum (Bridges et al. 1998). In our department,
introductory statistics is repeated more than any other course. Although many students
succeed in statistics, at least 17% either drop it or fall short of the C- required for
graduation.1
If we wish to do a better job equipping undergraduates with the quantitative tools
of our discipline, we need a better understanding of their difficulties. In the interest of
understanding the causes of high and low achievement in statistics courses, this study
tests three theories with broad currency among instructors.
1. Some instructors attribute students’ difficulties to distraction, such as heavy
courseloads or full- and part-time jobs.2 Research on national samples of college
students has found little relationship between grades and paid work (Ehrenberg &
Sherman 1987), but the effect of courseload seems relatively unexamined.
2. Some instructors attribute differences in student achievement to differences in
effort, for example attendance and completion of assignments. This is consistent
with psychological research suggesting that skill acquisition is closely linked to
time spent in deliberate practice (Ericsson et al. 1993).
3. Some instructors attribute students’ difficulties to poor math and logic skills.
Compared to students in other industrialized countries, US high school seniors
have low levels of math literacy, but a high probability of attending college (Barro

Why Students Fail—3

---

and Lee 2001, Mullis et al. 1998). It is predictable that students entering college
with poor math skills would have difficulty in a statistics course.
Note that these theories may be related. For example, math and logic skills may
be a result of prior effort, and present effort may be reduced by distractions. Although my
analyses explore some of these causal paths, my goal is not to trace students’ difficulties
to their ultimate causes. Instead, I try to identify proximate causes that might be altered
by classroom interventions.
To test these theories, I used measures of distraction, effort, and skill to predict
midterm and final exam scores. To preview the results:
1. Measures of distraction had little predictive value. As in studies of high school
students (Schoenhals et al. 1998), I found that paid work had a negligible effect
on exam scores when course effort was taken into account. Similarly, I found that
courseload had little effect on exam scores, though it may have affected effort in a
few extreme cases.
2. Measures of effort did a better job of predicting exam scores. Although
attendance seemed to make little difference, neglect of assignments caught up
with students by the final exam. This makes sense since attendance can be a
relatively passive activity, while homework assignments require the kind of
“deliberate practice” (Ericsson et al. 1993) required to build cognitive skills.
3. The most useful predictors were measures of math and logic skills. The math
pretest was an excellent predictor of both the midterm and final exam scores. The
logic pretest did not predict midterm scores, but did predict scores on the final
exam. This makes sense since the midterm consisted largely of calculating
descriptive statistics,
while the final also

Why Students Fail—4

---

emphasized logical arguments using statistical inference.
Although we were initially surprised by these results, they are consistent with
prior research. In general, educational research often finds that the greatest influences on
achievement have their impact outside the classroom and before the course begins (e.g.,
West et al. 2000; Downey, von Hippel, and Broh 2004). In particular, previous studies of
introductory sociology students implicate pretests and prior achievement, while finding
little effect of attendance or paid work (Szafran 1986; Neuman 1989). Previous studies,
however, have not extended these results to statistics courses, and have not considered
the effects of courseload and assignment completion.
The findings have changed our department’s understanding of students’
difficulties, and have provoked changes not only in my personal teaching style, but in
departmental discussions regarding advising and prerequisite courses. I will discuss these
changes later, under the heading of “Recommendations.”
DATA AND MODEL
Variables
In winter 2003, 45 students enrolled in my social statistics course—22 sociology
majors, 16 criminology majors, 3 double majors, and 4 students visiting from other
departments. Throughout the term, a variety of information was collected on these
students, including the following variables pertinent to student success:
1. Skill variables. On the first day of the term, students were given brief pretests on
MATH and LOGIC skills. The logic pretest consisted of 3 original questions

Why Students Fail—5

---

illustrating the types of reasoning required in inferential statistics. The math
pretest consisted of 12 questions drawn from a longer pretest in a statistics
textbook (Gravetter and Wallnau 2000, Appendix A).3 To facilitate comparison,
both pretests were scored on a 0-3 scale; each logic question was worth a full
point, and each math question was worth a quarter-point. Both pretests are
reproduced in the Appendix.
2. Distraction variables. On the first day of class, students were given a
questionnaire that asked, among other things, about their COURSELOAD: “How
many classes are you taking this quarter (including this one)?” The questionnaire
also asked “How many hours a week do you work for pay? (Answer 0 if you
don’t have a job.)” Paid hours of work was divided by 8 to give the length, in
days, of the student’s WORKWEEK.
3. Effort variables. Throughout the quarter, a graduate teaching assistant collected
assignments and kept attendance records. From these records we could tell how
many ASSIGNMENTS students SKIPPED, and how many WEEKS they were ABSENT,
both BEFORE and AFTER THE MIDTERM. Before the midterm there were 4
assignments in 5 weeks. After the midterm there were 2 assignments in 3.75
weeks. These figures omit days lost to snow or exams, and omit one last, short
assignment that was not collected due to a snow emergency.
4. Exam scores. About sixty percent of the course grade was based on a MIDTERM
and a FINAL EXAM. The midterm covered descriptive statistics, as well as
confidence intervals for a single mean or proportion. The final reviewed these
materials, and also covered confidence intervals and hypothesis tests for
comparing two or
more means or

Why Students Fail—6

---

proportions. Both exams were scored out of 100 points; the final included 6 points
of extra credit. Although the exams were not assigned letter grades, they
contributed to a course grade where scores in the 90s were As, scores in the 80s
were Bs, and so on.
The distributions of these variables are summarized in Figure 1. The top row
shows that final exam scores were generally lower than midterm scores. Subsequent rows
suggest some reasons why. After the midterm, attendance slacked off and students were
more likely to skip an assignment. The course moved from descriptive to inferential
statistics, so that logic as well as math skills became important.
FIGURE 1 HERE

Although Figure 1 is suggestive, the suggested relationships do not necessarily
explain why final scores were lower than midterm scores. It could simply be that the final
exam was more difficult. A formal model is needed to test the proposed relationships.
Model
The data were modeled using a system of two linear equations. In the first
equation, MIDTERM scores were modeled using measures of skill (MATH and LOGIC
pretests), measures of distraction (length of WORKWEEK and size of COURSELOAD), and
effort (WEEKS ABSENT, ASSIGNMENTS SKIPPED) from before the midterm. In the second
equation, FINAL scores were modeled using the same variables, as well as measures of
effort from after the midterm. More formally,
MIDTERM = β
β
0m + X1 1m + em




(1)
FINAL = β
β
β
0f + X1 1f + X2 2f + ef




(2)

Why Students Fail—7

---

where
X1 = (MATH, LOGIC, WORKWEEK, COURSELOAD,
WEEKS ABSENT BEFORE MIDTERM, ASSIGNMENTS SKIPPED BEFORE MIDTERM)
X2 = (WEEKS ABSENT AFTER MIDTERM, ASSIGNMENTS SKIPPED AFTER MIDTERM)
and where both error terms (em, ef) are normally distributed.
I expected that students who scored unexpectedly high (or low) on the midterm
might also score high (or low) on the final. To model this possibility, I allowed the errors
in predicting the midterm (em) to be correlated with errors in predicting the final (ef). The
error correlation relates this pair of seemingly unrelated regressions (Wooldridge 2002).
Missing values
Since Figure 1 reports less than 45 cases for each variable, it is evident that our
data have some missing values. There were diverse reasons for this. One student did not
take the initial pretests and questionnaire, and so is missing values for MATH, LOGIC,
COURSELOAD, and WORKWEEK. Another student did take the questionnaire, but neglected
the question regarding COURSELOAD. One student dropped the course before the midterm,
and so is missing both exam scores and all measures of effort. Another student dropped
the course after the midterm, and so is missing the final exam scores and measures of
effort after the midterm. One student, was given customized assignments and tests; these
were not comparable to those of other students and so were treated as missing.
The major cause of missing values, however, was a university policy affecting the
5 graduating seniors. Students who have registered to graduate at the end of the term
must be given an early “senior final,” after which they are not expected to attend class.
Since the senior final was
shorter and less

Why Students Fail—8

---

comprehensive than the regular final, and since the students taking the senior final had
less time to prepare, final exam scores were treated as missing from the records of
graduating seniors. These students were also missing values for how much of the course’s
last week they would have attended, had they been expected to do so. One approach to
the missing attendance records would be to treat these students post-midterm absences as
missing. A more careful approach is to break post-midterm absences into two
components, absences before and after the senior final. Only the second component is
missing values for graduating seniors. In effect, this means that we are adding a third
equation to our model.
WEEKS ABSENT AFTER MIDTERM =
WEEKS ABSENT BETWEEN MIDTERM AND SENIOR FINAL
+ WEEKS ABSENT AFTER EARLY FINAL




(3)
Unlike the first two equations, however, the third equation is deterministic instead of
random, and has no parameters to estimate.
I handled these missing values using multiple imputation, which replaces each
missing value with a sample of plausible values or imputations (Allison 2002). My
analyses used 10 imputations per missing value, which should be quite adequate given
the small number of missing values (Rubin 1987). My imputation model assumed that the
missing values are normally distributed4; however, I rounded and truncated the imputed
values to be consistent with other values in the gradebook.
Results using multiple imputation were similar to results using two alternative
approaches to missing values: (1) listwise deletion, which deletes any student with
missing values; and (2) normal maximum likelihood, which maximizes the likelihood of

Why Students Fail—9

---

all the values in both complete and incomplete student records, assuming a normal
distribution of missing values (Allison 2002; Little and Rubin 1989). Listwise deletion is
the most common method for handling missing values, though not the best. Maximum
likelihood approach is most appropriate for larger samples, but it provides useful chi-
square tests of model fit, which I report below.
RESULTS
The overall fit of the model w as reasonably good (χ2(4)=4.65, p=.33). Table 1
gives point estimates and 95% confidence intervals for the model parameters.
TABLE 1 HERE

Skill variables
The only significant predictor of midterm scores was the math pretest. On
average, an extra point on the 3-point math pretest predicted 0.27 to 17.33 extra points on
the 100-point midterm. In other words, the effect of math skills was almost surely
positive, and may have improved midterm scores by more than 1½ letter grades.
The math pretest was also a significant predictor of final exam scores. On
average, an extra point on the 3-point math pretest predicted 3.41-19.93 extra points—1/3
to 2 letter grades—on the final exam. That is, an extra point on the math pretest would,
on average, change a final-exam C to something between a C+ and an A. The logic
pretest affected the final nearly as much. An extra point on the 3-point logic pretest
improved the final exam score by 2.22-15.86 points—suggesting that the final, more than
the midterm, tested logical as well as computational skill. This makes sense, since the

Why Students Fail—10

---

Document Outline

• 
• ﾿
• • 
  • ﾿
• 
  • ﾿
  • 
  • ﾿
• 
  • ﾿
• 
  • ﾿
  • 
• ﾿
• 
  • ﾿
• 
• ﾿

---