The Politeness Effect 1

The Politeness Effect 1
Running head: THE POLITENESS EFFECT


The Politeness Effect: Pedagogical Agents and Learning Outcomes
Ning Wang1, W. Lewis Johnson2, Richard E. Mayer3, Paola Rizzo4, Erin Shaw5, Heather Collins6



1Institute for Creative Technologies
2Information Sciences Institute
University of Southern California
University of Southern California
13274 Fiji Way,
4676 Admiralty Way,
Marina del Rey, CA 90066 USA
Marina del Rey, CA 90066 USA
Email: nwang@ict.usc.edu
Email: johnson@isi.edu



3Dept. of Psychology
4Dept. of Computer Science
University of California, Santa Barbara
University of Rome “La Sapienza”
Santa Barbara, CA 93106-9660 USA
Via Salaria 113, 00198 Roma Italy
Email: mayer@psych.ucsb.edu
Email: pr04@libero.it


5Information Sciences Institute
6Dept. of Psychology
University of Southern California
University of California, Santa Barbara
4676 Admiralty Way,
Santa Barbara, CA 93106 USA
Marina del Rey, CA 90066 USA
Email: hcollins@umail.ucsb.edu
Email: shaw@isi.edu

Corresponding Author:
Ning Wang
1Institute for Creative Technologies
University of Southern California
13274 Fiji Way,
Marina del Rey, CA 90066 USA
Email: nwang@ict.usc.edu
Tel: +1(310) 574-5700
Fax: +1(310) 574-1631

---

The Politeness Effect 2
Abstract
Pedagogical agent research seeks to exploit Reeves and Nass’s Media Equation theory, which
holds that users respond to interactive media as if they were social actors. Investigations have
tended to focus on the media used to realize the pedagogical agent, e.g., the use of animated
talking heads and voices, and the results have been mixed. This paper focuses instead on the
manner in which a pedagogical agent communicates with learners, i.e., on the extent to which it
exhibits social intelligence. A model of socially intelligent tutorial dialog was developed based
on politeness theory, and implemented in an agent interface within an online learning system
called Virtual Factory Teaching System. A series of Wizard-of-Oz studies was conducted in
which subjects either received polite tutorial feedback that promotes learner face and mitigates
face threat, or received direct feedback that disregards learner face. The polite version yielded
better learning outcomes, and the effect was amplified in learners who expressed a preference for
indirect feedback, who had less computer experience, and who lacked engineering backgrounds.
These results confirm the hypothesis that learners tend to respond to pedagogical agents as social
actors, and suggest that research should focus less on the media in which agents are realized, and
place more emphasis on the agent’s social intelligence.

---

The Politeness Effect 3
The Politeness Effect: Pedagogical Agents and Learning Outcomes
1. Introduction
Researchers have for several years been investigating the potential of pedagogical agents to
promote learning. One of the most influential papers in this area was a study by Lester, Converse,
Kahler, Barlow, Stone and Bhogal (1997) that demonstrated a Persona Effect, in which learning
was facilitated by an animated pedagogical agent that had a life-like persona and expressed affect.
The rationale for this research has been the media equation hypothesis of Reeves and Nass (1996).
The media equation holds that people tend to respond to interactive media much as they do to
human beings. That is, they respond as if the media were social actors.
A number of pedagogical agent investigations have since been conducted, seeking to
understand the Persona Effect in more detail, and replicate it in a range of learning domains
(Johnson et al., 2000; Moreno et al., 2001). The results of these studies have been mixed. For
example, a study by André, Rist and Müller (1998) showed that animated agents reduce the
perceived difficulty of the material being learned, and a study by Bickmore (2003) showed that
subjects liked an animated agent that responded socially to them, but neither study reported
significant learning gains. Moreover, several studies (Atkinson et al., 2005; Graesser et al., 2003;
Mayer et al., 2004; Mayer et al., 2003; Moreno and Mayer, 2000, 2004) indicated that the agent’s
voice was the significant contributor to learning outcomes, not the animated persona. Thus the
Persona Effect is at best unreliable, and may in fact be a misnomer if the animated persona is not
the primary cause of the learning outcomes.
This paper examines a different approach to applying the media equation hypothesis to
intelligent tutoring. If as Reeves and Nass (1996) suggest learners respond to pedagogical agents as
if they were social actors, then the agents’ effectiveness should depend upon whether or not they

---

The Politeness Effect 4
behave like social actors. The agents should be socially intelligent, acting in a manner that is
consistent with their social role, in accordance with social norms. In fact, human tutors make
extensive use of social intelligence when they motivate and support learners (Lepper et al., 1993).
Thus, social intelligence in pedagogical agents may be important not just to gain user acceptance,
but also to increase the effectiveness of the agent as a pedagogical intervention.
To test this hypothesis, a model of motivational tutorial tactics was developed, based upon
politeness theory (Brown and Levinson, 1987; Johnson et al., 2004). A preliminary series of
Wizard-of-Oz studies was conducted in which subjects either received polite tutorial feedback that
promotes learner face and mitigates face threat, or received direct feedback that disregards learner
face. The polite version led to improvements in learning outcomes, and the effect was amplified in
learners who expressed a preference for indirect feedback. We also observed effects on learner
attitudes and motivation (Wang et al., 2005).
We term the effect demonstrated here the Politeness Effect. Our results suggest that
pedagogical agent research should perhaps place less emphasis on the Persona Effect in animated
pedagogical agents, and focus more on the Politeness Effect and related means by which
pedagogical agents can exhibit social intelligence in their interactions with learners.
2. Related work
In recent years, the recognition of the importance of affect and motivation in learning has led
increasingly to the development of socially-aware pedagogical agents as reflected in the work of
Del Soldato and du Boulay (1995) and De Vicente and Pain (2002). Heylen et al. (2003) highlight
the importance of these factors in tutors, and examine the interpersonal factors that should be taken
into account when creating socially intelligent computer tutors. Cooper (2003) has shown that
profound empathy in teaching relationships is important because it stimulates positive emotions and

---

The Politeness Effect 5
interactions that favour learning. Baylor and Ebbers (2003) have conducted experiments in which
learners interact with multiple pedagogical agents, one of which seeks to motivate the learner. Other
researchers such as Kort et al. (Aist et al., 2002; Kort et al., 2001), and Zhou and Conati (2003)
have been addressing the problem of detecting learner affect and motivation, and influencing it.
User interface and agent researchers are also beginning to apply the Brown and Levinson model of
politeness to human-computer interaction in other contexts (Andre et al., 1998; Cassell and
Bickmore, 2003; Miller, 2002).
Porayska-Pomsta (2004) used the Brown and Levinson model of politeness to analyse teacher
communications in classroom settings. Although there are similarities between her approach and
the approach described here, her model makes relatively less use of face threat mitigating strategies.
This may be due to the differences in the social contexts being modelled.
2.1. Politeness Theory
Brown and Levinson (1987) have devised a cross-cultural theory of politeness, according to
which everybody has a positive and negative “face”. Negative face is the want to be unimpeded by
others (autonomy), while positive face is the want to be desirable to others (approval). Some
communicative acts, such as requests and offers, can threaten the hearer’s negative face, positive
face, or both, and therefore are referred to as Face Threatening Acts (FTAs). Consider a critique of
the learner such as “You did not save your factory parameters. Save them now.” There are two
types of face threat in this example: the criticism of the learner’s action is a threat to positive face,
and the instruction of what to do is a threat to negative face.
In the Brown and Levinson model, evaluation of face threat depends upon several factors.
First, the relative weightiness of different face threats is culturally dependent. The weightiness of
a face-threatening act also depends upon the relative power between the speaker and the listener.

---

The Politeness Effect 6
Tutors generally have power relative to learners, so we would generally expect tutors to make use
of weaker politeness strategies when speaking to learners than the learners use in reverse. Finally,
the weightiness of a face threat depends upon the social distance between the two parties. As two
people interact over time their social distance often decreases, reducing the severity of face
threatening acts and increasing the likelihood that bald-on-record strategies (i.e., direct requests
that lack face-saving features) will be used.
Speakers use various politeness strategies to mitigate face threats, according to the severity, or
“weightiness”, of the FTA. In the above case (“You did not save your factory parameters. Save
them now.”), the tutor could omit the criticism of the learner and focus on the suggested action, i.e.,
to save the factory parameters. Alternatively the tutor could perform the face-threatening act off
record, i.e., so as to avoid assigning responsibility to the hearer. An example of this would be “The
factory parameters need saving.” The face threat of the instruction can be mitigated using negative
politeness tactics, i.e., phrasing that gives the hearer the option of not following the advice, e.g.,
“Do you want to save the factory now?” Positive politeness strategies can also be employed that
emphasizes common ground and cooperation between the tutor and learner, e.g., “How about if we
save our factory now?” Other positive politeness strategies include overt expressions of approval,
such as, “That is very good”.
2.2. Analyzing politeness in tutorial interactions
To investigate the role of politeness in learner-tutor interaction, we videotaped interactions
between learners and a human tutor while the students were working with a particular on-line
learning environment, the Virtual Factory Teaching System (VFTS) (Dessouky et al., 2001). VFTS
is a web-based factory modeling and simulation system. Students read through an on-line tutorial
in a Web browser, and carried out actions on the VFTS simulation as indicated by the tutorial.

---

The Politeness Effect 7
Learners were supposed to analyse the history of previous factory orders in order to forecast future
demand, develop a production plan, and then schedule the processing of jobs within the factory in
order to meet the demand. The tutor sat next to the students as they worked, and could interact with
them as the student or the tutor felt appropriate. Completing the entire scenario required
approximately two hours of work, divided into two sessions of around one hour each. To analyse
the interactions, and use them in designing learner-agent dialog, we transcribed them and annotated
them using the DISCOUNT scheme (Pilkington, 1999).
The politeness theory of Brown and Levinson (1987) proved to be effective in accounting for
the tutorial tactics observed in these dialogs. The following patterns of politeness strategies were
associated with each type of tutor support (listed from most to least frequent):
• Suggesting actions:
o To avoid threatening student negative face, the tutor mostly applied negative politeness
strategies, e.g.: “You will probably want to look at the work centres”, or “Want to look
at your capacity?”. A negative politeness strategy used quite often by the tutor is
“conventional indirectness”: a compromise between the desires to be direct and to be
indirect, resulting in a communicative act that has a non-literal meaning based on
conventions. Examples from our transcripts are: “They're asking you to go back and
maybe change it”, or “What they're telling you is to go and try to get the error terms”.
This strategy enables the tutor to deflect to the system or interface the responsibility of
requesting the student to perform an action.
o In other cases the tutor chose a positive politeness strategy, by phrasing suggestions as
activities to be performed jointly by the tutor and the learner, e.g.: “So why don’t we go

---

The Politeness Effect 8
back to the tutorial factory…”, or emphasize common goals between the tutor and the
learner, e.g. “Run your factory, that’s what I’d do.”
• Providing feedback:
o Negative feedback might threaten the student’s positive face, so the tutor mostly used
off-record politeness strategies, e.g.: “So the methodology you used for product 1
probably wasn’t that good.” In some cases, the tutor provided feedback by promoting
interest and reflection, as well as affecting face, using “socratic” communicative acts
such as: “Well, think about what you did…”.

In a preliminary study (Mayer et al., 2006), students were asked to rate 16 tutorial statements
on negative politeness (i.e., how much the tutor allows me freedom to make my own decisions”)
and positive politeness (i.e., how much the tutor was “working with me”). Consistent with Brown
and Levinson’s (1987) politeness theory, students rated direct commands and commands attributed
to machines as lowest in negative and positive politeness, rated guarded suggestions and guarded
questions as highest in negative politeness, and rated guarded suggestions and statements
expressing a common goal as highest in positive politeness. This sensitivity to the politeness of on-
line tutor’s statements was stronger for students with low rather than high computing experience.
These results have implications for the design of polite conversational agents in educational
software, which were directly tested in the present experiment.
2.3. Motivation Theory
Why is it that tutors attend to learner face, and seek to mitigate threats to learner face? It
could be a simple matter of common courtesy, as in any polite discourse. We conjecture that
there is more to it than that—that tutors employ politeness strategies in order to promote
learning. More precisely, we hypothesize that there is a relationship between learner face and

---

The Politeness Effect 9
learner motivational states, and that politeness strategies can affect the impact of tutorial tactics
on motivational states. The motivational state of the learner during learning can in turn have an
impact on learning outcomes.
Motivation is derived from the Latin verb movere (to move). In Pintrich and Schunk
(2002), motivation is defined as the process whereby goal-directed activity is instigated and
sustained. Motivation can influence what, when, and how we learn (Schunk, 1991). Students who
are motivated to learn about a topic are apt to engage in activities they believe will help them
learn, such as attending carefully to the instruction, mentally organizing and rehearsing the
material to be learned, taking notes to facilitated subsequent studying, monitoring their learning
progress, evaluating how well they’re doing and asking for help when they don’t understand the
material (Zimmerman and Martinez-Pons, 1992). Collectively, these deep learning activities can
improve the learning outcome. Also, motivation bears a reciprocal relation to learning:
motivation influences learning and what students learn influences their motivation (Schunk
1991).
Over the years, there has been disagreement about the nature of motivation and the
operation of motivational process. However, researchers agree on the behavioral indexes that
indicate the presence of motivation. The commonly used indexes are choice of tasks, effort and
persistence. Choice of task is a natural indicator of motivation. When students have a choice,
what they choose to do indicates where their motivation is. Despite the intuitive appeal of choice
of tasks, choice often is not a useful index of motivation in schools or experiment settings
because in many classroom and experiments, students typically have few if any choices (Brophy,
1983). Effort becomes the second index since learning often is not easy. Students motivated to
learn are apt to expend effort to succeed. Persistence, or time spent on a task, is also commonly

---

The Politeness Effect 10
used by researchers as a measure of motivation. Students who are motivated to learn tend to
persist at the task, especially when they encounter obstacles. Students who choose to engage in a
task, expend effort, and persist are likely to achieve at a higher level (Pintrich and Schrauben,
1992; Schunk, 1991).
Knowing the behavioral indication of motivation, tutors increasingly are exploring how to
structure cognitive and social factors to optimize learner motivation. The work of Lepper on
highly effective human tutors (Lepper et al. 1997; Lepper and Woolverton 2002) shows that up to
50% of the human tutor’s interactions with their students are focused on affective elements. This
is because learning, especially deep learning, tends to be affiliated with negative emotions, in the
face of cognitive disequilibrium (Piaget, 1952). Cognitive disequilibrium has a high likelihood of
activating conscious, effortful, cognitive deliberation, questions, and inquiry that aims to restore
cognitive equilibrium. The affective states of confusion, and perhaps frustration, are likely to
occur during cognitive disequilibrium (Kort, Reilly and Picard, 2001). Moreover, Lepper and
Hodell (1989) identified several factors that contribute to learner motivation: curiosity,
confidence, challenge, and control. We seek to create pedagogical agents that promote learners’
self-confidence and control, in particular, by influencing the learner’s sense of autonomy and
approval.
2.3.1. Sense of Autonomy
Sense of autonomy is a feeling that your action is unimpeded by others. Sense of autonomy
does not directly translate to “control” in educational psychology literature. The notion “control”
has two aspects of meaning. The first aspect refers to freedom of choice. Students are in control
if they are free to choose what they want to learn, set learning goals, make and follow plans to
achieve their goals. Even in a classroom or experiment settings, when a learning task is given,

---