ARTICLE IN PRESS
International Journal of Educational Development XX (2004) XXX–XXX
www.elsevier.com/locate/ijedudev
Closing the digital divide: evaluation of the World Links
program
Robert Kozma ∗, Ray McGhee, Edys Quellmalz, Dan Zalles
Center for Technology in Learning, SRI International, 2151 Filbert Street, San Francisco, CA 94123, USA
Abstract
In response to this digital divide between developed and developing countries in their use of computers to prepare
students for the global economy, the World Bank and, subsequently, the World Links organization provided schools
in developing countries with networked computers and training that supports integration of ICT into teaching. This
article synthesizes findings from 3 years of evaluative research on the program. The findings are based on surveys of
teachers, students, administrators, and technology coordinators, as well as a field test assessment of student learning.
The study examines the services provided, documents their impact, and draws implications for policy in developing
countries.
 2003 Elsevier Ltd. All rights reserved.
Keywords: International development; Educational technology; Educational policy
1. Introduction
development of a highly skilled workforce—can in
turn
contribute
to
technology
development.
Information and communication technology
Together, these two developments can create a
(ICT) can make a tremendous contribution to
“virtuous circle” that can reduce poverty and
human development—but only for those that have
improve the human condition.
access. Technological innovation affects human
Unfortunately,
these
same
technological
development in two ways (United Nations Devel-
resources are currently creating significant dispari-
opment Programme, 2001). It can directly contrib-
ties between developed and developing countries
ute to human capabilities by increasing people’s
according to a United Nations Development Pro-
ability to participate more actively in the social,
gramme report (1999). For example, the report
educational, economic and political life of a com-
points out that in South Africa, the best-connected
munity. It can also support economic growth
African country, 75% of the schools have no tele-
through the productivity gains that it generates.
phone line. For the sake of comparison, there are
Conversely, human development—particularly the
99 telephones for every 100 people in Monaco, but
in Uganda, there are only two for every 1000. The
report concludes that, “The global gap between
∗
have and have-nots, between know and know-nots,
Corresponding author. Tel.: +1-415-292-2471; fax: +1-
415-292-4343.
is widening” (p. 57).
E-mail address: robert.kozma@sri.com (R. Kozma).
This growing digital divide is further docu-
0738-0593/$ - see front matter  2003 Elsevier Ltd. All rights reserved.
doi:10.1016/j.ijedudev.2003.11.014

---

ARTICLE IN PRESS
2
R. Kozma et al. / International Journal of Educational Development XX (2004) XXX–XXX
mented by several more recent reports. Bridges.org
addition to strong economic performance. The
(2001) states that while there were 167 million
development of educational capacity in client
Internet users in North America and 113 million
countries is a key element of the Bank’s strategy.
in Europe, there were only 16 million in South
While the primary mechanism for Bank support is
America and 3 million in all of Africa, according
through its program of strategic loans, the Bank’s
to data at the time of the report. While there has
Economic Development Institute (now the World
been a dramatic increase in Internet users in
Bank Institute, http://www.worldbank.org/) con-
developed countries—from an estimated 4% of the
tributes to capacity building through training and
inhabitants in 1995 to 28% in 2000—the number
outreach programs (World Bank, 1997).
of users in developing countries has only inched
The Economic Development Institute initiated
forward to only 3.6% in 2000. Similarly, a study
the World Links for Development in the belief that
by the Organization for Economic Cooperation and
technology could be used to improve educational
Development (OECD, 2001b) shows that there has
outcomes in client countries and facilitate cultural
been a dramatic growth of Internet hosts—the
understanding across nations (World Bank, 1997).
source of content on the Internet. But while the
The program was started by connecting a single
number of hosts in developed countries has
high school in Uganda with one in the United
increased from 23 per 1000 inhabitants in 1997 to
States. By 1999, the program reached over 300
82 per 1000 in 2000, the number in developing
schools in 15 countries. In 2000, the World Bank
countries has only increased from 0.21 per 1000 in
created an independent non-profit organization
1997 to 0.85 per 1000 in 2000. This lack of content
called World Links (http://www.world-links.org/)
originating in and oriented to developing countries
to continue the program once the pilot was com-
is further compounded by the fact that 94% of the
pleted. World Links has since grown to serve over
world’s Internet content, as measured by the num-
a 1000 secondary schools in 26 developing coun-
ber of links to pages on secured servers, is in
tries in Africa, Latin America, the Middle East, and
English.
South and Southeast Asia.
The lack of access to technology reduces the
The World Links program aims to establish glo-
prospect that citizens of developing countries will
bal, educational on-line communities for secondary
be able to participate in the growing global econ-
school students and teachers around the world in
omy and minimizes the potential that technology
order to expand distance learning opportunities,
has
for
improving their health,
educational,
enhance cultural understanding across nations,
governmental, and cultural institutions (United
build broad support for economic and social devel-
Nations Development Programme, 1999).
opment, and train teachers to integrate information
technology into the classroom. There are five
components to the World Links program, each
2. The World Links program
elaborated below:
In response to this growing digital disparity and
Internet connectivity for secondary schools in
to requests from developing countries to assist
developing countries.
them in preparing their youth to enter an infor-
Training and educational content to promote
mation age and participate in the global economy,
economic and social development.
the World Bank’s Economic Development Institute
Regional and global partnerships with public,
conducted a pilot program, initially called World
private, and non-governmental organizations.
Links for Development, from 1997 to 2002. The
Telecommunications policy advice for the edu-
mission of the World Bank is to fight poverty and
cation sector.
contribute to sustained development (World Bank,
Monitoring and evaluation support.
1999). In its comprehensive development frame-
work, the Bank posits that sustainable development
Internet connectivity has been an essential
requires many social and structural elements in
component of the World Links program. For coun-

---

ARTICLE IN PRESS
R. Kozma et al. / International Journal of Educational Development XX (2004) XXX–XXX
3
tries and schools with the most need, the program
The program also provided ministries of edu-
provided up-to-date multimedia computer equip-
cation and other governmental agencies with con-
ment and networking for the school and computer
sultation and policy materials that supported the
laboratory. The program provided several alterna-
development of ICT and its use to improve edu-
tive networking models, depending on the net-
cational systems. Among the advice was the advis-
working resources and bandwidth that were avail-
ability of drawing on World Bank lending
able.
resources to support additional programs that use
Training was also an essential component. A
ICT in light of the results of the pilot program in
central tenet of the program is that the introduction
their country.
of technology will alone not improve education.
The fifth component of the World Links pro-
Teachers from participating schools received train-
gram was monitoring and evaluation. To inform
ing in the use of equipment and typical software.
the decisions of national policy makers and pro-
But more important was training on the integration
gram managers, an evaluation was designed to
of ICT into classroom teaching. The program
examine
the
World
Links
program
on
the
adopted a constructivist approach to teaching with
implementation and effectiveness of the inter-
ICT that emphasized a shift away from teacher-
vention and barriers that need to be overcome to
centered lecture-based instruction towards student-
assure its success. The evaluation was conducted
centered, project-based learning (Carlson, 2002).
by the Center for Technology in Learning at SRI
Among the ultimate goals of the program was to
International (http://www.sri.com/policy/ctl/).
encourage students’ development of higher-order
This article summarizes the findings from evalu-
thinking and information-reasoning skills. These
ative research that was conducted over a 3 year
skills are cited as particularly important for partici-
period (Kozma and McGhee, 1999; McGhee and
pation in the knowledge-based global economy
Kozma, 2000; Quellmalz and Zalles, 1999, 2000).
(OECD, 1996, 2001a; 21st Century Partnership,
The focus of the evaluation was on documenting
2003). To accomplish this goal, World Links train-
the services actually provided by the program,
ing was to provide teachers with pedagogical
studying the classroom implementation of the pro-
approaches to incorporate ICT into their teaching
gram and the effect that it had on classroom prac-
and support student acquisition of advanced skills
tice, examining initial indications of student and
instructional goals. By 2000, the World Links
teacher outcomes and long-term impacts, and ana-
teacher training curriculum included the following
lyzing the barriers that may limit the effectiveness
four phases:
of the program. Our findings indicate that the pro-
Phase I: Introduction to the Internet for teaching
gram provided a variety of intended services to
and learning.
schools and teachers and that these were effec-
Phase II: Introduction to educational telecollabor-
tively implemented in participating schools and
ative projects.
classrooms. The use of ICT in these schools was
Phase III: Integrating technology and curricula.
associated with student-centered, constructivist
Phase IV: Evaluating and diffusing innovative
pedagogical practices. The data we were able to
classroom practices.
obtain indicate that students acquired the skills
they would need to participate in the global knowl-
The program also created a set of regional and
edge economy. A range of barriers—from the need
global partnerships, such as I∗EARN, GLOBE,
for more reliable connections in schools, to the
and ThinkQuest, that created digital content, activi-
need for more time in the curriculum for computer-
ties, and training materials for teachers and stu-
based activities and national policies that integrate
dents. These partnerships and their services sup-
ICT into education reform—limited the program’s
ported teachers in their implementation of ICT and
implementation and impact. But the extent of the
the pedagogical strategies advocated by World
program’s success despite these barriers has impli-
Links.
cations for the use of ICT to support education

---

ARTICLE IN PRESS
4
R. Kozma et al. / International Journal of Educational Development XX (2004) XXX–XXX
improvement in developing countries and for the
World Links program was to contribute to such
policies that can increase its impact.
outcomes as improving the technological and
information skills of students and teachers and
long-term impacts such as improved test scores,
3. Study design
increased
cross-cultural
understanding,
and
improved job prospects in the developing knowl-
As mentioned above, the goal of our evaluation
edge economy. However, these ultimate goals are
was to document the implementation of the World
mediated by a variety of intermediate factors, most
Links program and assess its effectiveness. The
salient being the extent to which the services pro-
focus of the evaluation was on the services pro-
vided by the program are implemented and sup-
vided by the program, the extent to and ways in
ported by the participating schools. In addition,
which these services were implemented in the
there are other “exogenous” variables unrelated to
classroom, and student and teacher outcomes and
the program which may influence the implemen-
the extent to which they could be attributed to the
tation of the program and its outcomes and
program. In addition, we were concerned with the
impacts, factors over which the program and the
barriers that teachers encountered in implementing
Bank may have very little influence. We used this
the program. Our purpose was to provide program
conceptual framework to guide the construction of
staff with information that could be used to
our research instruments. Within this framework,
improve the program and to provide governments
our focus was on the services of the program, the
with information about the value of ICT invest-
ways their implementation affected classroom
ments in education and polices that may be needed
practice, and the impact of these practices, parti-
to increase the impact of these investments.
cularly on the skills students would need to partici-
The evaluation of the World Links program was
pate in the knowledge economy.
situated within a conceptual framework in which
Previous research suggests that the services pro-
the impact of educational improvement efforts is
vided by the World Links program—specifically
influenced by a number of variables, in addition to
the ICT infrastructure and teacher training—would
the intervention itself, which mediate the impact of
have an effect on classroom practice, particularly
the program (see Fig. 1). The ultimate goal of the
the training of teachers to integrate ICT into their
Figure 1.
Conceptual Framework for World Links Evaluation

---

ARTICLE IN PRESS
R. Kozma et al. / International Journal of Educational Development XX (2004) XXX–XXX
5
classes. Classroom studies (Means and Olson,
ever, we triangulated findings by the use of mul-
1995; Sandholtz et al., 1997; Means et al., 2001;
tiple sources (students, teachers, technology coor-
Schofield and Davidson, 2002) have shown that
dinators, and administrators), periods (the program
such interventions can have a direct effect on the
in 1999 and 2000), participating countries, and
pedagogical strategies used by teachers, the extent
methods (Tashakkori and Teddlie, 1998; Shavelson
to which computers are used within these stra-
and Towne, 2001). Survey instruments were
tegies, the amount of time students are engaged in
designed to collect information on the services pro-
the use of computers, and the kinds of projects that
vided by the program, teacher and student class-
students conduct with computers. The examination
room practices and the extent to which computers
of this relationship was a primary focus of the
contributed to these, the effect of the program and
evaluation.
the use of computers on teacher and student knowl-
The classroom implementation, so influenced,
edge, skills, and attitudes. We collected data from
may have a subsequent effect on short-term out-
students, teachers, technology coordinators, and
comes, such as the acquisition of certain knowl-
administrators. Also, sets of performance assess-
edge, skills, and attitudes. Ultimately, the inter-
ments were specifically designed to directly meas-
vention may have longer-term impacts on student
ure the impact of the program on student learning.
retention rates or their preparation for the work
To address certain research questions related to
world. The goal often cited in the literature is to
relative effects of the program vs. the mere use
provide students with the ability to access, analyze,
of computers, we included comparison groups of
evaluate, communicate, and use information to
teachers and students who used computers but did
solve
problems
and
create
new
knowledge
not participate in the program, as described below.
(Educational Testing Service (ETS), 2002; Inter-
national Society for Technology in Education
3.1. Surveys of students, teachers, and
(ISTE), 2000; OECD, 2000; Quellmalz and
administrators
Kozma, 2003)—skills that are needed to participate
in the global knowledge economy. However, the
The purpose of the surveys was to collect self-
relationship between the use of ICT and various
reported responses about the program services
outcomes and impacts is an area where research
received, the implementation of these services in
results are less conclusive, with some studies find-
the classroom, and the outcomes and impact of the
ing negative results (Pelgrum and Plomp, 2002;
program on students and teachers. Survey data
Wenglinski, 1998) and some finding positive
were collected during the 1998–1999 and 1999–
results (NCES, 2001; Wenglinski, 1998). The con-
2000 operational years. During 1998–1999, survey
flicting results suggest that the mere introduction
data were collected from samples of adminis-
of computers is insufficient to affect outcomes; the
trators, teachers, technology coordinators, and stu-
way computers are used in the classroom matters.
dents from five countries. During 1999–2000, sur-
In our evaluation, we examined this relationship
vey data were collected from more countries (12
between the use of computers, the use of specific
in all) but only from teachers.
pedagogical practices, and certain student out-
In 1998–1999, the evaluation was conducted in
comes. We designed this as a comparative study
five of the 13 currently participating countries:
so that some attributions could be made to the ser-
Chile, Paraguay, Peru, Senegal, and Uganda. In
vices provided by the program.
each of the five countries in which the evaluation
The evaluation was also situated within a practi-
was conducted, six World Links schools were
cal context of limited time and budget. As in many
asked to participate in data collection. Because the
evaluations, these pragmatic constraints influenced
responses were self-reports, the data were triangu-
the way in which we were able to assess the goals
lated with multiple types of respondents. At each
and examine the relationships imbedded in our
school, samples of students and teachers were sur-
conceptual framework. We had to rely on the self-
veyed along with the headmaster or headmistress
report of participants for much of our data. How-
and the school’s technology coordinator. In total,

---

ARTICLE IN PRESS
6
R. Kozma et al. / International Journal of Educational Development XX (2004) XXX–XXX
26 World Links schools participated in the evalu-
respondents used computers, they would then be
ation. There were nearly 20,000 students served by
asked what role computers played in these prac-
these schools, more than 12,000 girls and nearly
tices. If they were participants in the World Links
8000 boys. There were over 1200 teachers in these
program, they would then be asked the extent to
schools. A total of 661 World Links students and
which their participation contributed to the prac-
83 World Links teachers responded to the survey
tices. Consequently, the occurrence of certain prac-
from these schools. Also 25 administrators and 23
tices could be compared between program parti-
technology coordinators responded. The response
cipants and non-participants and between computer
rate was at or above 80% for all schools. Because
users and non-users to see if there were differ-
we wanted to assess the extent to which student
ences.
Subsequent
questions
could
provide
and teacher outcomes could be attributed to the
additional information about the extent to which
World Links program, two other groups were
the respondents felt that the use of computers or
identified for comparison purposes. One was a
the participation in the program contributed to any
group of non-participating or “least participating”
differences that existed.
teachers and students within World Links schools.
In 1999–2000, 12 of 15 countries that were part-
These were teachers and students in participating
icipating in the World Links program at the time
schools who did not participate in the program or
were included in the study. These were: Brazil,
participated in a minimal way; respondents in this
Chile, Colombia, Ghana, Mauritania, Mozam-
group included 191 teachers and 441 students. A
bique, Paraguay, Peru, Senegal, South Africa,
second comparison group was administrators, tea-
Uganda, and Zimbabwe. The intent of this study
chers, and students from nine schools that did not
was to see if 1998–1999 findings held for new
participate in the program but were asked to par-
countries that had subsequently joined the pro-
ticipate in the evaluation based on their compar-
gram. As mentioned above, data were collected
ability to World Links schools. Within these
only from teachers. The design included both
schools, 90 teachers, 378 students, and nine admin-
World Links and non-World Links schools: 98 and
istrators responded to surveys. To assure parti-
42, respectively. In World Links schools, four tea-
cularly useful comparisons, both computer using
chers were included for each school. To examine
and non-computer using teachers and students
the impact of the program on a broader range of
were surveyed in non-World Links schools. Of the
teachers in participating schools, two of the four
total respondents in non-World Links schools, 32
were teachers who had received formal training
teachers and 247 students were computer users.1
from the program; one had received training from
Relating to pedagogical practices and outcomes,
a colleague; and one teacher had not received train-
survey questions were constructed so that identical
ing. A total of 383 World Links teachers partici-
responses could be collected from both World
pated in the survey and 158 teachers responded in
Links participants and non-participants and both
non-World Links schools; the response rate was
computer users and non-users. For example, ques-
97%.2
tions were asked of all respondents about the use
of practices such as collaborating with other stu-
3.2. Assessment of student learning
dents on a project, gathering data for a research
project, writing project reports, and so on. If the
Because we wanted to go beyond self-report of
the impact of the program on student outcomes,
we devised a direct measurement of student
1 The total number of respondents by country in 1998–1999
was: Students—Chile (255), Paraguay (222), Peru (390), Sen-
egal (262), and Uganda (350); teachers—Chile (48), Paraguay
2 The total number of teacher respondents by country in
(47), Peru (109), Senegal (76), and Uganda (84); technology
1999–2002 was: Brazil (56), Chile (40), Columbia (42), Ghana
coordinators—Chile (6), Paraguay (4), Peru (6), and Uganda
(40), Mauritania (36), Mozambique (32), Paraguay (36), Peru
(6); administrators—Chile (6), Paraguay (6), Peru (7), Senegal
(56), Senegal (56), South Africa (56), Uganda (55), and Zim-
(8), and Uganda (6).
babwe (36).

---

ARTICLE IN PRESS
R. Kozma et al. / International Journal of Educational Development XX (2004) XXX–XXX
7
achievement. There were some unique challenges
species would survive. Students were also asked to
in the design of student learning assessments for
insert and annotate a relevant graphic. A parallel
this study. First, the World Links program was
“paper-and-pencil”
version
of
the
task
was
implemented in a wide range of subject areas,
designed for students in the comparison group who
including science, social studies, language, and
did not use ICT. The assessments were pilot tested
computer science. Furthermore, different countries
in a World Links school in Uganda and in Parag-
had different curricula and expectations for student
uay. Scoring rubrics were developed and scorers
learning. Both these factors made it problematic to
were trained to assess the quality of students
design an instrument that would measure the learn-
reasoning with information, communication, and
ing of specific school subject matter. However, an
technology skills.
overriding consideration in designing the student
The student assessment field test conducted in
assessment was the interest in measuring one of
2000 involved six World Links schools and four
the most important outcome goals of the program:
comparable non-World Links schools in Uganda.
to prepare students for work in the knowledge-
There were a total of 200 students assessed; 121
based global economy. Consequently, rather than
from World Links schools and 79 from non-World
measure the learning within the multitude of sub-
Links schools. Students in non-World Links
ject areas in which the program was implemented,
schools were not assessed on their technology
the assessment of student learning focused on ICT
skills.
skills and the use of ICT to search for, organize,
and communicate information. These are the kinds
of skills that are most likely to be influenced by the
4. Results
program, regardless of the subject area in which it
is implemented, and they are the skills identified
4.1. Services provided
in policy documents (ISTE, 2000; OECD, 1996,
2001a; World Bank, 2002) as those needed for
Our evaluation began with an analysis of the ser-
workers in the global knowledge economy.
vices that the World Links program actually pro-
With this goal in mind and drawing on the Edu-
vided to participating schools. The World Links
cational
Technology
Standards
for
Students
program is a distinctive blend of infrastructure and
developed by the International Society for Edu-
human capacity development. Perhaps, the most
cational Technology (ISTE, 2000), the SRI team
visible contribution of the World Links program is
identified key skill components for three outcome
the donation of computer hardware and software.
areas: reasoning with information, communication,
Equipment was provided only to those schools
and technology use. The SRI team then designed
where the current lack of equipment limited their
sets of performance assessments that engaged stu-
participation in the program. At the time of the
dents in complex tasks of searching for, organiz-
1998–1999 survey, the technology coordinators in
ing, and analyzing information and designed indi-
60% of the schools reported that their schools had
cators that measured skill components, such as
received equipment from the program. The coordi-
formulating a search query and communicating an
nators reported having received an average of
argument (Quellmalz and Zalles, 1999, 2000). As a
nearly nine computers for each school. The large
culminating task in one of the assessments, World
majority of these computers were located in a com-
Links students were asked to write a newsletter for
puter laboratory along with other computers that
other students about the plight of two endangered
the school had previously acquired.
species. In response to this task, students gathered
One of the outcomes of the World Links pro-
information from Web pages about the two
gram—one
that
significantly
increases
its
endangered species, specified a line of inquiry for
efficiency—is the additional resources that the
further research and a Web search query phrase,
donation of equipment and participation in the pro-
and then prepared a news article making evidence-
gram fostered. The relatively modest donations
based predictions about the likelihood that the two
from the program were leveraged to get additional

---

ARTICLE IN PRESS
8
R. Kozma et al. / International Journal of Educational Development XX (2004) XXX–XXX
resources, as they were often matched by signifi-
the survey in 1999–2000, due in part to the
cant local or national contributions. For example,
inclusion of a broader range of teachers in the
the computer laboratories were often built with
World Links schools. In these schools, 62% of the
funds from a country’s Education Ministry or from
teachers said they received training on designing
local resources. In Senegal, the Ministry of Edu-
and leading collaborative groups; 52% received
cation built or remodeled laboratories at each of
training on using student groups in teaching; and
the World Links schools at a cost in excess of $
45% received training on collaboration with teach-
20,000 for each school. Contributions of additional
ers to develop instructional materials.
equipment in Uganda were often made by old girls
As a result of their experience, 93% of World
and old boys clubs, as well. Other local funds, typi-
Links teachers in the five countries studied in
cally student fees that ranged from $ 1 to $ 50 a
1998–1999 expressed satisfaction with the way the
year, covered operating costs, such the cost of
program was implemented. In the 1999–2000 sur-
paper, ink, and dial up access. In most cases, stu-
vey, 90% the teachers in the 12 countries
dents were able to use the resources even if they
expressed satisfaction.
could not afford to pay the fee. In some schools,
Administrators also received training as part of
parents
and
students
supplemented computer
their school’s participation in the World Links pro-
resources
through
fund-raising
events.
For
gram; only two of the 18 administrators who
example, in a poor, all-girls school in Peru, stu-
responded to the question in the 1998–1999 survey
dents raised funds by performing musical and the-
said they did not. Training in the use of Internet
atrical shows for parents and neighbors.
software and in the design of collaborative student
While equipment donation was, perhaps, the
projects were mentioned most often as the topics of
most visible component of the program, it was
the training. Correspondingly, all 21 administrators
through the professional development of teachers
who responded to the question said they were sup-
that the World Links program made its most sig-
portive of the program and 91% said they were
nificant contribution. In the 1998–1999 survey,
satisfied with its implementation to date.
World Links technology coordinators reported that
The school technology coordinator played a cru-
336 of the teachers in their schools, or an estimated
cial role in teacher professional development.
28% of their entire teaching staff, had received
Technology coordinators were frequently teachers
training as part of their school’s participation in the
who had some prior computer experience. Often
program. During the first year, most of the training
they taught mathematics or science courses. The
topics focused on the use of technology; 92% of
coordinators served a large number and a wide var-
the teachers said they received training on the use
iety of functions with 70% or more saying that they
of the Internet and 75% received training on the
trained teachers in the use of hardware, appli-
development of Web pages. Teachers also said
cations software, the use of the Internet, and in the
they received training on the use of computer hard-
design of collaborative projects and ways to inte-
ware (70% of the participating teachers) and appli-
grate technology into the curriculum. The coordi-
cations software (64%). The workshops also
nators also worked extensively with students. More
focused on constructivist approaches to learning
than 80% of the coordinators said they trained stu-
that emphasized project-based learning and student
dents in the use of hardware, software, and the
collaboration. Nearly, 90% of the teachers said
Internet. Many coordinators also serviced the hard-
they received training on how to design and lead
ware, with 57% saying that they maintained the
collaborative student projects; 70% received train-
computer equipment and 52% saying they adminis-
ing on how to use student groups in their teaching;
tered the school’s network. These teachers often
and 59% received training on how to collaborate
provided these special services while continuing
with other teachers on the development of instruc-
their teaching duties.
tional materials.
In summary, while many schools received com-
These percentages were smaller for the larger
puters from the World Links program and lever-
group of teachers in 12 countries that responded to
aged these donations to obtain more resources, the

---

ARTICLE IN PRESS
R. Kozma et al. / International Journal of Educational Development XX (2004) XXX–XXX
9
primary service provided by the program was
orative, project-based learning, World Links stu-
teacher training. Training covered both the use of
dents frequently conducted projects with other stu-
hardware, software, and the Internet, as well as the
dents, often with students from other schools and
use of computers within new, constructivist-ori-
other countries. The titles of some of the student
ented pedagogical strategies.
projects in Uganda were Women in Mathematics,
The Democracy Schools Project, The Utopian
4.2. Implementation
Vision, Refugees, The Wetlands, Faces of War, and
Culture and Technology. In Senegal, topics
The second focus of the evaluation was on the
included Technology and the Generation Gap,
extent to which these services were used in the
Oppression and Human Violence, Women and
classroom. We found that World Links training and
Development, and Girls’ Education in Senegal.
the availability of networked computers were asso-
These projects were shared with students in Eur-
ciated with significant differences in what World
ope, Canada, the United States, and South Korea
Links teachers and their students did in their
for their reaction. Box 1 describes a few of the
classrooms, compared to computer-using teachers
World Links ecology-related student projects in a
and students in non-World Links schools. In the
bit more detail. In these projects, students worked
1998–1999 survey, World Links students in five
together while using computers, software appli-
countries were much more likely than computer-
cations, and the Internet to collect information,
using students in non-World Links to report that
analyze it, write up reports, and post their results
they engaged in a number of pedagogical practices
on the Internet or communicate with students else-
in their classrooms (see Table 1). Some of these
where.
differences related to the use of various techno-
In summary, participation in the World Links
logies. For example, while 52% of the World Links
program lead to the classroom use of a variety of
students said they used the Internet and 40% said
computer applications and resulted in significant
they used a search engine, only 16% of the com-
differences in the use of collaborative, project-
puter-using non-World Links students said they
based pedagogical practices, relative to their use
used the Internet and 19% said they used a search
by non-participating schools.
engine. Similarly, while 34% of the World Links
students said they used email, only 21% of the
4.3. Outcomes
computer-using non-World Links students said
they did so. But some of the most dramatic differ-
The goal of the World Links program was to
ences relate to other learning activities. For
influence the learning outcomes of secondary
example, 38% of the World Links students
school students and teachers in developing coun-
reported that they gathered data for a research pro-
tries and, ultimately, to have an impact on other
ject, 31% collected information about another
variables such as school attendance, examination
country or culture, and 23% collaborated on a pro-
scores, graduation rates, and the employment of
ject with students from another country. On the
students. While outcome and impact variables are
other hand, 22% of computer-using non-World
the most difficult to measure and most susceptible
Links students reported that they gathered data for
to a range of influences apart from the program,
a research project, 18% collected information
we attempted to get an initial indication of the pro-
about another country, and only 4% collaborated
gram’s effect by asking students, teachers, and
on a project with students from another country.
administrators questions about them. In addition,
These differences held up when World Links tea-
we compared reported outcomes in participating
chers were asked parallel questions about the class-
schools to those in non-participating schools. As
room practices of their students in the 12 countries
well, we directly measured student outcomes in a
participating in the 1999–2000 study (see also
sample of World Links and non-World Links
Table 1).
schools in one country. We used the triangulation
As a result of the program’s emphasis on collab-
of these responses as in indication of their veracity.

---

ARTICLE IN PRESS
10
R. Kozma et al. / International Journal of Educational Development XX (2004) XXX–XXX
Table 1
Classroom practices as reported by students and teachers
How often did you use computers to do each of the following:
Percentage of each group who responded “1–3 times a
month” or “once a week or more”a
World Links
Non-World Links
World Links teachers
students 1999
computer-using
2000b
students 1999
Use applications software
52c
42
71
Use CD-ROMs
34c
21
50
Use e-mail
47c
25
62
Use a search engine to find information on the Web
40c
19
56
Use bulletin boards or listservs
26c
15
35
Use the Internet
52c
16
59
Use drill and practice materials
51
46
56
Use spread sheets to analyze data
34c
31
28
Produce a Web page
23c
9
16
Collaborate on a project with other students in the same class
47c
30
53
Collaborate on a project with students from another school in
25c
13
27
your own country
Collaborate on a project with students from another country
23c
4
23
Exchange information with another students from another country 28c
8
29
Gather and analyze resource materials on a problem or topic
41c
25
44
Gather data for a research project
38c
22
40
Gather evidence to argue a position about an issue
34c
25
39
Write project reports
35c
24
35
Use graphics in a report
34
28
35
Collect information about another country or culture
31c
18
26
Draw conclusions or make predictions using data you gathered or 35c
28
34
obtained from resource materials
Communicate with your parents or other members of the
41c
31
28
community about what you do in school
a The response options provided to respondents were: not at all, 1–5 times during the year, more than five times during the year
but less than once a month, average of 1–3 times a month, and average of once a week or more.
b Teachers were asked about the extent of student practices and were provided with the same response option as students.
c World Links and non-World Links students are statistically different (pair-wise t-test, p
0.05) for full range of responses.
Nonetheless, these findings must be considered
come of the program was on students’ opinion
preliminary until direct measures of learning can
about their ability to get better jobs upon gradu-
be systematically and widely collected.
ation; 77% of the World Links students rated this
outcome as very high. In addition, 71% rated the
4.3.1. Student outcomes
program very high on its effect on their communi-
In the 1998–1999 survey, a majority of students
cation skills and 61% on its effect on their ability
in the World Links program said that it had a sig-
to reason with information. A large majority said
nificant effect on a variety of knowledge, skills,
that the program improved their attitudes toward
and attitudes (see Table 2). Perhaps, it is not sur-
school (66%) and their school attendance (64%).
prising that 66% of students rated the program very
And 60% rated the program very highly in its
highly for its impact on their attitudes toward tech-
effect on their knowledge about other cultures as
nology and over 60% said their technology skills
a result of their participation. In the 1998–1999
improved. However, the most highly rated out-
survey, an overwhelming 92% of the students in

---

Document Outline

• Closing the digital divide: evaluation of the World Links program
  • Introduction
  • The World Links program
  • Study design
    • Surveys of students, teachers, and administrators
    • Assessment of student learning
  • Results
    • Services provided
    • Implementation
    • Outcomes
      • Student outcomes
      • Student outcomes compared
      • Student outcomes assessed
      • Teacher outcomes
      • Summary
    • Barriers and differences between countries
  • Summary and conclusions
• References

---