Bulletin number 32
Delivering and Assessing
Learning on Mobile Devices:
Report on a pilot project by SQA, May and
June 2007












June 2008

Published by the Scottish Qualifications Authority
The Optima Building, 58 Robertson Street, Glasgow G2 8DQ
Ironmills Road, Dalkeith, Midlothian EH22 1LE


www.sqa.org.uk





The information in this publication may be reproduced in support of SQA
qualifications. If it is reproduced, SQA should be clearly acknowledged as the
source. If it is to be used for any other purpose, then written permission must be
obtained from the Publishing Team at SQA. It must not be reproduced for trade or
commercial purposes.

© Scottish Qualifications Authority 2008

---

SQA is committed to the use of robust evidence in the development and
evaluation of policy and its implementation, and carries out or commissions
research across a range of topics to support this.

The publication of Research Bulletins allows us to disseminate the results of our
research activity to practitioners, policy makers, parents, academics and anyone
else who has an interest in the key role that qualifications play in economic
growth and social inclusion in Scotland.

---

Disclaimer
This report is the result of a small-scale research project. The information it
contains reflects the opinions and experiences of the researcher and should not be
viewed as a comparison of products by the Scottish Qualifications Authority.
Products mentioned are examples of devices that could be used for e-learning and
e-assessment — there are many others. The inclusion of a product should not be
viewed as a recommendation by the Scottish Qualifications Authority, its staff or
appointees. Some of these products, including their costs, may also have changed
since the report was compiled or published.

---

Contents

Executive summary 1
1 Introduction
3
2 Hardware platforms 4
3 Formatting teaching and learning materials 6
4 Using the materials 10
5 Using mobile devices for assessment 12
6 Candidate and teacher experience 14
7 Vendor technologies 15
8 Conclusions and recommendations 17

---

Executive summary
This report presents the findings of a small-scale pilot project undertaken by the
Scottish Qualifications Authority between May and June 2007. The remit for the
project was to take a preliminary look the prospects for using mobile devices,
such as mobile phones and smartphones, for the delivery and assessment of
learning.

Three devices were selected as broadly representing the types of device in current
use for mobile internet access. There are other products which provide the same
functions and similar features. The three devices we selected were:

♦ The T-Mobile Vario II MDA: a Windows Mobile-based smartphone.
♦ The Nokia N800 Internet Tablet: for standalone WiFi internet access.
♦ The Nokia 6151 Mobile Phone: a typical budget 3G mobile telephone.

The teaching and learning materials selected were for the SQA Internet Safety
Unit1. Although these materials render well on full-sized screens, there are
several factors that cause problems when trying to display them on mobile
devices. These include the requirement for a lot of scrolling, redundant
navigational features, the amount of white space, the size of graphics, and the use
of add-on technologies, such as Java and Flash.

Changes were made to the materials themselves, including removing or reducing
graphics, using bolding, avoiding add-ons and keeping pages short.

The materials were developed using Course Genie2. Changes were made to the
Course Genie settings to simplify navigation, reduce the amount of white space
displayed, and to remove the logo from the top of each page. The cumulative
effect of these changes was to produce clear and readable pages which can be
navigated with a minimal amount of downward scrolling.

All the devices tested displayed the materials in a usable format and permitted
effective navigation, although the devices with larger screens provided the best
user experience. Download speeds were excellent using the latest 3G technology
and the slightly older GPRS technology. Excellent results were also obtained
using WiFi access.

Although there appears to be great potential for using mobile devices for
assessment, the mobile assessment aspect of this project was not particularly
successful. The online assessments for the SQA Internet Safety qualification are
hosted on the SOLAR website, but most of the devices used could not get past the
first page, probably because subsequent pages appear to be Flash-based. Only the
Nokia N800 Internet Tablet displayed the SOLAR pages correctly.

1 An Intermediate 1 National Unit designed to enable candidates to make safe and legal use of the
Internet — http://www.sqa.org.uk/files/nu/F0H510.pdf

2 Course Genie has since been renamed Wimba Create.

1

---

Course Genie questions, used in the online materials, worked well on devices
running the latest Windows Mobile 6, but not on earlier versions. They also
worked on mobile phones running the Opera Mini browser.

It may be useful to investigate alternative methods for online assessment, such as
assessment via text messaging (SMS). Mobile devices may also be suitable for
collecting audio and video evidence for a portfolio-based assessment and
uploading it to a website, where an assessor or verifier could examine it online.

The project included a brief trial of the Mobile version of the SQA Internet Safety
materials with a small group of secondary pupils, using relatively modern mobile
telephones with GPRS access to the internet via proprietary browsers. The pupils
enjoyed the experience and the trial highlighted a number of operational
difficulties which will inform future developments.

During the project, we encountered two potentially useful vendor-specific
technologies: M-Learning and Zoho Challenge. These could usefully be
investigated further in future projects.

The preliminary study has highlighted three major points:

♦ It is possible, and even relatively easy, to adapt existing learning materials so
that they can be displayed on a variety of mobile devices.
♦ Assessment on mobile devices is not nearly as straightforward, largely due to
the fact that the mechanisms used make use of technologies, such as Flash,
which are poorly implemented on mobile devices at present.
♦ In general, things worked better on the newer and more sophisticated devices
than on the older and simpler ones.

Future developments would probably be best restricted to specific platforms,
particularly devices running the Windows Mobile 6 operating system and mobile
phones which support the Opera Mini browser.

The outlook for mobile delivery is positive and there is already a large amount of
material available which could be converted. The picture regarding mobile
assessment is more complex and requires further investigation, but it may improve
as technological developments take place.


2

---

1 Introduction
The initial remit for the project was:

♦ Re-format teaching and learning materials for rendering on mobile devices.
♦ Explore suitable formats for rendering materials on mobile devices.
♦ Explore potential of mobile devices for assessment (using the existing SQA
Internet Safety item bank).
♦ Research candidates’ and teachers’ views on mobile learning and mobile
assessment.
♦ Research candidates’ and teachers’ experience of using the SQA Internet
Safety materials and assessments on mobile devices.
♦ Explore vendor technologies to support mobile learning/mobile assessment.

The project was very brief and was very much a preliminary look at the prospects
and potential pitfalls in this area. Decisions had to be made early in the project
about the teaching materials and assessments to be used and the hardware
platforms to be covered.

The teaching materials selected were those developed recently for the Internet
Safety Unit, which had been piloted in the latter half of 2006. These were selected
because both teaching and assessment materials were already available in online
format.


3

---

2 Hardware platforms
There is a vast range of different types of hardware available, so we decided to
restrict coverage to three types of devices. The devices were selected because they
broadly represent the types of device in current use for mobile internet access.

♦ The T-Mobile Vario II MDA is an HTC Hermes in T-Mobile livery. Similar
phones are available, with slightly different styling, from several other
manufacturers.3 It represents the current state-of-the-art in Windows Mobile-
based smartphones. At the outset of this study the current release was
Windows Mobile 5, but during the study Windows Mobile 6 was released,
bringing significant improvements. The operating system and associated
software is very similar to that used on many standalone PDAs (ie, those
which are not also telephones), so information obtained about this platform
should also apply to PDAs.
♦ The Nokia N800 Internet Tablet is a standalone device for internet access. It
runs the Maemo operating system, a variant of Linux, and uses the Opera
browser. Unlike the other devices tested, it is not a mobile phone, but accesses
the internet via Bluetooth or WiFi. It is mainly of interest due to its large
screen size (800 x 400 pixels).
♦ The Nokia 6151 Mobile Phone is a typical budget mobile telephone which
supports 3G internet access. It is not Windows-based, but has its own
proprietary browser. It also runs the Opera Mini browser from Opera
Software, which works on a wide range of current phones. The 6151 was
recently recommended as a ‘Best Buy’ by Which? magazine.


3 When this report was written the HTC Hermes variants were among only a few Smartphones
offering 3G access. At the time of publication there are many others.

4

---

More about the selected hardware platforms
T-Mobile Vario II MDA
The Vario II is a compact
smartphone with a slide-out
QWERTY keyboard. It is 3G and
HSDPA enabled, allowing internet
access at broadband speeds up to
1.8Mb. It incorporates a 400MHz
Samsung processor, 64MB dynamic
memory, Microsoft Word and Excel
editing, plus a PowerPoint and PDF
viewer. Connectivity includes

Bluetooth, infrared and USB2. It has
a 2.1 megapixel camera with flash
and video capture and a secondary
video-conferencing camera.

Nokia N800 Internet Tablet
The Nokia N800
features a high-
resolution widescreen
display and supports
the Opera 8 browser
and Flash 9. It allows
internet calling with
integrated web camera
and instant messaging
and has an e-mail
client and a full-screen
finger keyboard. It has
128MB RAM and
256MB Flash Storage
and supports Bluetooth
and WiFi 802.11b/g.

Nokia 6151 3G Mobile Phone
The Nokia 6151 is a budget 3G phone which allows
for high data-transfer speed. The phone measures 108
x 47 x 19mm and weighs 98g.

The Nokia 6151 is equipped with a 1.3 megapixel
camera, 30MB internal memory, a microSD memory
card slot, and a stereo FM radio.

Connectivity features include tri-band GSM networks

support plus 3G, EDGE, and Bluetooth support.


5

---

3 Formatting teaching and learning materials
The teaching and learning materials selected for the project were those developed
for the Internet Safety Unit, available online at:

www.e-learning-computing.com/netsafe.

This screenshot shows a typical page, introducing the topic of Trojans:



Although these materials render well on full-sized screens, there are several
factors that cause problems when trying to display them on mobile devices. Many
of these cause screens to be cluttered, and can reduce usability.

♦ Considerable scrolling, both horizontal and vertical, is required. This makes it
difficult to get an idea of the overall content of the screen.
♦ Redundant navigational features (eg back/forward arrows as well as explicit
hyperlinks and menu choices) take up a lot of room. These can be useful when
plenty of screen area is available, but not when space is at a premium.
♦ There is a lot of white space. Although white space can improve readability, it
can be wasteful when there is limited screen area available.
♦ Graphics are too large, meaning that they take a long time to download and
take up a lot of space on the screen.
♦ Many web pages rely on add-on technologies, such as Flash (for animated
graphics) or Java (for user interaction). These are not always fully
implemented on mobile devices.

The screenshots given below illustrate these problems. They show different parts
of the same page, initially displayed unmodified on a Vario II:


6

---

Document Outline

• ÿ
• • ÿ
    • ÿ
• ÿ
• ÿ
• ÿ
• • ÿ
  • ÿ
  • ÿ

---