Advances in Multimedia - An International Journal (AMIJ)

---

ADVANCES IN MULTIMEDIA - AN
INTERNATIONAL JOURNAL (AMIJ)







VOLUME 2, ISSUE 2, 2011

EDITED BY
DR. NABEEL TAHIR








ISSN (Online): 2180-1223
Advances in Multimedia - An International Journal is published both in traditional paper form and
in Internet. This journal is published at the website http://www.cscjournals.org, maintained by
Computer Science Journals (CSC Journals), Malaysia.


AMIJ Journal is a part of CSC Publishers
Computer Science Journals
http://www.cscjournals.org

---

ADVANCES IN MULTIMEDIA - AN INTERNATIONAL JOURNAL
(AMIJ)

Book: Volume 2, Issue 2, May 2011
Publishing Date: 31-05-2011
ISSN (Online): 2180-1223

This work is subjected to copyright. All rights are reserved whether the whole or
part of the material is concerned, specifically the rights of translation, reprinting,
re-use of illusions, recitation, broadcasting, reproduction on microfilms or in any
other way, and storage in data banks. Duplication of this publication of parts
thereof is permitted only under the provision of the copyright law 1965, in its
current version, and permission of use must always be obtained from CSC
Publishers.



AMIJ Journal is a part of CSC Publishers
http://www.cscjournals.org

(c) AMIJ Journal
Published in Malaysia

Typesetting: Camera-ready by author, data conversation by CSC Publishing Services - CSC Journals,
Malaysia



CSC Publishers, 2011

---

EDITORIAL PREFACE

This is second issue of volume two of the Advances in Multimedia - An International Journal
(AMIJ). AMIJ is an International refereed journal for publication of current research in computer
science and computer security technologies. AMIJ publishes research papers dealing primarily
with the technological aspects of computer science in general and computer security in particular.
Publications of AMIJ are beneficial for researchers, academics, scholars, advanced students,
practitioners, and those seeking an update on current experience, state of the art research
theories and future prospects in relation to computer science in general but specific to computer
security studies. Some important topics cover by AMIJ are Animation, Computer Vision,
Multimedia Signal Processing, Visualization, Scanning, Multimedia Analysis, Multimedia
Retrieval, Motion Capture and Synthesis, Displaying, Dynamic Modeling and Non-Photorealistic
Rendering, etc.

The initial efforts helped to shape the editorial policy and to sharpen the focus of the journal.
Starting with volume 5, 2011, AMIJ appears in more focused issues. Besides normal publications,
AMIJ intend to organized special issues on more focused topics. Each special issue will have a
designated editor (editors) - either member of the editorial board or another recognized specialist
in the respective field.

This journal publishes new dissertations and state of the art research to target its readership that
not only includes researchers, industrialists and scientist but also advanced students and
practitioners. The aim of AMIJ is to publish research which is not only technically proficient, but
contains innovation or information for our international readers. In order to position AMIJ as one
of the top ADVANCES IN MULTIMEDIA - AN INTERNATIONAL JOURNAL, a group of highly
valuable and senior International scholars are serving its Editorial Board who ensures that each
issue must publish qualitative research articles from International research communities relevant
to Advance Multimedia fields.

AMIJ editors understand that how much it is important for authors and researchers to have their
work published with a minimum delay after submission of their papers. They also strongly believe
that the direct communication between the editors and authors are important for the welfare,
quality and wellbeing of the Journal and its readers. Therefore, all activities from paper
submission to paper publication are controlled through electronic systems that include electronic
submission, editorial panel and review system that ensures rapid decision with least delays in the
publication processes.

To build its international reputation, we are disseminating the publication information through
Google Books, Google Scholar, Directory of Open Access Journals (DOAJ), Open J Gate,
ScientificCommons, Docstoc and many more. Our International Editors are working on
establishing ISI listing and a good impact factor for AMIJ. We would like to remind you that the
success of our journal depends directly on the number of quality articles submitted for review.
Accordingly, we would like to request your participation by submitting quality manuscripts for
review and encouraging your colleagues to submit quality manuscripts for review. One of the
great benefits we can provide to our prospective authors is the mentoring nature of our review
process. AMIJ provides authors with high quality, helpful reviews that are shaped to assist
authors in improving their manuscripts.


Editorial Board Members
Advances in Multimedia - An International Journal (AMIJ)

---

TABLE OF CONTENTS




Volume 2, Issue 2, May 2011



Pages
30 - 38
Affordable Multi-touch Teaching Station for Engineering Classes
Cheong Soon Nyean, Yap Wen Jiun, R. Logeswaran N. Rajasvaran















Advances in Multimedia - An International Journal (AMIJ), Volume (2), Issue (2) : 2011

---

Soon-Nyean Cheong, Yap Wen Jiun & Rajasvaran Logeswaran
Affordable Multi-touch Teaching Station for Engineering Classes


Soon-Nyean Cheong


sncheong@mmu.edu.my
Faculty of Engineering
Multimedia University
Cyberjaya, 63100, Malaysia

Yap Wen Jiun




wjyap@mmu.edu.my
Faculty of Engineering
Multimedia University
Cyberjaya, 63100, Malaysia

Rajasvaran Logeswaran



loges@mmu.edu.my
Faculty of Engineering
Multimedia University
Cyberjaya, 63100, Malaysia

Abstract

This paper presents the development of a cost-effective multi-touch teaching station that supports
existing educational applications and a custom-made Multi-touch Teaching Module used by
instructors to teach undergraduate engineering laboratory classes at Multimedia University. The
technology-enhanced teaching station was developed based on an optical tracking technique
known as Frustrated Total Internal Refraction (FTIR), to achieve multi-touch capabilities on the
table surface. The Adobe Flash-based Multi-touch Teaching Module supports popular multi-touch
gestures including panning, rotating, and zooming in and out, on multimedia educational content
such as streaming video lectures, animations and schematic diagrams. The Multi-touch Teaching
Module enhances the way lecturers manipulate teaching materials such as enlarging a complex
electronic circuitry with just fingers' gestures when operating the teaching station. When operating
the multi-touch teaching station with existing educational applications, lecturers are able to use it
as a normal touch screen to directly interact with the application instead of using a mouse or
keyboard, making the interface more intuitive. Some lecturers have evaluated the teaching station
and provided positive feedback over the standard computer, because it is much easier to operate.
In short, the paper summarizes the experience in developing a universal multi-touch teaching
station and Multi-touch Teaching Module, as well as the system evaluation by instructors in an
undergraduate engineering laboratory class, which is valuable for those who intend to enhance
teaching platforms with such technology in a university.

Keywords: Multi-touch, Gesture Control, Teaching Station, Natural User Interface, Effective
Teaching.


1. INTRODUCTION
New technologies such as interactive whiteboard, digital visualizer, tablet PC and Smart Podium
have been adopted by universities in developed countries as a way to support modern
educational activities by providing educators with an intuitive user interface to teach more
effectively. These equipments allow educators to handwrite impromptu notes easily with a stylus,
keeping the metaphor of a paper notebook. Commercial off-the-shelf interactive whiteboards and
visualizers allow instructors to add digital notes on the fly. The tablet PC and Smart Podium
provide similar functionality with the additional feature that when operated with collaborative
learning software such as Classroom Presenter [1, 2, 3], provides an innovative learning
environment that supports lecturer-students interaction and promotes effective teaching that gives
students a more valuable experience. Despite the positive results in students' learning experience
Advances in Multimedia - An International Journal (AMIJ), Volume (2) : Issue (2) : 2011
30

---

Soon-Nyean Cheong, Yap Wen Jiun & Rajasvaran Logeswaran
found in the studies of the integrations of educational collaborative software with recent electronic
educational equipments, acquiring such technologies in developing countries is limited due to the
huge amount of up-front investment required. Furthermore, existing affordable solutions such as
the tablet PC or Smart Podium pose a challenge for educators to manipulate teaching materials
intelligently with fingers' gesture on a large interface (22 to 32 inches). The design and
implementation of an affordable multi-touch teaching station that offers a large (32 inches)
intuitive user interface and Multi-touch Teaching Module that allow instructor to interact with
teaching materials conveniently using fingers' gesture in a budget-constrained environment will
be discussed.

2. REVIEW OF EXISTING TECHNOLOGIES
A multi-touch tablet PC is a mobile computer that consists of a digitizer screen that allows stylus
or finger input, enabling a very natural way of writing and drawing. A number of universities
adopted the tablet PC as part of their teaching tool for instructors to easily write notes or sketch
schematic diagrams during lecture session to improve teaching or learning process [4, 5].
Examples of two-touch-point tablet PCs with a 12-inch screen include the Fujitsu LifeBook T4410,
Lenovo ThinkPad x201 and Dell Latitude XT2, with a selling price in the range from USD 1,500 to
USD 2,500, depends on the customizations (prices from Google product search as of 1 March
2011).

Smart Podium from Smart Technologies is a mobile display that allows the instructor to control
computer applications or to add digital ink over applications such as Microsoft PowerPoint using a
stylus or finger [6]. Once the Smart Podium is connected to a computer and projector, the
instructor can comfortably interact with teaching materials at the front of the classroom while
students can simultaneously view the same content on the large projected screen. The Texas
A&M University adopted Smart Podium as a tool to ease instruction in engineering courses that
required frequent impromptu additions of annotations or diagrams, calculations and writing of
equations during lectures, as reported by Smart Technologies in 2006. The Smart Podium has a
screen size ranging from 15 to 22 inches with a selling price from USD 1,500 to 3,000 (details at
http://www.wedgwood-group.com/interactive_panels.htm).

The tabletop is a computer on which input layer is overlaid on top of the large horizontal display.
Lately this technology has been implemented in the education industry in developed countries [7,
8]. Existing commercially tabletops usually support multi-touch capability in recognizing at least
two contact points at the same time. Existing multi-touch tabletops can be divided into two main
types, one uses the electronic field sensing technology while the other uses camera-based
technology [9, 10]. Multi-touch tabletops allow multiple interactions by the user via finger touches
on the table surface, making it easier to access digital content, giving rise to the term natural user
interface. Examples of off-the-shelf tabletops include Microsoft Surface (screen size of 32 inches)
and Mitsubishi DiamondTouch (screen size of 32 and 42 inches), with the cost around USD
10,000 to 20,000, including the on-site installation [11]. Smart Table from Smart Technologies is a
tabletop with a screen size of 27 inches that was designed specifically for educational purposes
and comes with a price tag of USD 6,650 (see http://www.smarttables.co.uk/products/). Rapid
adaptation of the multi-touch tabletop in universities has not been realized because of the high
price tags.

3. SYSTEM DESIGN AND IMPLEMENTATION
The primary objective of designing the multi-touch teaching station in this work is to
technologically enhance instructors' teaching experience by providing a more natural user
interface for easier and effective teaching at an affordable price. This includes providing a large
horizontal digital writing area (32 inches) for instructors to hand draw or write digital notes
intuitively with their fingers or an Infrared (IR) pen. Furthermore, instructors would be able to
control teaching slides more effectively through fingers' gesture to zoom in and out, pan and
rotate when conducting a lecture.

Advances in Multimedia - An International Journal (AMIJ), Volume (2) : Issue (2) : 2011
31

---

Soon-Nyean Cheong, Yap Wen Jiun & Rajasvaran Logeswaran
The design of the camera-based multi-touch teaching station developed in this work is shown in
Figure 1. It consists of an IR camera, an acrylic surface with IR LEDs installed along the edges, a
standard projector, two mirrors, a diffuse compliant surface and a computer CPU. The design
principle is based on the FTIR technique [10] where when finger touches the table surface, IR
sources will be reflected and tracked by the IR camera. The projector is used to provide a large
visual feedback on the table surface. Mirrors are used to create a long virtual path between the
projector and the projection surface such that the teaching station can be built at a reduced height
with a large display. IR LEDs are installed along the edges of the acrylic surface to provide an IR
illuminated surface. An IR band pass camera is mounted at the bottom of the teaching station to
act as an optical sensor to differentiate between finger touches (reflected IR source) and
projected teaching materials (visible light) on the table surface. The diffuse compliant surface on
top of the acrylic is made out of tracing paper coated with silicon material that serves as
projection surface as well as a proxy to easily frustrate the total internal reflection whenever it is
touched.



FIGURE 1: The design of the Camera-based Multi-touch Teaching Station.

Figure 2 shows the multi-touch teaching station data path, beginning from touches on the
teaching station surface to generating visual feedback on the teaching station display via an
assortment of software components. Whenever fingers touch the table surface, total internal
reflection will occur causing the IR light source to be refracted. The IR camera captures the IR
sources and streams the video feed to the IR tracking module known as Community Core Vision
(CCV) (available at http://ccv.nuigroup.com). CCV is a cross platform open source computer
vision tracking application that takes in a video stream source as input and generates coordinate
and blob size as output for each detected IR source in the video stream. Most of the processes
involved in generating those blob tracking data such as video capturing, background removal,
high-pass sharpening, high-pass noise filtering, brightness amplification and contour tracking are
handled by OpenCV (available at http://opencv.willowgarage.com), an open source vision library
from Intel that provides various C-language-based image processing algorithms. The accuracy of
the IR sources (finger touches) corresponding to the computer coordinates are assured by the
dynamic mesh calibration that is normally performed once during setup. Data from CCV is output
and broadcasted as TUIO messages utilizing the OSC libraries included in CCV to any TUIO-
enabled application directly or to XML-enabled applications via Flash Open Sound Control
Advances in Multimedia - An International Journal (AMIJ), Volume (2) : Issue (2) : 2011
32

---

Soon-Nyean Cheong, Yap Wen Jiun & Rajasvaran Logeswaran
(FLOSC) [12]. FLOSC is a Java application that performs as a proxy to convert UDP port 3333
TUIO messages to TCP port 3000 XML messages.



FIGURE 2: Multi-touch Teaching Station data path.

The CCV of the multi-touch teaching station when operating with the Multi-touch Teaching
Module (MTTM), will pass TUIO UDP data via FLOSC to become XML messages that are
transferred to TCP port 3000 that the MTTM polls. XML messages received by the MTTM include
touch IDs, 2D coordinates and different events including touch up, touch down, touch out, touch
move etc. that are used for gesture controls. The MTTM in this work was developed with Adobe
Flash using ActionScript 3.0 to take advantage of its superior quality in manipulating and showing
multimedia content in teaching materials [13]. The instructor can upload teaching materials such
as PowerPoint slides, schematic diagrams, videos etc. to the MTTM. When using the MTTM,
instructor can manipulates teaching materials easily with fingers' gesture such as panning,
rotating, zooming in and out etc., especially when dealing with complex schematics diagrams.
The Slide Canvas class is called within the MTTM to act as the entry point to display a slide or
diagram in JPEG format. The RotatableScalable class is used to support fundamental gestures
on teaching slides, as shown in Figure 3. Teaching materials can be dragged freely with a single
touch. Rotating and size changing of teaching materials are possible only if there are two touch
points. The rotation involves a computation based on the tangent of the angle made by the two
points with a reference line while the size changing involves a more complex computation to first
determine the center point, and then compare the two distances between the two coordinates
before and after the manipulation. The code listing illustrated in Figure 4, showed how the MTTM
Advances in Multimedia - An International Journal (AMIJ), Volume (2) : Issue (2) : 2011
33

---

Soon-Nyean Cheong, Yap Wen Jiun & Rajasvaran Logeswaran
extends the RotatableScalable class and actively listens to port 3000 for data sent from the
FLOSC.



FIGURE 3: Fundamental gestures in manipulating teaching slides.



FIGURE 4: The code listing of using RotatableScalable Class.

When the multi-touch teaching station is running existing engineering applications such as
Matlab, PSpice, AutoCAD etc., the CCV channels the TUIO data to the Multi-Touch Vista
Framework, a user input managing layer obtainable in CodePlex open source hosting site at
http://www.codeplex.com/MultiTouchVista. This layer bridges the connection between the
teaching station surface touch inputs and the engineering applications by processing and
decoding the TUIO UDP messages from CCV into some 255-touch-point stylus input events
understand by the Windows 7 operating system. This enables instructors to use their finger as the
input device to interact with existing applications running on the multi-touch teaching station.

Table 1 shows that the total hardware cost to deploy the 32-inch multi-touch teaching station was
roughly USD 1034 using a conversion rate of USD 1.00 = MYR 3.05 (as at 1 March 2011). The
cost will be higher if a projector with better specification is used.

4. RESULTS
Figure 5(a) shows the implementation of multi-touch teaching station prototype in Multimedia
University. The 32-inch-diagonal-display teaching station supports both single-touch on existing
educational applications and multi-touch on the MTTM with gesture controls. Once the teaching
station is connected to a computer and projector, the instructor could easily interacts with the
teaching materials using fingers (without mouse and keyboard) in front of the classroom while
students pay attention to the same content on a large projected screen. The MTTM provides
instructors the flexibility to manage the course of the lessons by choosing and zooming in a
particular slide in the teaching materials in a non-linear manner. The multi-touch teaching station
Advances in Multimedia - An International Journal (AMIJ), Volume (2) : Issue (2) : 2011
34

---