International Journal of Artificial Intelligent and Expert Systems (IJAE) Volume 1, Issue 1, 2010

---

International Journal of
Artificial Intelligent and Expert
Systems (IJAE)







Volume 1, Issue 1, 2010









Edited By
Computer Science Journals
www.cscjournals.org

---

Editor in Chief Dr. Bekir Karlik

International Journal of Artificial Intelligent
and Expert Systems (IJAE)
Book: 2010 Volume 1, Issue 1
Publishing Date: 31-05-2010
Proceedings
ISSN (Online): 2180-124X

This work is subjected to copyright. All rights are reserved whether the whole or
part of the material is concerned, specifically the rights of translation, reprinting,
re-use of illusions, recitation, broadcasting, reproduction on microfilms or in any
other way, and storage in data banks. Duplication of this publication of parts
thereof is permitted only under the provision of the copyright law 1965, in its
current version, and permission of use must always be obtained from CSC
Publishers. Violations are liable to prosecution under the copyright law.

IJAE Journal is a part of CSC Publishers
http://www.cscjournals.org

©IJAE Journal
Published in Malaysia

Typesetting: Camera-ready by author, data conversation by CSC Publishing
Services – CSC Journals, Malaysia


CSC Publishers

---

Editorial Preface

The International Journal of Artificial Intelligence and Expert Systems
(IJAE) is an effective medium for interchange of high quality theoretical and
applied research in Artificial Intelligence and Expert Systems domain from
theoretical research to application development. This is the first issue of
volume first of IJAE. The Journal is published bi-monthly, with papers being
peer reviewed to high international standards. IJAE emphasizes on efficient
and effective Artificial Intelligence, and provides a central for a deeper
understanding in the discipline by encouraging the quantitative comparison
and performance evaluation of the emerging components of EXPERT
SYSTEMS. IJAE comprehensively cover the system, processing and
application aspects of Artificial Intelligence. Some of the important topics are
AI for Service Engineering and Automated Reasoning, Evolutionary and
Swarm Algorithms and Expert System Development Stages, Fuzzy Sets and
logic and Knowledge-Based Systems, Problem solving Methods Self-Healing
and Autonomous Systems etc.

IJAE give an opportunity to scientists, researchers, and vendors from
different disciplines of Artificial Intelligence to share the ideas, identify
problems, investigate relevant issues, share common interests, explore new
approaches, and initiate possible collaborative research and system
development. This journal is helpful for the researchers and R&D engineers,
scientists all those persons who are involve in Artificial Intelligence and
Expert Systems in any shape.

Highly professional scholars give their efforts, valuable time, expertise and
motivation to IJAE as Editorial board members. All submissions are evaluated
by the International Editorial Board. The International Editorial Board ensures
that significant developments in image processing from around the world are
reflected in the IJAE publications.


IJAE editors understand that how much it is important for authors and
researchers to have their work published with a minimum delay after
submission of their papers. They also strongly believe that the direct
communication between the editors and authors are important for the
welfare, quality and wellbeing of the Journal and its readers. Therefore, all
activities from paper submission to paper publication are controlled through
electronic systems that include electronic submission, editorial panel and
review system that ensures rapid decision with least delays in the publication
processes.

To build its international reputation, we are disseminating the publication
information through Google Books, Google Scholar, Directory of Open Access
Journals (DOAJ), Open J Gate, ScientificCommons, Docstoc and many more.
Our International Editors are working on establishing ISI listing and a good

---

impact factor for IJAE. We would like to remind you that the success of our
journal depends directly on the number of quality articles submitted for
review. Accordingly, we would like to request your participation by
submitting quality manuscripts for review and encouraging your colleagues to
submit quality manuscripts for review. One of the great benefits we can
provide to our prospective authors is the mentoring nature of our review
process. IJIP provides authors with high quality, helpful reviews that are
shaped to assist authors in improving their manuscripts.


Editorial Board Members
International Journal of Artificial Intelligence and Expert Systems (IJAE)

---

Editorial Board

Editor-in-Chief (EiC)

Dr. Haralambos Mouratidis
University of East London, (United Kingdom)

Editorial Board Members (EBMs)

Professor. Yevgeniy Bodyanskiy
Kharkiv National University of Radio Electronics (Ukraine)

Assistant Professor. Bilal Alatas
Firat University (Turkey)

---

Table of Contents




Volume 1, Issue 1, May 2010.



Pages
1 - 6
A Hybrid Oriya Named Entity Recognition system: Harnessing the

Power of Rule

Sitanath Biswas, S. P. Mishra, S Acharya, S Mohanty

























International Journal of Artificial Intelligence and Expert Systems (IJAE) Volume (1) Issue (1)

---

Sitanath Biswas, S. P. Mishra, S Acharya & S Mohanty
A Hybrid Oriya Named Entity Recognition system:
Harnessing the Power of Rule


Sitanath Biswas



sitanath_biswas2006@yahoo.com
ITER, SOA University, Bhubaneswar

S. P. Mishra



smitaprava@yahoo.com
ITER, SOA University, Bhubaneswar

S Acharya



Sweta_acharya20@yahoo.co.in
AIET, Bhubaneswar

S Mohanty



sangham1@rediffmail.com
Utkal University, Bhubaneswar





Abstract
This paper describes a hybrid system that applies maximum entropy (MaxEnt)
model with Hidden Markov model (HMM) and some linguistic rules to recognize
name entities in Oriya language. The main advantage of our system is, we are
using both HMM and MaxEnt model successively with some manually developed
linguistic rules. First we are using MaxEnt to identify name entities in Oria corpus,
then tagging them temporary as reference. The tagged corpus of MaxEnt now
regarded as a training process for HMM. Now we use HMM for final tagging. Our
approach can achieve higher precision and recall, when providing enough
training
data
and
appropriate
error
correction
mechanism.


1 INTRODUCTION

Name Entity Recognition (NER) is an important activity in the Natural Language Processing
pertaining to Information Extraction (IE), Machine Translation (MT), Information Retrieval (IR) etc.
NER is the task of identifying and classifying all proper nouns in a document as Person name,
location name, organization name, number, time etc.
This paper presents an Hybrid NER system for Oriya Language and the goal of the system is to
recognize different types of NEs- person, designation, title-Person, organization, abbreviation,
location, time, number, and measure.

To develop a MaxEnt and HMM based Oriya NER system, we have identified suitable features
like Orthography features, suffix and prefix information, morphology information, part-of-speech
information as well as information about the surrounding words and their tags in Oriya language.
We have used gazetteers for identification of designation, title, of the person names etc. We have
also used person and location name gazetteers in our system for better identification of NEs. We
have discovered that linguistic rule also plays a crucial role in identifying NEs so we have used a
no. of linguistic rules of Oriya language in our system like the rule to recognize time, number etc.
According to the specifications defined by MUC, the NER tasks generally work on seven types of
named entities as listed below with their respective markup:


International Journal of Artificial Intelligence and Expert Systems (IJAE), Volume (1): Issue (1)
1

---

Sitanath Biswas, S. P. Mishra, S Acharya & S Mohanty

PERSON (ENAMEX)

ORGANISATION (ENAMEX)

LOCATION (ENAMEX)

DATE (TIMEX)

TIME (TIMEX)

MONEY (NUMEX)

PERCENT (NUMEX)
The paper is organized as follows. A brief survey of different techniques used for the NER task in
different languages and domains are presented in Section 2. A discussion on the training data is
given in Section 3. The MaxEnt and HMM based NER system is described in Section 4 and 5.
Various features used in NER are then discussed. Next we present the experimental results in
Section 8. Finally Section 9 concludes the paper.

2 PREVIOUS WORKS

There are several classification methods which are successful to be applied on this task. Chieu
and Ng [1] and Bender et al.[2] used Maximum Entropy approach as the classifier. Conditional
Random Filed (CRF) was explored by McCallum and Li [3] to NER. Mayfield et al.[4] applied
Support Vector Machine (SVM) to classify each name entity. Florian et al. [5] even combined
Maximum Entropy and Hidden Markov Model (HMM) under different conditions. Some other
researchers are focused more on extracting some efficient and effective features for NER. Chieu
and Ng [1] successfully used local features, which are near the word, and global features, which
are in the whole document together. Klein et al. [6] and Whitelaw et al.[7] reports that character-
based features are useful for recognizing some special structure for the name entity. Linguistic
approach uses hand-crafted rules, which needs skilled linguists. Some recent approaches try to
learn context patterns through ML which reduce amount of manual labour. Talukder et al.(2006)
combined grammatical and statistical techniques to create high precision patterns specific for NE
extraction.

In rule-based approaches, a set of rules or patterns is defined to identify the named entities in a
text. These rules or patterns consist of distinctive word format, such as particular preposition prior
to a named entity. For instance, a string of words behind titles such as ‘sri’, ‘srimati’, etc will be
identified as name of a person, whereas a word after a preposition such as , ‘deikeri’, ‘pakhare’,
etc is most likely to be a location. By implementing a finite set of carefully predefined pattern
matching rules, the named entities within a text could be found systematically.


3 TRAINING DATA

The annotated data used in our system is in the IOB formatted text in which a B - XXX tag
indicates the first word of an entity type XXX and I -XXX is used for subsequent words of an
entity. The tag O indicates the word is outside of a NE. The training data for Oriya contains more
than 56K.

4 MAXIMUM ENTROPY MODEL

For the development of our Oriya NER system, we have used MaxEnt model which is the Java
based open-nlp MaxEnt toolkit and freely available at www.maxent.sourceforge.net. It gives the
probability values of a word belonging to each class. That is, given a sequence of words, the
probability of each class is obtained for each word. To find the most probable tag corresponding
International Journal of Artificial Intelligence and Expert Systems (IJAE), Volume (1): Issue (1)
2

---

Sitanath Biswas, S. P. Mishra, S Acharya & S Mohanty
to each word of a sequence, we can choose the tag having the highest class conditional
probability value.

A Maximum Entropy approach models a random process by making the distribution satisfy a
given set of constraints, and making as few other assumptions as possible. The constraints are
specified as real-valued feature functions over the data points. The expected value of each
feature function under the ME distribution must equal the empirical expected value of function as
found in the training dataset. In all other respects, the target distribution should be as uniform as
possible, which means it must have the highest entropy.

Let X be the set of conditions, usually very big, and Y the set of possible outcomes. We assume
that there is a true joint distribution P(x,y), but we are interested only in modeling the conditional
P(y|x). For this purpose we can use a training set {(xk,yk)}k=1..N generated by the true distribution,
and a set of features fi :X×Y→R. Typically, the features are binary and test for specific conditions.
It can be shown that the unique most uniform distribution that satisfies all feature constraints has
the form:

1


(*) p(yx) =
exp
 f x, y
i
i 

Z x


 i


where λi –s are the parameters chosen to maximize the likelihood of the training data, and Z(x) is
a normalization constant, which ensures that for every x the sum of probabilities of all possible
outcomes is 1. The most common procedure for parameter estimation is the Generalized Iterative
Scaling algorithm.

4.1 MAXIMUM ENTROPY MARKOV MODELS

A MaxEnt consists of |Y| conditional ME models py’(y|x) = p(y|x,y'), one for each y'. The model
py’(y|x) estimates the probability of appearance of the label y immediately after the label y' in the
context x. The probability of a whole label sequence y = y1 y2… ym, given the sentence x = x1 x2…
xm, is the product

m1
P(yx) = P y x .
p
y
x

0  1
1 
y  i1
i1 
i
i 1
The best tagging can be found using Dynamic Programming similar to Vitterbi algorithm. The
model p0(y|x) used at the beginning of a sentence is separate.

4.2 FEATURES

Features play an important role when building any MaxEnt model based system. The different
features are Orthographic features (like capitalization, decimal, digits), affixes, left and right
context (like previous and next words), NE specific trigger words, gazetteer features, POS and
morphological features etc. In English and some other languages, capitalization features play an
important role but In Indian languages there is no capitalization of letters for distinguishing proper
nouns from other category of words and no such database is available from which one can
search the proper names like other nouns. The Indian languages are also morphologically rich in
nature. The word reordering inside a sentence is also a common feature of these languages. In
the following we have discussed about the features we have identified and used to develop the
Indian language NER systems.




International Journal of Artificial Intelligence and Expert Systems (IJAE), Volume (1): Issue (1)
3

---