Intelligent Library Systems:
Artificial Intelligence Technology and
Library Automation Systems
By Charles W. Bailey, Jr.
Copyright 1991 (C) by Charles W. Bailey, Jr.
Edited version published as: "Intelligent Library Systems: Artificial Intelligence Technology
and Library Automation Systems." In Advances in Library Automation and Networking, vol. 4,
ed. Joe A. Hewitt, 1-23. Greenwich, CT: JAI Press, 1991. http://www.elsevier.com
Artificial Intelligence (AI) encompasses the following general areas of research: (1) automatic
programming, (2) computer vision, (3) expert systems, (4) intelligent computer-assisted
instruction, (5) natural language processing, (6) planning and decision support, (7) robotics, and
(8) speech recognition.1 Intelligent library systems utilize artificial intelligence technologies
to provide knowledge-based services to library patrons and staff.
Artificial Intelligence is a broad, complex area of study, which can be difficult for non-
specialists to understand. Yet, its ultimate promise is to create computer systems that rival
human intelligence, and this clearly has major implications for librarianship. If we are make
progress in the area of intelligent systems, we must have an well-developed understanding of
AI technologies, a historical perspective on accomplishments to date, and a realistic perspective
of AI as a tool with appropriate and inappropriate uses in light of current constraints. Many
authors have previously provided in-depth overviews of AI technologies. The interested reader
should consult either a basic2 or a more challenging3 introductory work for a detailed treatment
of AI. There have also been several good reviews of research and development efforts relevant
This paper examines certain key aspects of AI that determine its potential utility as a tool for
building library systems. It discusses the barriers that inhibit the development of intelligent
library systems, and it suggests possible strategies for making progress in this important area.
While all of the areas of AI research indicated previously may have some eventual application
in the development of library systems, this paper primarily focuses on a few that the author
judges to be of most immediate significance--expert systems, intelligent computer-assisted
instruction, and natural language applications. This paper does not discuss the use of AI
knowledge-bases in libraries as subject-oriented library materials.
2.0 The Nature of Intelligence
To understand "intelligent" systems, we must first attempt to understand the nature of
intelligence. Theories of human intelligence abound, but there is no consensus about what
constitutes intelligence.10 This lack of a widely accepted definition of intelligence is an obstacle
for AI researchers.
Based on a review of major models of human intelligence, Cook et al. conclude that the
following ten factors are most pertinent to expert system research:
Acquisition: the ability to acquire new knowledge.
Automatization: the ability to refine procedures for dealing with a novel situation
into an efficient functional form.
Comprehension: the ability to know, understand, cognize and deal with novel
Memory management: the ability to represent knowledge in memory, to map
knowledge on to that memory representation, and to access the knowledge in
Metacontrol: the ability to control various processes in intelligent behaviour.
Numeric ability: the ability to perform arithmetic operations.
Reasoning: the ability to use problem-solving knowledge.
Social competence: the ability to interact with and understand other people,
machines or programs.
Verbal perception: the ability to recognize natural language.
Visual perception: the ability to recognize visual images.11
This is certainly a very useful list for its intended purpose; however, if we encountered a system
exhibited these traits and no others, would we consider that system to be intelligent by human
standards? Probably not. The reason is that our notion of human intelligence is quite likely
determined by the entire gestalt of human existence--the fact that we are transient organic
beings that possess five senses and feel as well as think. In short, computers lack:
All that man is,
All mere complexities,
The fury and the mire of human veins.12
To illustrate this point, let's briefly examine one elusive feature of human intelligence: intuition.
Well-known artificial intelligence critics Hubert and Stuart Dreyfus, propose a five stage model
of human skill acquisition.13 At the novice level, a learner obediently follows rules provided
by an instructor, regardless of the specific situation. Based on practical experience, a learner
at the advanced beginner level starts to grasp important elements in the situation that no one can
teach. When the competent level is attained, the learner weighs the importance of different
factors in the situation, devises goal-oriented plans, and puts those plans to work. At the
proficiency level, the learner starts to make rapid, correct judgments about the solutions to
particular problems without rational deliberation. At the final level, the expert relies heavily
on intuition for normal problem solving activity in his or her area of expertise. Dreyfus and
What should stand out is the progression from the analytic behavior of a detached
subject, consciously decomposing his environment into recognizable elements, and
following abstract rules, to involved skilled behavior based on an accumulation of
concrete experiences and the unconscious recognition of new situations as similar to
whole remembered ones.14
Dreyfus and Dreyfus believe that expert systems are not likely to achieve the proficiency and
expert levels of skill acquisition, and, consequently, these systems should be called "competent"
Given the complexity of human intelligence, how soon can we expect truly intelligent
computers? Moravec has the startling answer: "I believe that robots with human intelligence
will be common within fifty years."16 However, Pfaffenberger believes "the artificial
intelligence technology required to create an intelligent system probably cannot be achieved
using today's computers, or any possible future extension of them."17 The question is obviously
a controversial one, and, at this stage, the answer is a matter of opinion. Nonetheless, it is
prudent to monitor the goals and progress of computer scientists who are attempting to develop
true computer intelligence. Some of the visions of these researchers echo science fiction, and
if their goals were realized they would have a major impact on human life as we know it.18-19
In the long-term, it may be possible to fully emulate human consciousness; however, it is
currently unclear how long it will take to develop the AI tools required for this task, if these
tools can be developed at all. It is also uncertain whether intelligent systems must be housed
in robot bodies with advanced sensory capabilities in order to achieve human-level intelligence.
If so, robotics technology will become a critical factor that either facilitates or inhibits the effort
to develop truly intelligent systems.
Until major technological advances are made, we can be expect that "intelligent systems" will
mimic certain key aspects of human intelligence, not replicate it. These are likely to be well-
understood cognitive capabilities, although limited simulation of human emotion may occur as
well to facilitate human-machine interaction. Despite the limited intelligence of these systems,
they will be able to perform useful work within restricted task domains.
3.0 Barriers to Intelligent Systems
Although there are a few exceptions, intelligent systems are generally not in operational use
today in libraries. After at least ten years of research and development, why is that we have so
few production systems? Several critical problems will be discussed here.
3.1 General Limitations
Liebowitz identifies inadequacies in the following areas of expert system technology, leading
to what he terms "artificial stupidity" in these systems: (1) common sense reasoning, (2) "deep"
reasoning about the underlying principles of an area of knowledge, (3) explanation features, (4)
ability to learn, (5) support for distributed expert systems, and (6) knowledge acquisition and
Yen and Tang confirm the difficulty of performing common sense reasoning in expert systems.
They point to additional problems, including: (1) difficulties in allowing end-users to tailor
expert systems to meet their needs, (2) high system development and maintenance costs, (3)
inherent complexity of expert system development, (4) limited natural language capabilities,
and (5) inability of expert systems to recognize the limits of their knowledge, deal with
problems at those limits, and reject problems that exceed those limits.21
3.2 Common Sense Reasoning
Common sense is simply "general knowledge that every human being supposedly has about the
world," and, consequently, common sense reasoning is the use of this knowledge to make
inferences about everyday objects and events.22 If we can build specialized medical expert
systems to diagnose diseases, why is common sense reasoning about what humans view as
simple problems so difficult? Sheil indicates:
Our ordinary interactions assume a great deal of shared knowledge about an enormous
variety of topics. But when we judge a task's difficulty, we tend to forget that fact and
focus only on the amount of information that must be added to our base of common
Intelligent systems lack that common base of human knowledge, severely constraining the types
of functions that they can perform. Major breakthroughs in other significant problem areas,
such as natural language understanding, are likely to be dependent on progress being made in
Consequently, it is significant that Lenat and his colleagues at the Microelectronics and
Computer Technology Corporation are engaged in a long-term project, called Cyc, to develop
a large-scale knowledge base, which would initially have enough encoded knowledge to permit
a computer to understand a one-volume encyclopedia and a newspaper.24 Work began in 1984,
and, by 1994, Cyc will be given its "final exam." This will include tests of its ability to
facilitate development of expert systems, English-language communication skills, knowledge
acquisition abilities, and learning capabilities. It is anticipated that Cyc will be foundation upon
which much more advanced intelligent systems will be constructed by AI researchers.
3.3 Natural Language Processing
Natural language processing systems could be utilized for a variety of purposes, including
"natural language interfaces to databases and expert systems, text understanding, text
generation, and machine translation."25
Research in natural language processing focuses on:
lexical/morphological analysis, which deals with words and the smallest
meaningful units in language;
syntax, focusing on the relationship between words in larger structural units, such
semantics, which deals with meaning, and
pragmatics, which deals with the relationship between linguistic expressions and
Since they require deeper levels of knowledge, semantic and pragmatic analysis are
considerably more difficult than morphological and syntactic analysis. Unfortunately, semantic
and pragmatic capabilities are likely to be needed to provide human-equivalent communication
capabilities. Reflecting the complexity of the task of processing natural language, Smith
indicates that: "Natural language systems cannot yet, and perhaps never will be able to handle
truly unrestricted natural language."27
Discussing the technological obstacles to natural language processing, Obermeier states:
Currently available NLP products and systems are too expensive and not user-
friendly for two reasons: (1) basic research problems in understanding language and
languages remain unsolved, and--somewhat as a consequence--(2) brute force algorithms
prevail that have implicit limits that have been reached. . . . The underlying cause of the
poor quality of NLP technology is the lack of proven theories, the unfounded support
of 30-year-old formalisms that have never produced any visible results (e.g., ATN), and
the ill-defined area of NLP in the first place.28
Natural language interfaces are utilized in database management systems; however, these
systems frequently contain a limited number of highly-structured data elements.29 The number
of potential ways one would one want to retrieve these data elements is reasonably finite.
On the other hand, information retrieval systems primarily contain textual information on a wide
diversity of topics, and only "quasi-natural language" interfaces, which perform restricted
linguistic processing on search requests, have been successfully used with large-scale
databases.30 In general, Warner characterizes the efforts of information retrieval researchers in
this area as follows:
Traditionally, the focus has been on morphology and syntax, although semantics has
recently been gaining favor. Pragmatics . . . has barely begun to be explored.31
Clever low-level natural language processing techniques can permit the use of free-text queries
in large information retrieval systems; however, until semantic and pragmatic processing are
feasible, difficult problems remain in adequately matching the true subject content of queries
with that of document surrogates and documents themselves.
Since higher-level natural language processing is more tractable in restricted domains, certain
task-oriented staff functions in library automation systems may be good candidates for natural
language applications, but care must be taken so that staff efficiency is increased--not
decreased--by this strategy (e.g., function keys may be faster than words for some tasks).
3.4 Knowledge Acquisition, Representation, and Maintenance
Ideally, there would be two primary ways of creating and updating knowledge bases in
intelligent systems: (1) intelligent systems would distill new knowledge from full-text and other
electronic information sources; and (2) human experts would add their unique insights to this
knowledge base by unrestricted natural language dialogues with intelligent systems.
Unfortunately, current methods of knowledge base creation and maintenance are typically fairly
tedious. Human experts must be interviewed in detail to try to record their knowledge.
Knowledge must be encoded into a knowledge structure, which requires that the "knowledge
engineer" have some understanding of artificial intelligence techniques to structure knowledge
appropriately. Raw knowledge must be structured within a meaningful and consistent
framework to be represented in the computer in a useful way. The correct knowledge
representation scheme to use (e.g., rules, frames, scripts, or semantic networks) for a particular
kind of knowledge is not always readily apparent. Moreover, different types of knowledge may
be encoded in different knowledge representation schemes, and there must be thought given to
how these different types of knowledge relate to one another and how they will function
together in the overall context of the intelligent system. Once knowledge is encoded, it must
be entered manually by keyboarding. The time investment to determine, represent, and enter
knowledge can be significant.
Another reason for this time investment, which may not be solved by future automated
techniques, is that experts cannot always articulate how they solve problems. So the knowledge
engineer building a reference expert system might have a cooperative, top-notch reference
librarian as his or her expert, but that individual may not be able to easily categorize different
types of reference questions and explain the general strategies used to answer different types
of questions. Of course, this example is in a domain where there are few formal rules, making
it a worst case. Presumably, a highly-structured area like cataloging would be different;
however, based on a survey of expert system applications to AACR2 cataloging, Meador and
Wittig conclude: "There have been problems in every attempt to convert AACR2 into the highly
structured rules necessary to run an expert system."32 It appears that the pioneers who build
intelligent library systems are likely to devote a considerable amount of effort to knowledge
acquisition issues. Until improved manual and automated methods of knowledge acquisition
and maintenance are devised, Brooks statement holds: "There are no short cuts as far as
knowledge base development is concerned."33
Dreyfus and Dreyfus question whether advanced knowledge can be encoded at all:
If one asks the experts for rules one will, in effect, force the expert to regress to the level
of a beginner and state the rules he still remembers but no longer uses. If one programs
them on a computer, one can use the speed and accuracy of the computer and its ability
to store and access millions of facts to outdo a human beginner using the same rules.
But no amount of rules and facts can capture the knowledge an expert has when he has
stored his experience of the actual outcomes of tens of thousands of situations.34
A related problem is that many affordable expert system tools utilize simple knowledge
representation structures like rules, but lack a repertoire of sophisticated structures. If the expert
system tool does possess such knowledge structures, there may be a significant price to be paid
in terms of system performance on affordable hardware platforms. These factors can force the
development of systems in logic programming languages (e.g., Prolog) or in procedural
languages (e.g., Pascal). Either one of these system development strategies can be time-
consuming and complex. For example, the development of the well-known PLEXUS expert
system, which is written in Pascal, took three and one-half years.35
3.5 Difficulty in Scaling Up Prototypes to Operational Systems
Intelligent systems are often created utilizing a software development methodology called
The objective of software prototyping is to validate a proposed design by constructing
a low-cost system that has enough functionality to test out major design decisions on
Prototyping allows developers to fairly quickly create one or more systems that approximate
the final system; however, there is no guarantee that the software techniques utilized in the
small-scale prototype will work in the larger-scale production system.37 This can lead to a false
sense of accomplishment. As noted before, many library expert systems are prototypes, not
In many knowledge bases, the system developer attempts to keep individual rules independent
of each other. This simplifies knowledge base maintenance and makes the knowledge base
more easily extensible. Employing the knowledge base, the intelligent system uses inferencing
techniques, such as forward- and backward-chaining, to approximate human reasoning.
However, as a knowledge base becomes larger, it becomes more difficult to debug the logic of
an essentially unstructured system:
In point of fact, however, the sequence in which the rules is expressed takes on
enormous significance, since the inference engine evaluates them in a linear, sequential
fashion. . . . Moreover, for every ten rules that are entered, there are at least four times
as many logical corollaries, each of which must be recognized as an outcome and
specifically addressed by the insertion of a clause. . . . Furthermore, a very large expert
system may break down irreparably as further expansion is attempted because its overall
structure and the pattern of corollaries have grown beyond the capacity of the
programming team to conceptualize them all.38
3.6 Level of Effort, Technical Expertise, and Expense
The level and calibre of effort that must be expended to create an intelligent system is directly
related to the power and complexity of that system. The more "intelligent" the system is, the
greater the effort that must be expended to create it and the greater the degree of expertise that
is needed to do so. The need for skilled personnel combined with expensive development tools
(e.g., advanced expert system shells) or techniques (e.g., original programming in logic or
procedural languages) makes the creation of sophisticated intelligent systems a potentially
Librarians and library automation vendors are already engaged in an accelerating effort to
provide library patrons with access to a diversity of new computer systems.39-40 Assuming that
the needed expertise was present to create intelligent systems, what priority will libraries and
vendors give to developing these systems? The reality is that staff resources, especially
computer specialists, are a precious and finite commodity. It will take more staff with greater
skill levels to create a complex intelligent system than a simple one, and this will inevitably
affect decisions about what types of intelligent systems to build.
Unfortunately, there appears to be a limited pool of artificial intelligence expertise in the library
and library automation vendor communities. Given the scope and complexity of the library
automation systems that have been developed to date, there is a highly skilled body of computer
professionals in these organizations; however, artificial intelligence is a specialized and
somewhat esoteric area of computing that requires skills that are unlike those obtained by
building conventional systems. Consequently, the likelihood is that retraining and new hiring
will need to be done before any significant, widespread work is done in the area of intelligent
Information and computer scientists have been active developers of intelligent information
retrieval systems.41-48 This work has made a significant contribution to the literature, but it has
produced many more prototypes than operational systems. Research will lay the theoretical
foundations for the development of operational systems, but it is unlikely to produce them. That
is not its purpose or intent.
Librarians have also done work in the area of library expert systems.49-56 Some of this work
appears rudimentary when compared to the work of computer and information scientists.
Nonetheless, librarians have developed some exemplary systems (e.g., the PLEXUS57 and
What are the barriers that prevent librarians from developing sophisticated expert systems?
The type of tools librarians are likely to use (e.g., low-cost expert system shells) impose definite
limits on what can be accomplished. Using these shells, it is fairly easy to create small systems
with limited knowledge bases; however, some important problems require larger knowledge
bases, more complex knowledge representation schemes, and greater analytic power than
inexpensive expert system shells currently provide. For example, there is a considerable
difference between creating an expert system that recommends 50 reference works in a single
discipline and a system that recommends 1,000 reference works in all disciplines. In the first
case, an inexpensive expert system shell may work well, but, in the second case, it may be
The fact that many librarians have little or no training in artificial intelligence techniques is
another problem. This lack of formal or informal training limits our conceptual horizons, and
it reduces the repertoire of technological tools that we can skillfully deploy to create intelligent
systems. Hopefully, library schools will provide more in-depth training to new generations of
Since library staff are rarely devoted full-time to building expert systems and hardware and
software budgets are frequently tight, resource constraints also impose limits on the types of
systems that librarians can create.
Finally, risk aversion is a problem. When library administrators invest scarce resources in
innovative projects, they usually expect success, preferably rapid success. Unfortunately, the
closer to the cutting edge a project is, the greater the chance that it will fail to produce a fully
functional system. Playing it safe often leads to systems designed for "success," not
sophisticated functionality. At this stage in the evolution of library expert systems, more
calculated risk taking is needed in system development efforts.
Given these problems, where will future intelligent library systems come from? In the late
1960's and early 1970's, a few libraries developed single-function or integrated online systems.
Some of these systems became quite important latter to the library community as a whole
because they were successfully marketed by library automation vendors as turnkey systems.
Vendors also created their own turnkey library systems. Today, few libraries develop their own
integrated library system; most buy a turnkey system from a vendor. This is a major reason why
integrated systems are so prevalent today--each library does not have to build its own system.
As long as we are in an era of hand-crafted intelligent systems, libraries will make limited use
of these systems. We need turnkey intelligent systems, which can be modified for local use.
As in the past, the source of these systems may be mixed, with both vendors and a few
exceptional libraries producing systems that vendors can successfully market. However, there
must be significant market demand for these systems, appropriate artificial intelligence tools
to build them with, and skilled staff to develop them.
Vendors are beginning to show some interest in intelligent systems. As a spin-off of the
PLEXUS project, Tome Associates has developed TOME SEARCHER, an intelligent front-end
to commercial computer science, electrical engineering, and information technology databases.61
Other vendors have initiated research projects or developed operational systems that incorporate
some aspects of artificial intelligence technology.62-65
4.0 Strategies for Future Progress
By recognizing the limitations of contemporary artificial intelligence techniques, we can
establish realistic goals for intelligent library systems and devise appropriate system
development strategies. This section discusses some promising approaches to the application
of artificial intelligence techniques in library automation systems.
4.1 Targeted Development Efforts
Artificial intelligence is a means to an end. Like any tool, it has strengths and limitations. Our
true goal is not to create systems based on artificial intelligence technologies--it is to create the
most powerful, flexible, and easy-to-use systems possible for our ourselves and our patrons.
AI is one tool in the toolbox, which should be employed when the characteristics of the task at
hand indicate that an AI solution that is called for.
Some of our goals may not be well suited for AI techniques or they may require a judicious,
limited application of AI technology. For example, Brooks has expressed pessimism about the
appropriateness of AI as a tool to build information retrieval systems:
For several reasons, IR does not seem to be an ideal problem domain for an
expert system application. It is a domain that is neither well bounded nor narrow nor
homogeneous. In some retrieval environments and for some aspects of the retrieval
process, there may be no obvious human experts, and what experts there are often do not
agree. . . . Further, although little research has been conducted in the kinds of knowledge
required by the knowledge base of an intelligent IR system, all the indications are that
the knowledge needed would be extensive and wide-ranging and would include
knowledge of the subject domain of the queries and documents being processed.66
One response to the stated problems is to abandon efforts to create intelligent retrieval systems;
however, another approach is to try to overcome the inherent difficulties by restricting the goals
and domain of the system. For example, the CANSEARCH system builds on the knowledge
inherent in MeSH subject headings to provide assistance to researchers searching MEDLINE
for cancer information.67 CANSEARCH is not a global solution to the problem of providing
intelligent information retrieval, but it effectively addresses one specialized need.
We need to carefully analyze complex problem areas looking for aspects of these areas that are
amenable to the application of AI techniques. For example, providing the user with intelligent
assistance in selecting an appropriate database from a wide variety of remote and locally-
mounted databases may be an easier task than helping the user to devise optimal search
strategies for each of those databases. By use of a mix of AI and conventional programming
techniques, we may be able to build powerful systems that solve many, but not all, of the
problems associated with a domain like information retrieval.
Moreover, we need to actively identify domains that are inherently well-bounded, but are
complex enough to truly require AI techniques. It may be that certain aspects of acquisitions,
cataloging, circulation, interlibrary loan, preservation, and serials are fertile ground for the
selective application of AI techniques. However, aside from cataloging, little effort has been
made to create intelligent systems in these areas. Attempts to create expert cataloging systems
have generally run aground because of the ambiguities inherent in interpreting AACR2;
however, further revisions of the code could specifically address these ambiguities with the aim
of facilitating the creation of intelligent cataloging systems.68
4.2 Machine Intelligence vs. Machine-Aided Intelligence
One important determinant of the complexity and feasibility of intelligent systems is the locus
of control in the system. Smith contrasts machine intelligence with machine-aided intelligence:
Where machine intelligence dominates, an effort is made to keep as much control as
possible within the computer by automating decision-making and execution of tasks.
Where machine-aided intelligence dominates, the user is in control with the computer
providing suggestions and gathering information to aid the user's decision-making.69
Given current constraints, machine intelligence systems will be very difficult--if not impossible-
-to create for large, complicated domains where levels of performance approaching human
intelligence are required. It is likely that the ambitions of machine intelligence systems must
be much more modest, restricting the usefulness of AI techniques to a smaller set of applications
than would otherwise be the case. However, by focusing on how AI can be used to augment--
not replace--humans, a much wider range of applications can be fruitfully considered.
It is possible to conceive of a variety of AI-based tools that would assist users in performing
various tasks. For example, the prototype DANEX system guides researchers in performing
certain types of statistical data analysis.70 A variety of prototype "intelligent agent" systems
have been created to perform restricted, repetitive tasks for users, such as compiling monthly