Knowledge Based Systems

BCT 2403 Knowledge Based Systems
Defining Intelligence
Web-site Source: http://www.cortland.edu/psych/mi/theory.html
Andy Carvin(Traditional view)

Many people define intelligence, as a persons ability to solve problems, utilize logic, and think
critically. It also represents how people comprehend, examine and respond to outside stimuli. In
this case intelligence can be viewed as a "singular, collective ability to act and react in an ever-
changing world." (Carvin) in Carvin, Andy. (1999). MI-The Theory. EdWeb: Exploring
Technology and School Reform.
General intelligence(Source web-site: http://www.bsu.edu/classes/huffman2/edpsy350/chapter8/)

Abilities that allow us to be flexible and adaptive thinkers, not necessarily tied to
acquired knowledge.
Reasoning, integration and abstraction as may be seen in solving maze or puzzle based
problems.
Alfred Binet (General intelligence)

Intelligence entails the operation of specific mental functions, including memory, attention, verbal fluency and
creativity, that are strictly controlled by practical judgement.
Howard Gardner
"The standard view of intelligence is that intelligence is something you are born with; you only
have a certain amount of it; you cannot do much about how much of that intelligence you have;
and tests exist that tell you how smart you are." (Howard Gardner in Checkley) in Checkley,
Kathy. (1997). The first sevenand the eighth: a conversation with Howard Gardner.
Educational Leadership, 55, p8-13.
Multiple Intelligence Theory(- several types of intelligence-Howard Gardner)
"Intelligence refers to the human ability to solve or make something that is valued in one or more
cultures. As long as we can find a culture that values an ability to solve a problem or create a
product in a particular way, then I would strongly consider whether that ability should be called
an intelligence." (Howard Gardner in Checkley)
Ability can be called an intelligence only if it meets several criteria:
Representation in the brain for the ability (Biological Basis)
Presence of populations that are especially good or especially impaired in an
intelligence (i.e. exhibited in extreme forms in idiot savants, prodigies and geniuses)
1
Possess a clear developmental history

Used in the performance of culturally valued roles around the world (Cultural Basis)
Gardner has identified eight intelligences as : Linguistic-Verbal; Logical-Mathematical; Musical

Bodily-Kinesthetic; Spatial; Intrapersonal; Interpersonal; Naturalist
Together, these eight intelligences provide a theoretical foundation for recognizing the different
talents and abilities that people, and especially students, possess. They are generally used
concurrently, complementing one another as skills develop.
In Gardners words:
"What makes life interesting, however, is that we dont have the same strength in each
intelligence area, and we dont have the same amalgam of intelligences. Just as we look
different from one another and have different kinds of personalities, we also have
different kinds of minds."
This theory contradicts the traditional view of intelligence being predetermined and unalterable.
MI states that we can get better at each of our intelligences, although some people will improve
in some areas more readily than others.
Linguistic-Verbal Intelligence
Defined with characteristics:
Use words and language in various forms
Ability to manipulate language to express oneself rhetorically or poetically
Sensitive to meaning, order, function, sound, rhythm of words
Adept at reading and writing
May utilize native as well as other languages
Typically excels at school
Dominant mode of teaching
Potential careers: Writer, speaker, orator, lawyer, poet, politician, teacher, professor, editor, journalist,
salesperson, actor, administrator, contractor
Tools for Teachers: Reading, biographies, poetry, storytelling, book sharing, humor/jokes, wordplay, riddles,
creative writing, debate, journal writing, explanations, feelings, reports, lists, library research.
Tips for Parents: Read with your children; Listen to their questions, concerns, and experiences; Provide books
and paper for reading and writing activities; Encourage children to share with you what they have read or written;
Provide opportunities to visit libraries and bookstores; Play games like Scrabble, Hangman, Boggle, and Yahtzee.
Logical-Mathematical Intelligence
Ability to detect patterns
Approach problems logically
Reason deductively
Adept at math and science
Manipulate numbers, quantities, and operations
Typically precise and methodical
Prefer ordered and sequential activities
2
Currently highly valued by society

Potential careers: Scientist, engineer, accountant, computer analyst, computer programmer, logician,
mathematician, actuary, bank teller.
Tools for Teachers: Analyzing, calculations, classifying, pattern games, time lines, numbers, logic problems,
experimentation, critical thinking, sequencing, outlining, compare and contrast, graphic organizers, scientific
thinking, Venn diagrams, writing problems, reasoning problem solving, computer instruction
Tips for Parents: Let your children experiment; Invite them to help make a cake or mix paint colors to make
new ones; Let them help with family budget, budget their own allowances, or operate a calculator; Provide
opportunities for setting table, sorting clothes, organizing drawers; Play games like Uno, checkers, and chess.
Musical Intelligence
Ability to discern meaning in rhythmically arranged sets of pitches
Think "in music"
Recognize nonverbal sounds: pitch, rhythm, and tonal patterns
Adept at remembering melodies and converting sounds into rhythms
May be the least developed intelligence
One of the earliest talents to emerge
Potential careers:Composer, singer, performer, conductor, music critic, violinist.
Tools for Teachers: Chants, rhythmic patterns, listening, rap, singing, music performance, humming,
instrumental sounds, music education, musical chairs, listening centers, descriptive and rhythmic words (to enhance
communication).
Tips for Parents: Allow children to select their own music cassettes/CDs; Encourage singing along and clapping
to rhythm; Involve children in music lessons; Attend concerts and musicals.
Bodily-Kinesthetic Intelligence
Use ones mental abilities to coordinate ones own bodily movements
Ability to understand things by using ones body or parts of the body
Use body to solve a problem, make something, put on a production
Respond well to nonverbal communication
Frequently cant sit still, need to touch things
Adept at all sorts of athletic and fine motor activities
Typically chooses to be a participant rather than an observer
Challenges the popular belief that mental and physical activity are unrelated
Most controversial of the proposed intelligences
Potential careers:Dancer, athlete, mime, actor, clown, comedian, craftsperson.
Tools for Teachers: Acting, drama, dancing, exercise, physical gestures, experiment, inventing, movement, role
playing, human graph, demonstrations, hands-on thinking.
Tips for Parents: Involve children in dancing, acting, or sports; Provide a variety of anipulatives (clay, fabric,
blocks) for experimentation; Walk, jog, hike, bowl, play tennis, or bike as a family; Enjoy sliding, swinging, and
riding toys; Provide chores like sweeping, setting table, emptying trash; Play games like charades, Simon says, and
hide-and-seek.
3
Spatial Intelligence
Visualize mental models, manipulate them spatially, and draw them in detail
Typically thinks in images and pictures
Ability to orient themselves in a house or outdoors
Used in both arts and sciences
Enjoys chess and jig-saw puzzles
Often daydreamers
Like multiple colors
Taught best using photos, pictures or having them draw ideas
Not limited to visual domain-can also be formed in blind children
Potential careers: Artist (like painter, sculptor) architect, navigator, sailor, graphic or industrial designer,
inventor, lighting specialist.
Tools for Teachers: Guided imagery, collages, drawing, designs, posters, mapping, charting, puzzles, mazes,
fantasy, imagination, labeling, pictures, pretending, use color in activities.
Tips for Parents: Provide opportunities for solving or inventing; Let children choose the color for their bedroom,
design furniture arrangement; Visit art museums; take photographs; Provide art mediumsbrushes, paints, crayons,
markers; Play games like Pictionary or cards.
Intrapersonal Intelligence
Ability to understand oneself, including feelings and motivations
Includes the ability to delay gratification
Can discipline themselves to accomplish a wide variety of tasks
Adept at independent work, typically enjoy solitude
"March to a different drummer"
May be daydreamers
Draw upon feelings to guide behavior
Potential careers:Psychotherapist, religious leader, poet, artist.
Tools for Teachers: Reflection, autobiography, concentration, focusing, goal setting, self-identification, thinking
strategies, higher order reasoning, metacognition, free choice time, awareness of personal feelings, independent
work and projects.
Tips for Parents: Give children quality time to work or play alone; Provide time for reflection; Encourage writing
in a diary or journal.
Interpersonal Intelligence
Ability to "read people"discriminate among other individuals especially their moods,
intentions, motivations;
Able to understand and empathize with what others are thinking and doing;
Very social and street smart;
Adept at group work, typically assume a leadership role.
4
Potential careers: Teacher, salesperson, counselor, therapist, social worker, community organizer, clinician,
politician.
Tools for Teachers: Feedback, cooperative groups, discussion, group projects, teamwork, role playing, person-
to-person communication, empathy practices, interviews, team assessments, peer tutors, skits, plays, debates.
Tips for Parents: Play a family game; Encourage discussion and problem solving; Encourage children to
participate in activities.
Naturalist Intelligence
Ability to recognize and classify living things like plants, animals
Sensitive to other features of the natural world (clouds, rock configurations)
Recognize patterns in nature
Prefer to be outdoors
Most recently "discovered" intelligence
May extend into consumer realmdiscriminate among cars, sneakers, types of
makeup
Potential careers:Botanist, chef, taxonomist, scientist.
Tools for Teachers: Create observation notebooks, draw natural scenes, photography, describe changes in
environment, binoculars, telescopes, microscopes, magnifiers, classify and categorize things in nature, design Venn
diagrams.
Tips for Parents: Take children on nature walks to observe and journal; Encourage "backyard experiments"
with bugs, plants; Star watch from the backyard or local observatory; Share, as a family, observations from
notebooks; Collect leaves, flowers, rocks and try to identify using a field guide.
Sternberg's Theory of Triarchic Intelligence(Professor Robert Sternberg of Yale

University)
(Summarized from: Sternberg, Robert (1988) The Triarchic Mind: A New Theory of
Intelligence. NY: Viking Press.)
This concept of intelligence equates to combinations of individual preferences from three levels
of mental self-management. These three areas correspond with:
1. Functions of governments of the mind;
2. Stylistic preferences; and
3. Forms of mental self-government.
Examples: As a combination a person might prefer legislative functions, internal variables and
hierarchic habits of mental self-government; while another individual might prefer executive
functions; external variables and anarchic habits of mental self-government, and so forth.
I. Functions of governments of the mind:
Legislative - creating, planning, imagining, and formulating.
Executive - implementing and doing.
Judicial - judging, evaluating, and comparing.
5
II. Scope - stylistic variables:

Internal - by themselves
External - collaboration
III. Forms of mental self-government:
Monarchic people perform best when goals are singular. They deal best with one goal or need at a time.
Hierarchic people can focus on multiple goals at once and recognize that all goals cannot be fulfilled
equally. These people can prioritize goals easily.
Oligarchic people deal with goals that are of equal weight well, but they have difficulty prioritizing goals
of different weight.
Anarchic people depart from form and precedent. Often they don't like or understand the need for rules and
regulations. These people operate without rules or structure, creating their own problem-solving techniques
with insights that often easily break existing mindsets.
Emotional intelligence(Daniel Goleman)

Goleman defines EQ as:
EQ is not destiny - emotional intelligence is a different way of being smart. It includes knowing
your feelings and using them to make good decisions; managing your feelings well; motivating
yourself with zeal and persistence; maintaining hope in the face of frustration; exhibiting
empathy and compassion; interacting smoothly; and managing your relationships effectively.
Those emotional skills matter immensely - in marriage and families, in career and the workplace,
for health and contentment.
Persons having EQ exhibit the following common attributes. In essence, the panorama of skills
found in the context of emotional intelligence help individuals manage both the self and others in
the following areas: 1. Impulse control; 2. Self-esteem ; 3. Self-motivation ; 4. Mood
management ; 5. People skills.
Summary on the definition of intelligence

Left as an exercise.
COGNITIVE FACULTIES
Include mechanisms for: Vision; Learning ; Motions co-ordination; Hearing ; Speech and
language; Problem solving(adaptation).
COGNINITIVE ABILITIES
Cognitive abilities include the following:
Spatial ability- structural visualization, paper folding, handling blocks;
Numerical ability- number series identification, numerical reasoning, computational ability;
Memory- remembering designs, patterns, numbers, observations, visual details;
Thinking- analytical reasoning, presenting ideas logically, inductive and deductive reasoning,
recognizing relationships, fluent expressions;
Vocabulary- numbers of words known;
Perceptual speed- clerical speed and accuracy.
6
Exercises
What is intelligence?
Can intelligence be mechanized?
7
DEFINING ARTIFICIAL INTELLIGENCE
There is no agreed definition of the term artificial intelligence. However, there are various
definitions that have been proposed. These are considered below.
Artificial intelligence is a study in which computer systems are made that think like human
beings, Haugeland, 1985 & Bellman, 1978.
Artificial intelligence is a study in which computer systems are made that act like people. People
in this group include Kurzweil, 1990 who particularly thought that AI is the art of creating
computers that perform functions that require intelligence when performed by people.
Rich & Knight were also in this group and they considered AI as the study of how to make
computers do things which at the moment people are better at.
Artificial intelligence is a study in which computers that rationally think are made. People in this
group are Charniac & McDermott, 1985 who think that AI is the study of mental faculties
through the use of computational models.
Winston, 1992 is also in this group who think that AI is the study of computations that make it
possible to perceive, reason and act.
Artificial intelligence is the study in which systems that rationally act are made. People in this
group include Schalkeoff, 1990 who consider AI to be a study that seeks to explain and emulate
intelligent behaviour in terms of computational processes.
Luger & Stubblefield, 1993, are also in this group and they consider AI to be a branch of
computer science that is concerned with the automation of intelligent behavior.
TURING TEST FOR INTELLIGENCE (ACT HUMANLY)
HUMAN
COMPUTER Interpersonal link
INTERROGATOR
(teletext)
The human interrogator thinks he/she is communicating with a human.
To pass Turing Test the computer must:

Process natural language;
Represent knowledge;
Reason;
Learn and adapt to the new situations.
8
Total Turing test included vision & robotics.
Cognitive Sciences view of artificial intelligence()

Text Book Definition:
"Artificial Intelligence seeks to produce, on a computer, a pattern of output that would be
considered intelligent if displayed by human beings." (Gardner, 1984)
Controversy exists over a single definition of AI. Some definitions focus on programs,
programming language, or simulations. In order to classify machines as "thinking", it is
necessary to define intelligence/knowledge.
What is knowledge?
How is knowledge represented?
How do we use knowledge?
Within the field of AI, tension has risen between scientists in two main topics:
Generalists vs. Experts
Generalists prefer overarching programs, those which can be applied throughout.
Experts prefer highly detailed programs which are knowledgable in specific
domains but restricted in applicability.
Scientific Level of the Field
Some scientists feel that AI is simply a form of applied engineering and not a
field of psychology.
Some related terms
Human thinking
This has been studied so that it may be modelled. This is cognitive modelling. Ways have been
investigated to find out how mechanize the way the human mind works. Problems are physical
and non physical and they have not been resolved.
Rational thinking
Laws of thought have been devised to manipulate though processes. Earlier people involved
were Socrates, Aristotle and Plato.
Rational actions
Rational action is one in which inputs from the environment are taken and is in the best interest
of the actor.
9
Rational agent is something that perceives and acts. AI may also be seen as a study in which
rational agents are constructed.
10
FOUNDATIONS OF ARTIFICIAL INTELLIGENCE

These are some of the bases that enabled the development of artificial intelligence. They
contributed to its growth directly or were necessary in the development of artificial intelligence.
Philosophy (428 BC to present)
Plato in 428 BC presented dialog of thought in which piety and pious actions were considered.
Algorithms were sought to distinguish pious from non pious actions.
Ren Descartes (1596 - 1650) presented the principle of dualism in which issues on distinction
between matter and mind are raised.
Wilhelm Leibnitz (1646 - 1714) argued for materialism in which the world is seen to be
controlled by physical laws.
John Locke (1632 - 1704) advanced empricism in which the source of knowledge is seen as only
possible through senses.
David Hume (1711-17776) proposed induction in which generalization can be made from the
specific.
Bertrand Russel (1872-1970) proposed logical positivism in which all knowledge can be
characterized by logical theories. The mind is seen as a connection between knowledge and
action.
Mathematics (800 - now)
Algorithms are traced from the work of an Arab mathematician of 9th century called al-
Khowarazmi.
George Boole (1815- 1864) formulated the rules of thought that provide a language that is used
for logical inference(1847).
Gottlob Frege (1848-1928) introduced the rules for first order logic except for notations.
Alfred Tarki (1902- 1983) introduced the reference theory in which objects in logic are related to
real objects.
David Hilbert (1862- 1943) raised a question looking for an algorithm that may be used for any
proposition involving natural numbers.
Psychology (1879 - now)
11
Herman Von Helmboltz (1821- 1894) and Wilhelm Wundt (1832-1920) used scientific method
to study human vision. They proposed that perception involves unconcious logical inference.
John Watson (1878- 1958), Edward Thorndike (1879 - 1949) maintained that only measurable
percepts (stimuli) should be studied and their resulting actions. They rejected mental constructs
such as knowledge, beliefs, goals and reasoning steps as scientific.
William James (1842- 1910) proposed cognitive psychology in which the brain posses and
processes information.
Computer Engineering (1940 - now)
Work in AI and computer science has been complementary. It is the advances in computer
engineering that has enabled the production of software.
Lingusitics (1957 - present)

Noam Chomsky published syntax structure.
In 1960s much of the work was done in knowledge representation.
Today we have computational linguistics which is natural language processing.
12
HISTORY OF ARTIFICIAL INTELLIGENCE

In this section we trace the evolution of artificial intelligence. Some similar phenomena are
included.
Gestation period (1943 - 1956)
Warren McCulloth & Walter Pitts (1943) proposed a model of artificial neurons. They also
suggested that neural networks could learn.
Donald Hebb (1949) demonstrated a rule for updating connection strengths between neurons so
that learning could occur.
Claud Shannon (1950), Alan Turing (1953) wrote chess programs.
Marvin Minsky & Dean Edmonds build SNARC, the first neural network computer.
1956: There was a workshop at Dartmouth College organized by John McCarthy, helped by
Minsky, Shannon and Rochester. 10 researchers attended. Allen Newel and Herbet Simon
demonstrated a program called Logic Theorist which was a reasoning system that proved Russel
& Whiteheads theorems. The name Artificial Intelligence was coined at the conference.
Enthusiasm and great expectation (1952- 1969)
Much enthusiasm was generated by earlier successes of the Logic Theorist and later General
Problem Solver.
General Problem Solver at Carnegie Mellon University imitated how humans reason.
Arthur Samuel (1952-) wrote many programs for checkers.
John McCArthy (1958) defined LISP which is a dominant AI language.
James Slagle (1963) wrote SAINT progam which solved closed form integration problems found
in first year courses.
Tom Evans (1968) wrote ANALOGY program which solved simple geometric problems.
Bertram Raphael (1968) wrote SIR which was a program that could answer questions in a
restricted area in English.
Frank Rosenblatt (1962) advance the perceptron convergence theorem which was about a
learning algorithm that can adjust connection strengths of a perceptron to match any input data.
In other words perceptrons can learn.
13
Challenges- a dose of reality (1966- 1974)
Claim by Herbet Simon (1957) that computers could think, learn and create and that it would
not take long before computers could do what men would do, turned out to be wild dreams.
Problems arose with demonstrations as the programs contained no knowledge of their own.
Typical example is the translator:
The spirit is willing but the flesh is weak <> Vodka is good but mean is rotten.
Most AI problems turned out to be intractable in that time needed grew exponentially compared
to size of instances.
Fundamental structures used to generate intelligent behaviour were limited. Minsky also showed
that there were cases in which perceptrons could not learn.
Knowledge based systems- key to power? (1969- 1979)

The earlier methods were found to be weak due to weak information domain. The solution was
to use knowledge that can be used by several reasoning steps and solves typical cases in a narrow
area of expertise.
DENDRAL, 1961 by Buchanan et al. Was a program that would take in a formula such as
C6H13NO2 and mass spectrum data from electron bombardment and generate as its output the
possible structures and mass spectrum that would be observed. It however had problems of
intractability with large molecules.
MYCIN, 1971 by Fiegenbaum, Buchanan, Shortliffe was a program that could diagnose blood
infections. It had 450 rules and performed much better than junior doctors.
PROSPECTOR, 1979 by Duda et al. Was a program that had geological data. It recommended
exploratory drilling site that proved to have substantial molybdenum deposit.
LUNAR, 1973 by William Woods was a program in geology that could be asked questions about
rock samples from the moon.
In summary, the importance of knowledge was discovered.
AI becomes an industry (1980-1988)
R1, 1982 by McDermott, was an expert system used by Digital Equipment Corporation to
configure orders for the new system. $40 million was saved annually by 1986.
By 1988, DECs AI group deployed over 40 expert systems.

14
In 1988 Du Pont had over 100 expert systems in use and 500 in development and estimated $10
million in savings.
In 1981, Japanese announced the 5th computer generation project to build intelligent computers
using PROLOG . This generated much interest in AI.
Return to Neural Networks (1986 - now)

Disillusionment with MYCIN type expert systems grew as they needed more than rules. The
work of Byson and Ho (1969) on back-propagation learning was re-invented.
Rumelheart & McClelland (1986) distributed a collection of works in parallel distributed

processing.
Recent events (1987 - now)

There has been a change in content and methods of research. The emphasis now is on building
on existing theories and not inventing new ones. It is now desired that claims be build on
rigorous theorems or evidence. There is also a need to show relevance to real world applications.
The dominant areas include Hidden Markov models, and belief networks in which reasoning is
combined with uncertain evidence.
State of the art

1997/8 Computer program beats the International Grandmaster in Chess.
PEGASUS- speech understanding program that handles air travel transactions. You say where
and when.
MARVEL (Schwuttke, 1992) is a real time expert system that handles massive stream of data
from spacecraft.
ROBOT DRIVER (Pornerleau, 1993) drove at 55 mph. It used video cameras, sonar and laser
range finders to drive the car.
Heckerman, 1991 reported an expert system that gets a better diagnosis than a medical doctor.
King et al, 1993, Kotter et al, 1994 reported a system that automatically calls an ambulance on
accident occurring.
Future of Artificial Intelligence(http://www.student.richmond.edu/~cschlenk/futureof.htm)
The future of AI will certainly be just as important as it will be interesting. There is no doubt that AI will
provide significant applications for everyday life. Today our lives almost revolve around the technologies
provided by computers. And in the future we will probably rely on them more.
EXERCISES
1. What is Artificial Intelligence?
2. Describe the Turing Test.
3. How did AI evolve?
4. Discuss the various disciplines that contributed to the development of AI.
15
5. Discuss the events in the development of AI.
16
APPLICATION DOMAINS OF ARTIFICIAL INTELLIGENCE
In this section we consider the different ways in which artificial intelligence has been applied.
Artificial intelligence has been found useful in many areas including medicine, education,
military and industry.
Since Artificial intelligence(AI) does much to do with reasoning consider some of these
reasoning activities and AI can be used in these areas:
Access. Give access to information from large databases.
Advice. Give advice to on health, emergency or to consumers.
Analyze. Detect patterns from large databases.
Assist. Help operators for example in a factory.
Auction. Apply rules of auction to many items.
Bid. Offer a bid at an auction.
Broker. Be a broker at a stock exchange; purchase from an open market.
Classify. Give hints on classification especially on the Web.
Categorize. Categorize for example the customers.
Compare. For example in legal cases find precedence.j
Control. For example as in work-flows or chemical processes.
Coordinate. As in a project team.
Data-mine. As in point of sales.
Decide. For example in placing detectors, etc.
Diagnose. As in diseases; plant faults, etc.
Evaluate. As in determining courses of action or effects of changes.
Explain. As in finding out why decisions are made.
Filter. As in isolating important mails from a mass of e-mails.
Guide. As in directing movement; avoiding danger.
Handle. As in dealing with routine information and passing only critical issues to humans.
Help. As in assisting engineers with technical issues.
Invent. As in constructing new circuits for a simplified job.
Learn. As in remembering and avoiding old mistakes.
Manage. As in supervision of information distribution or networks.
Mediate. As in going between parties.
Monitor. As in checking the movements in currency exchange rates.
Negotiate. As discussing requests for licences.
Optimize. As in making optimal designs, or using rules of thumb.
Organize. As in planning, predicting, processing, ranking, or representing.
17
Main branches of AI
Machine vision. This is area dealing with visual recognition of objects. Cameras and sensors are
used. Some of the recent practical research include ways of recognizing people using the retina.
The challenges in it involve mapping visual realities to representations that allow easy pattern
recognition. If it matures then computers can recognize their owners; vehicles can find their way
around; robots can conduct chores in the house moving around.
Speech synthesis and recognition. This is an area that attempt to find how to make computers
recognize voice inputs and respond vocally. Speech synthesizers have been used to demonstrate
speech in minimal and constrained circumstances. The main challenges include speech
recognition and generation. On maturity computers can talk, discuss, read articles aloud and take
voice commands.
Machine learning. Here the investigation is focused on making computers acquire knowledge,
skills and be adaptive. The challenges include knowledge acquisition, knowledge representation,
learning operators and how to help humans learn. On maturity computers will learn from
experience, solve problems and be adaptive.
Robotics. Here the investigation is focused on movement and positioning of arms and other parts.
The main challenges include the arm positioning and locating positions. On maturity the robots
will be used for additional domestic and industrial applications. Currently they are used for such
industrial applications such as sorting, assembling and welding.
Natural language and understanding. Here investigations consider grammars and semantics of
languages. Understanding and text or document translation are some of the important problems.
The challenges include getting uniform grammar and semantic representations; and
understanding. On maturity, the computers should be able to speak in natural language and
understand. Inter-language translations will also be possible.
Problem solving. This is an area that originally attempted to build a general problem solver. Its
focus was later revised to specialized problem solvers such as expert systems. Its main
challenges include finding out how to formulate the problems, looking or searching for solutions,
representing knowledge. On maturity, computers will solve various problems and problem
solving can be embedded into many industrial applications.
18
SOME APPLICATIONS OF ARTIFICIAL INTELLIGENCE
There are currently many applications of artificial intelligence. Some of the areas in which
artificial intelligence is used are also regarded as application problem domain areas for artificial
intelligence. These areas include military, medicine, industry and business.
Military applications
There was revived interest in artificial intelligence due to breakthroughs in robotics and
knowledge based systems.
Initially AI was perceived as a tip of the iceberg in a body of knowledge in IT that would be
important in key areas in international security such as command control, strategic defense and
verification.
Earlier experience in world war II saw the British use computers to decipher the German codes
and they could monitor bombings. In 1950s, number crunching enabled construction of thermo-
nuclear bombs; it was also possible to model simulation of fission triggered chain reaction; it
was also possible to manage information on locomotion of warheads.
In 1980s there were interests in semi-automated operation weapons, development of smart

weapons.
The main sponsor of research in AI has been US- Defense Advanced Research Projects Agency
(DARPA).
Medical applications
AI applications in medicine have been fuelled mainly by expert systems and neural networks.
These applications have been on diagnosis, drug manufacture, planning, monitoring and control
of equipment.
Industry
AI is used in all spheres of industry in areas related to planning, design, control and manufacture.
Most of the industrial uses have been fuelled by expert systems, robotics, neural networks and
genetic algorithms. We consider a few examples of reported work in industry below.
Robotics. Work have been reported on inter-frame displacement, vision and position estimation
in the house.
Natural language. Work has been reported on automatic programming, composition of

stereotype documents in foreign languages; natural language interface to databases.
19
Civil engineering. Reported work include: hazardous site evaluation; identification and ranking
of severe storms; urban storm sewer systems; offshore structure applications.
Mechanical engineering. Reported work include: marine system design; drilling stations
management; trouble shooting of large industrial plants; automated valve expert capture.
Electrical engineering. Reported work include: managing VLSI CAD data; single board
computer design; VLSI transistor sizing; network communication design.
Software engineering. Reported work include: intelligent interface design; entities that mediate
between two agents knowing little about each other; help, reason, mediate, hold dialog and assist
in direct manipulation of computers or machines.
Business. In business AI has been used in many areas including planning, forecasting and
advising.
Education. In education AI is being used to assist in tutoring and other computer assisted
learning activities.
20
PROBLEM SOLVING TECHNIQUES IN ARTIFICIAL INTELLIGENCE

Problems are tackled in AI using two main broad approaches. The first one is using the search
technique and the second one is by modeling natural phenomena (e.g. evolution, and neural
networks).
NEURAL NETWORKS
Human neuron nucleus

axon
synapse
dendrites
cell body
Artificial neuron
Wi,j
Input links aI Output links
- input function
- activation function
ai -activation value of unit i.
I1 H3 W13
W14 w35
O5
I2 H4
W24 w45
21
GENETIC ALGORITHMS
These are models used to solve problems based on the theory of evolution and natural selection.
Mutation and cross-over of genes are modelled.
Genes
A B C D
Mutation
Involves changing a value of a component of the gene (chromosome) eg.
C B
Cross-over
Involves exchanging values of components (chromosomes) across genes.
C B A D
In genetic algorithms, a new generation is reproduced using mutation and cross over of members
of the current population. The resulting new population is selected make the numbers constant
using a selection criteria called the fitness function.
A general genetic algorithm

Function GeneticAlgorithm(Population, FitnessFn):Individual;
Var parents:composite structure;
Begin
Repeat
Parents:=SELECTION(population,FitnessFn);
Population:=REPRODUCTION(parents);
Until Some individual is fit enough
End;
22
SEARCHING AS A PROBLEM SOLVING TECHNIQUE

Since searching is extensively used, we look at searching as a technique of solving problems in
more detail.
Searching is the process of looking for the solution of a problem through the a set of possibilities
(state space).
Search conditions include:

Current state -where one is;
Goal state the solution reached; check whether it has been reached;
Cost of obtaining the solution.
The solution is a path from the current state to the goal state.
Process of Searching
Searching proceeds as follows:
1. Check the current state;
2. Execute allowable actions to move to the next state;
3. Check if the new state is the solution state; if it is not then the new state becomes
the current state and the process is repeated until a solution is found or the state space is
exhausted.
Search problem
The search problem consists of finding a solution plan, which is a path from the current state to
the goal state.
Representing search problems

A search problem is represented using a directed graph. The states are represented as nodes while
the allowed steps or actions are represented as arcs.
Defining a search problem

A search problem is defined by specifying:
State space;
Start node;
Goal condition, and a test to check whether the goal condition is met;
Rules giving how to change states.
23
Example of a search case study [Bratko, p.259]

Three blocks A, B, C on a table are considered. A block can be grasped when there is no other
block on top of it. Only one block can be moved at a time.
Possible moves
Put a block on table;
Put a block on top of another block;
Remove a block from the top of another and place on top of another block.
Problem
Initial state(current state)
C
A
B
Goal state (final state)

A
B
C
State space C B
A A A A
B BC BC C
A B
B B C C
A AC ABC AB A
B C
AC AB
A A
B C
C B
The state space has 13 elements or nodes.
24
The solution to our problem is any member of the set of all paths from original to goal state such
as the path indicated in bold.
General search algorithm
Function Gsearch (Problem, QueuingFn): Solution|Failure;

Var nodes:structure;
Begin
While
Begin
Node:= remove_front_node(nodes);
If Goal_test(problem, STATE(node)) succeeds then
Solution:=Node
Else
Nodes:=QueuingFn(node, OPERATOR(problem));
End;
EndWhile
End;
EXHAUSTIVE SEARCH STRATEGIES
Searching may yield a solution or the state space may be exhausted without a solution.
Exhaustive search may be performed using breadth first or depth first (iterative deepening)
strategies. The may also be regarded as traversal techniques.
Depth first
The branch (child) nodes are visited first.
d
b c
e f g
25
The arrows show the depth first search progression.
Breadth first
This is search strategy in which the nodes of the same level are visited first.
d
b c
e f g
The arrows show the breadth first search progression
LIMITATIONS OF EXHAUSTIVE SEARCH METHODS

Exhaustive search has the advantage of guaranteeing solutions if they exist. However, exhaustive
search has some problems associated with the need to visit all states at times.
One problem is about combinatorial problems associated with some search cases. At times the
number of possibilities rise and demands on storage and processing cannot be met.
The other problem is that the state space can be extremely large such that complete search is time
consuming.
Sometimes the state space can be inexhaustible, in that it is difficult to enumerate.
GUIDED SEARCH
Guided search is one solution to the problems associated with exhaustive search. Guided search
is also called heuristic search and is a search strategy in which the knowledge about the problem
domain is used to guide the search mechanism thus enabling it to avoid wrong options.
Heuristics are rules that apply most of the time but not all the time. There are several heuristics
that are used to guide search mechanisms, but the ones considered here is called the best-first
search heuristic.
26
BEST-FIRST SERARCH HEURISTIC
In the best-first search heuristic the node with the most promising chance of being the solution is
visited first. Nodes are regarded as most promising when they are associated with the most
desired numerical values called heuristic estimates. Heuristic estimates are computed values that
assess the suitability of a node as candidate solution. The functions used to compute heuristic
estimates are called heuristic functions. At times the nodes have associated costs or benefits. A
heuristic function would therefore seek to minimize costs or maximize benefits.
Consider an example, in which nodes have associated costs and also subsequent paths have
associated costs as well. In this case we consider a heuristic that seeks to minimize the costs. The
search is going to be guided by cost minimization.
Original problems minimize{f(x):= sum path}

s s
5 a 2 2
e 7
2 f(a)=7 a e f(e)=9
4 b 5
2 f f(b)=13 b
4 c 4 f f(f)=18
2
c
3 g 2 f(c)=19
g
2 f(g)=22
3 d
t d
3 goal f(d)=25
t
Exercise: re-trace the path given that the heuristic function only adds the sum of path and the
cost of the current node.
OTHER SEARCH PROBLEMS

There are so many cases in which the search technique may be used. These include the following
cases:
8-puzzle problems.
1 2 3
4 8
27
5 6 7
8-queens problem.
Vacuum world
V ..
..
REAL LIFE APPLICATIONS OF THE SEARCH TECHNIQUE

Travelling minimize the distances in route finding.
VLSI layouts
Robot navigation robot movements
Assembly sequencing.
EXERCISES
Investigate how to program models related to agents, neural networks and genetic algorithms.
1. Discuss examples of agents.

2. Discuss how to implement the genetic algorithm.
3. How are problems solved in artificial intelligence?
4. What is searching?
5. Discuss how to handle problems using searching technique.
6. Discuss exhaustive search and heuristic search.
7. Discuss relevance of the search technique and other examples where the search technique
may be applied.
28
KNOWLEDGE REPRESENTATION
Definition
Knowledge. True rational belief(philosophy).OR facts, data and relationships (Computational
view).
Representation. Structure+ operations; OR map + operations; OR game layout and rules of play;
OR abstract data types.
Knowledge representation. Framework for storing knowledge and manipulating knowledge OR

Set of syntactic and semantic conventions that makes it possible to describe things. Bench-
Capon, 1990.
Knowledge representations schemes

Propositional calculus- assertions describing things, use logical connectives and boolean logic.
Discuss examples.
Advantages. Can reason about the world; based on proven theory.
Disadvantages. Components cannot be individually examined.
Predicate calculus- is an extension of propositional calculus. Predicates are used these are of the
form function(arguments), where function is any object or relationship. The quantifiers are used.
Discuss examples.
Advantages. It has well defined rules for manipulation; it is expressive.
Disadvantages. Cannot handle uncertainty; uses small primitives for descriptions whose numbers
can be many.
Rules
These are formalization of used to specify recommendations, give directives or strategy.
Format: IF <premises> THEN <conclusion>.
Related ideas: rules and fact base; conflict set - source of rules; conflict resolution- deciding on
rules to apply.
Discuss examples.
Advantages: easy to use; explanations are possible; capture heuristics; can handle uncertainties to
some extent.
Disadvantages: cannot cope with complex associated knowledge; they can grow to
unmanageable size.
29
Semantic networks
Semantic networks are graphical representation of entities and their relationships. The nodes are
objects or events; the arcs are the relationships or moves.
Discuss examples: body parts; sentences.
Advantages. Easy to translate to predicate calculus.

Disadvantages. Cannot handle quantifiers; nodes may have confusing roles or meanings;
searching may lead to combinatorial explosion; cannot express standard logical connectives; can
represent only binary or unary predicates.
Frames
These are knowledge representation formalisms in which stereotyped information on objects are
represented.
Features: capture object attributes and their values; search done by matching;
Structure: Node and collection of attributes(slots).
Discuss examples.
Advantages: can cope with missing values- close matches are presented.
Disadvantages: has been hard to implement, especially inheritance.
Exercises
1. Define terms related to knowledge representation.
2. Discuss the search process.
3. What is a problem?
4. Describe blind search.
5. Describe guided search.
6. Describe the knowledge representation formalisms giving their advantages and
disadvantages.
30
KNOWLEDGE ACQUISITION
Definition
Knowledge acquisition is the process of extracting knowledge (facts, procedures, rules) from
human experts, books, documents, sensors or computer files and converting it into a form that
can be stored and manipulated by the computer for purposes of problem solving.
THE KNOWLEDGE AQUISITION PROCESS
Identification
This is a stage in the problem is identified, its characteristics are examined and it is broken down
into smaller sub-problems. The participants are identified (experts, users, knowledge engineers).
The resources are identified or provided. The situation is studied.
Conceptualization
The best method of representing knowledge is identified and selected.
Formalization
The knowledge is extracted from the sources. Software and hardware issues are resolved.
Implementation
The program is written or a prototype is developed.
Testing
Test cases are used. The experts are consulted on acceptability of results.
METHODS OF KNOWLEDGE ELICITATION
Face to face interview with experts the experts are interviewed by knowledge engineers.
Protocol analysis this is a documentation of how the expert behaves and processes information
during problem solving. Usually the experts think aloud.
Observation the experts are observed at work.
Questionnaires these are questions that are sent to experts for responses.
Analysis of documented knowledge this is extraction of knowledge from such sources as books,
journals, articles, magazines, mass media materials.
31
Rule induction (computer aided knowledge acquisition) rule induction can be viewed as a
system that accepts examples and develops classification rules.
Examples Rule Induction New cases Rule Induction classification
Training Usage
Issues with knowledge acquisition

Machine representation is lower in form than human usage of knowledge;
Many participants are involved and they have varied backgrounds causing communication
challenges(Keng., Dom. Expert, System designers, Users, etc.).
Experts may not express their knowledge);
Mismatch between the way experts hold their knowledge and the way computers represent
knowledge.
EXERCISES
1. What is knowledge acquisition?
2. Describe the steps in knowledge acquisition.
3. Explain how you may acquire knowledge to build a system.
4. Discuss ways of acquiring knowledge.
5. Discuss problems associated with knowledge acquisition.
32
INFERENCE
Definition
Inference is the process of drawing a conclusion from given evidence. It may also be seen as
reaching a decision through reasoning.
Some inference strategies in Artificial Intelligence Applications
Reasoning by analogy- the proponents claim that this is a natural way humans handle problems.
The problematic situation is examined, then it is linked with other similar situations that are then
used to build a solution.
Example: Bone fracture; find someone with a fixed bone and get advice.
Special case: case-based reasoning- build a (problem, solution data bank) and use it to match
new problems and adapt solutions.
Formal reasoning: use rules, facts; predicate calculus; mathematical logic.
Generalization and abstraction: use sets; induction and deduction.
Procedural reasoning: use formulae and algorithms to solve problems.
Meta-level reasoning: use knowledge about what is known to solve problems.
Fuzzy logic: this is less precise and less logical method of reasoning. Fuzzy sets- sets whose
membership is probalistic;
Fuzzy phenomena include: rain; heap of sand; harvest; height; size; beauty.
Rule-based inference controls
Forward chaining this is an inference mechanism which begins with evidence(facts), then sets
out to draw a conclusion. The facts and rules are scanned and rules fire as they meet available
evidence.
Backward chaining- this is an inference mechanism which begins with the goal (conclusion) and
sets out to look for evidence to establish or disprove the goal.
Example
A: It is raining.
B: It is sunny.
C: It is cold.
D: Walk to work.
E: Carry umbrella.
33
F: Jump in a matatu.
R1: If A C then E.
R2: If B then D.
R3: If A C then F.
Problem
Use backward and forward chaining inference mechanisms to:
i. Advice a pupil on what to do on a raining day.
ii. Advice a pupil on what to do on a cold raining day.
EXERCISES
1. What is inference?
2. Is inference problem solving?
3. What are AI inference strategies?
4. Describe forward and backward chaining inference controls.
5. What does an inference engine do?
34
PATTERN RECOGNITION AND CLASSIFICATION
Everyday life consists of many occasions when we handle familiar occurrences at times
unawares. For instance when reading we quickly recognize the letters. We do so by classifying
the character object. Sometimes templates or stereotypes may be used to classify.
In this section we will examine how the process of classification can be mechanized by
examining the techniques used for pattern recognition.
Definitions
Pattern recognition. Pattern recognition system is a device that extracts features of an object then
classifies the object.
Feature. A feature is any measurement on input pattern so that it may be categorized. Usually
only useful features are categorized. Try to explain how A and H differ.
Classifier. A classifier is a system that takes inputs then places it into a category.
FEATURE VECTORS AND FEATURE SPACE
Feature vector. A feature vector is an measurement of features that make up the input pattern in
which each feature is unique.
Feature space. A feature space is an n-dimensional space created by the n-elements in the feature
space.
Example: Rugby players and Ballet dancers

Feature space- height, weight
Height
Weight
Legend: - Ballet dancer - Rugby player
35
DISCRIMINATION FUNCTION
Discrimination function is a function that maps input features into a classification space. They
are used in many pattern recognition techniques.
Decision boundary
Height
Weight
Decision boundary is a discriminating function that in above case is a straight line. Such a line,
that is used to separate classes is an example of linear classifiers.
CLASSIFICATION TECHNIQUES
There are two broad classes of classification techniques. These are numeric and non-numeric
methods.
The numeric methods are classification techniques that use measurements of geometric patterns.
They can be deterministic or statistical.
The non- numeric methods are classification techniques that that depend on symbolic processing
such as those dealing with fuzzy sets.
Our scope of treatment will only cover numeric-methods of classification include deterministic
and statistical techniques discussed below.
36
k-NEAREST NEIGHBOUR CLASSIFICATION
The nearest neighbour classification is one in which a new pattern is placed in the class to which
it is closest.
Height X
d1 d2
Class A Class B
Weight

d1 is the shortest distance to class A
d2 is the shortest distance to class B
X is the unclassified pattern
Consider the a discriminant function f(X), where f(X) = closest(Class A) closest (Class B).
If f(X) is positive then place in class B.
If f(X) is negative then place in class A.
In practice, all (k) members are taken into account. In this case we have k-nearest neighbour,
where k is the number of samples. A general problem is to find a reliable measure of distance
from one class of samples. There are several k-nearest neighbour methods that are discussed
below.
Hamming distance
Let X = (x1, x2, x3,, xn) and Y = (y1, y2, y3,, yn) be any two input vectors, then the
hamming distance between then denoted by H is given by:
n
H = |xi-yi|
1
The hamming distance is commonly used to compare binary vectors where it gives the number
of different bits. Note that |xi-yi| = xI XOR yI .
37
Euclidean distance
This is an accurate distance metric based on Pythagorass theorem. It is defined as given below.
Euclidean distance between then denoted by d(X,Y)euc is given by:
n
d(X,Y)euc = ( (x -y ) )
i I
2
X 1
Euclidean distance
Y
A special occurs when the input vectors are binary. In this case the Euclidean distance is the
square root of the Hamming distance.
The amount of computations are reduced by approximating Euclidean distance using the
following methods that are faster but less accurate.
City-block distance
X
City-block distance
Y
Let X = (x1, x2, x3,, xn) and Y = (y1, y2, y3,, yn) be any two input vectors, then the City-
block distance between them, denoted by d(X,Y)cb is given by:
n
d(X,Y)cb = |xi-yI|
1
No square roots are taken.
Square distance
Squre-block distance between them, denoted by d(X,Y)sq is given by:
38
d(X,Y)sq = max{ |xi-yI|},

where the maximum value of the differences( or the longest side) is taken.
FURTHER REMARKS ON LINEAR CLASSIFIERS
Linear classifiers occur many times. They depend on linear functions or straight line boundaries
that are used as decision boundaries. Sometimes a weight vector may be used to adjust the
decision boundary so that classification is correctly performed. This is depicted below.
Height linear decision boundary
Class A Class B
Weight vector
Weight

n
f(X) = wixI where X is the boundary, w is the weight.
1
For two classes A and B, one classification rule would be f(X) > 0 place in Class A,
and f(X) < 0 place in Class B.
Usually the challenge is to find w so that classification is correct, especially in nearest neigbour
methods.
Linear separable problems

These are problems for which it is possible to find a single linear decision boundary(line or
plane).
Several classes may be handled systematically such as: ABCD could be handled as A or BCD, B
or CD, then C or D.
Sometimes it may be hard to get class boundaries, in such a case the samples may be split as
below: class 1
class 3 class 2
39
class 4
Usually the nature of the problem determines which technique to use.
STATISICAL CLASSIFICATION METHODS
These are methods that use statistical techniques to classify. They are based on conditional
probability that is generalized in Bayes theorem.
Let G1, G2, G3, , Gn be possible classes or groups.
G2 .
G1
Gn
.
Let P(G) be the probability of a pattern X belonging to group Gi.
The conditional probability of event X given event Y has occurred is written as P(X|Y).
Statistical classification problem

Given a set of measurements, X, what is the probability that it belongs to calss Gi, in other words
what is the probability, P(Gi|X).
Bayes classifier
X belongs to class Gi if P(Gi|X) > P(Gj|X) , for all i=1, 2, 3, .., n, i j.
Usually models are assumed on the patterns of data distribution such as the normal distribution.
Bayes rule:
P(Gi|X) = P(X| Gi) P(Gi)

P(X|Gj) P(Gj)
j
P(Gi) is easy to get as this depends on the number of groups and assuming independence.
P(X| Gi) is obtained from the distribution being used such as the normal distribution or some
other probability distribution function.
40
Bayesian classifiers can be optimized to perform very well. It is a theorem that Bayesian
classifiers reduce to linear classifiers.
EXERCISES
1. What is pattern recognition?
2. What is classification?
3. Explain the meanings of the terms feature vector and feature space.
4. What is discrimination function?
5. Describe the nearest-neighbour classification technique.
6. What is statistical classification?
7. Discuss Bayesian classification technique.
8. Discuss an example showing how Bayes theorem may be used in a classification a problem.
41
EXPERT SYSTEMS
Towards a definition
Expert systems are computer systems that are best appreciated first by examining what the terms
mean.
The first term to consider is the term system. A system is an environmental entity that is
considered to have the characteristics that include: existence of a purpose or goal, composition
by several inter-related components that contribute individually to the its goal; and existence of
boundaries. In addition, a system may be adaptive, closed, open, static or dynamic.
The term system is used here in the sense of computer system or computer programs in
particular.
The word expert is derived from human experts. Expertise is a property of human beings
describing their abilities to cope with difficult problem solving situations. Human experts usually
solve problems in specified areas of expertise with appreciable level of competence. We meet
human experts as experienced doctors, educators, psychiatrics, engineers, craftsmen,
programmers, analysts, administrators, politicians, etc. Human experts have extensive task
specific knowledge acquired from training, reading, practice and experience. The knowledge
could include:
Theories about the problem area;
Rules and procedures regarding general problem area;
Rules (heuristics) on what to do in a given problem case;
Global strategies for solving typical problems;
Meta-knowledge(knowledge about knowledge);
Facts about the problem area.
Experts take a long time to develop and when they solve problems they may do so
in one or more of the following ways:
Recognize and formulate the problem;
Provide the solution to the problem fast;
Explain the solution to a problem;
Learn from experience;
Restructure knowledge;
Break rules;
Tell the relevance of situations or issues;
Recognize their limitations but still offer workable solutions(degrade gracefully).
Expert system is therefore a set of computer programs that mimic the human expert. The
programs will take and solve problems in restricted problem domains or areas of expertise.
42
Expert systems must therefore have knowledge similar to the ones held by human experts and
use it to solve problems of the type solved by human beings.
STRUCTURE OF AN EXPERT SYSTEM
KNOWLEDGE BASE
Facts: what is known about the problem domain

User area
Rules:logical reference such as between symptoms

and causes
User interface
Knowledge
Engineer
Recommended action Explanation facility

Documented
knowledge
INFERENCE ENGINE
.draws conclusion .interprets, .schedules, enforces
consistency checks Expert
Knowledge
Knowledge
Blackboard (Workspace) refinement
Plan Agenda
Solution Problem description
43
Adapted from Turban & Ronson [P. 447]
The development environment is used by the experts to build the components and enter the
knowledge into the expert system.
The consultation environment is used by the non-experts to obtain expert help or knowledge
from the expert system.
The knowledge acquisition subsystem assists in the accumulation, transfer and transformation
of problem solving expertise from experts or documented knowledge sources to a computer
program. Knowledge can be acquired from human experts, textbooks, multimedia documents,
databases, special research reports and the internet.
The knowledge base contains the knowledge needed for understanding, formulating and solving
problems. Knowledge will include facts in the problem area and the theory of the problem area.
Heuristics are rules that are used to solve some typical problems; they may also be kept.
Structures used to store knowledge are called knowledge representation formalisms and they
include rules, semantic networks, frames, propositions and predicates.
Inference engine is the brain of the expert system. It is sometimes called control structure or
rule interpreter. It provides a methodology and implementation of the reasoning mechanism that
uses the knowledge held in the knowledge base and the blackboard to formulate conclusions. It
gives a means of using the knowledge by developing the agenda that organizes and controls the
steps taken to solve problems during consultation. The parts of the inference engine consists of:
Interpreter (rule interpreter in many system) that executes selected agenda items by applying
relevant rules from the knowledge base.
Schedule that maintains control over the agenda. It uses priorities and other criteria to
estimate the effect of applying rules on the agenda.
Consistency enforcer maintains a consistent representation of the emerging
solution.
User interface the part that enables the users or builders to submit their items to
the expert system and also the expert system to respond to the users or builders.
Blackboard (workspace) is the area in working memory where current problems

are described, data is entered and intermediate results are kept. The blackboard
may contain a plan of how the problem should be solved, an agenda of potential
actions awaiting execution and a solutions indicating candidate hypotheses and
alternate courses of action.
44
Explanation subsystem (justifier) is the part that traces responsibility for

conclusions to their sources. It may explain:
Why some questions are asked by the expert system;
How some conclusions are reached;
Why some alternatives are rejected;
The plan used to reach the solution;
The remaining facts to establish before the final conclusion.
Knowledge refining system is the part that enables the analysis and use of
knowledge so that leaning may take place and improvements may be made.
PARTICIPANTS IN EXPERT SYSTEMS
The Expert is the person who has a special knowledge, judgement, experience or methods used
to solve problems or advise on a given class of problems.
The Knowledge engineer is the is the one who extracts and structures knowledge from sources
such as the human experts, so that it may be put into the computer for use in expert an system.
Many times he may be the expert systems builder also.
The User is someone who is involved with an expert system as a non-expert and the expert
system is an consultant providing advice or solutions to problems. The user may also be student
who wants to learn from the expert system which acts as an instructor. The user may be an expert
systems builder who adds knowledge to the knowledge base of the expert system which acts as a
partner. The user may also be a partner, who is someone who uses the expert system as an
assistant or colleague. In this case the expert system gives a second opinion.
CLASSIC PROBLEM AREAS ADDRESSED BY EXPERT SYTEMS
Expert systems have been used in several typical problem areas. Several application systems
have been demonstrated in these areas. These areas are given below.
Category Problem area addressed

Interpretation Infer situation descriptions from observations
Prediction Infer consequences of given situations
Diagnosis Infer malfunctions from observations
Design Configure objects under constraints
Planning Develop plans to achieve goals
Monitoring Compare observations and plans, flag exceptions
45
Debugging Prescribe remedies to malfunctions

Repair Execute plans to administer prescribed remedy
Instruction Diagnose, debug, and correct student performance
Control Interpret, predict, repair and monitor system behaviour
BENEFITS OF EXPERT SYSTEMS
Increased productivity and output since expert systems work faster than humans.
Decreased decision-making time as expert systems can make decisions faster.
Increased process and product quality as errors can be significantly reduced.
Reduced down time as expert systems (ES) where ES are used for diagnosis of malfunctions and
prescribing repairs.
Capture of scarce resource as ES can store the expertise held by humans who may be taking
long to train.
Flexibility as ES can sense changing needs and advice accordingly, such as a product out of
production.
Easier equipment operation where ES is used to operate a complex equipment.
Elimination of the need for expensive equipment where ES is used to perform

monitoring and control tasks with low-cost equipment.
Operation in harzadous environments where ES is used where humans are not safe
such as in nuclear power plants, toxic environments.
Accessibility to knowledge and help desks where ES is used to supply information

and scarce knowledge or support help desks.
Increased capability of other computerized system where ES integrate with other

systems to make applications work faster or produce higher quality results.
Integration of several experts opinions where ES are built by consulting several

experts.
46
Ability to work with incomplete or uncertain information where ES may use

existing knowledge to solve a problem even though some other facts may still be
missing or uncertain.
Provide training where ES is used by novices because of its contents in the

knowledge base.
Enhanced problem solving and decision making where ES integrates analysis and
judgement of top experts.
Improved decision-making processes where ES provide rapid feedback on

consequences.
Improved decision quality as ES is reliable and does not become bored, tired or
hold attitudes.
Ability to solve complex problems as ES can hold a massive amount of knowledge

and thus being able to handle complex problems.
Knowledge transfer to remote locations as ES can hold specialist knowledge that

may be used in remote locations such as eye disease diagnosis and treatment
system demonstrated in Algeria and Egypt by World Health Organization.
Enhancement of other CBIS as ES adds the necessary intelligence capabilities

needed in other CBIS.
LIMITATIONS OF EXPERT SYSTEMS
The following problems are associated with expert systems:

Knowledge is not always readily available;
Difficulty in extracting expertise from humans;
Variations in problem assessment by different experts;
Human experts cannot abstract when under pressure;
Cognitive limitations of users;
Only work well in a narrow domain;
Experts may not always validate their conclusions;
Experts may not always use understood vocabulary;
Knowledge engineers are few and expensive;
47
End-users may not trust expert system;

Subjectivity and biases in knowledge transfer.
SUCCESS FACTORS FOR EXPERT SYSTEMS
Implementation of expert systems may succeed because of the following factors:
Management support and user involvement;

There must be a high level of knowledge;
There must be at least one cooperative expert;
The problem must be qualitative (fuzzy);
The problem must be sufficiently narrow in scope;
There should be a good ES shell (store & manipulate knowledge naturally);
The should be a friendly user interface for novice users;
The problems must be difficult enough to warrant the use of ES;
There should be competent knowledge-based system developers;
The ES should positively impact on end-users.
Materials above are adapted from [3, pp. 439-463 ]
A METHODOLOGY FOR THE DEVELOPMENT OF EXPERT SYSTEM

FOR DECISION SUPPORT
Determine the problem scope. Find out the actions, outcomes and evidence to
consider.
Draw a conceptual diagram. Draw a diagram indicating relevant factors and which
ones influence the others. Identify the aspects that determine the utility of
outcomes.
Assign probabilities. Assign conditional probabilities.
48
Assign utilities. Utilities refer to values that indicate importance of actions.

Functions can be used for these.
Enter available evidence. Where some initial evidence exist in each case it should
be entered.
Evaluate the diagram. Calculate optimal action according to the available

evidence.
Obtain new evidence. Decide on the new evidence to acquire through appropriate
acquisition methods.
Perform sensitivity analysis. Check if the best decision depends on small changes
assigned to probabilities and utilities. Do so by varying the parameters and running
the evaluation again. Where small changes lead to different decisions then more
data should be collected.
EXERCISES
1. Describe the qualities and qualifications of a human expert.
2. Discuss expertise.
3. Discuss the definition of expert systems.
4. Describe the structure of a human expert.
5. Describe the people involved with expert systems.
6. Discuss the advantages and limitations of expert systems.
49
INTELLIGENT AGENTS (IA)

Intelligent agents must have been coined from the term agent. In conventional interpretation
agents are associated with personal representatives that interact with others to accomplish a task.
In computational sense, agents can be seen as programs that help the users with routine tasks.
However, several meanings to the term agent have been proposed as will be discussed below.
Definition of intelligent agent
Working definition of "agent"

An agent is a reusable software component that provides controlled access to (shared) services
and resources. Example: a printer agent that provides printing services schedules requests to a
shared printer.
Agents are the basic building blocks for applications, and applications are organized as networks
of collaborating agents. Example: a desktop agent "recruits" the services of a screen and a
connection agent to physically connect a call.
Other definitions of intelligent agent
Definition 1
".. sophisticated entity acting rationally and intentionally, either isolated or in cooperation with
other agents." [Dictionary for AI terminology]
Definition 2
".. software entity which functions continuously and autonomously in a particular environment,
often inhabited by other agents and processes."
Definition 3
An agent is a tuple <P,A,I,in,out> where P is a set of input objects (percepts), A is a set of output
objects (actions), I is a set of internal states, in is a function from P X I into I (the agent's input
function). We define the agent's environment to be a tuple <A,P,W,see,do> where A is the set of
outputs from the agent, P is the set of inputs to the agent, W is a set of states, see is a function
from W to P, and do is a function from A X W into W [Russel & Novig].
Definition 4
"Intelligent agents continuously perform three functions: perception of dynamic conditions in the
environment; action to affect conditions in the environment; and reasoning to interpret
perceptions, solve problems, draw inferences, and determine actions." [Hayes, 1995]
50
Agent and conventional programs
Common properties that make agents different from conventional programs:

Agents are autonomous, that is they act on behalf of the user.
Agents contain some level of intelligence, from fixed rules to learning engines that allow
them to adapt to changes in the environment.
Agents don't only act reactively, but sometimes also proactively(initiate actions).
Agents have social ability, that is they communicate with the user, the system, and other
agents as required.
Agents may also cooperate with other agents to carry out more complex tasks than they
themselves can handle.
Agents may move from one system to another to access remote resources or even to meet
other agents.
Components of agents
The following may be specified for a typical agent:
Owner. This is the user name, parent name or master agent name.
Author. This is the person or the process that created the agent.
Account. This is the owners account that may be used for billing or pointer to the account.
Goal. This is the statement of successful task.
Subject description. These are attributes of the goal.
Creation and duration. This is the request or response time expected.
Background. This is supporting information.
Intelligent system. This is the engine that runs the agents intelligence, it may consist of rule-
based expert system or neural network.
51
Characteristics of intelligent agents

Intelligent agents have several characteristics that are discussed below.
Autonomy. Agents act or decide on their own. Sometimes they may do this to circumvent
obstacles, or as they handle high level requests and seek more clarifications if necessary.
Background operation. Agents work in the background, usually out of sight, perhaps somewhere
in the cyberspace.
Singularity of task. Agents work on a single task.
Communication. Agents interact with other agents or humans.
Automation of repetitive tasks. Agents work on special repetitive tasks.
Support conditional processing. Agents may be rule-based systems thereby showing flexibility.
Learning. Agents can learn; this goes beyond rule-based systems.
Reactivity. Agents can perceive the environment and then respond.
Proactiveness. Agents can take initiatives such as inhibiting behavior instead of just acting in
response to environmental inputs.
Temporal continuity. Agents are continuously running processes.
Personality. Agents have a personality; they can interact with humans.
Mobility. Agents can move across different architectures.
The need for agents

The following issues underline the need for agents:
Information overload. The managers have so much information and they need some kind of help
to cope.
Massive bank of information over the years. A lot of information has been accumulated over the
years and there is need to analyze it and discover any other knowledge held.
Internet. The Internet requires search tools.
Service support. Service support is necessary in many areas including network security,
electronic commerce or employee support.
52
Simplification of distributed computing. Agents can act as intelligent resource managers.

Overcome user interface problems. Agents act as personal assistants adapting to the users.
Handle information service management problem. Agents can provide services, service
customization, monitor interacting features where systems are combined, enable using varied
terminals, enable resource sharing and selection, diagnose problems, bill electronic users and
provide security(firewalls).
TYPES OF AGENTS
Nwana's classification of Agents

1. Mobility: static or mobile;
2. Reasoning model: deliberative or reactive;
3. Ideal attributes: autonomy, learning and cooperation;
4. Role: information, management;
5. Hybrid: combination of the above.
Classifying agents from application point of view
Autonomous agents
Robotic agents Computational agents

Biological
agents
Artificial life
Software agents agents
Task specific Entertainment Viruses

agents agents
53
Classifying agents from organizational and personal view
Organizational agents. These are agents that work on behalf of businesses or computer
applications.
Personal agents. These are agents that help users surf the Internet. They work for the users that
create them.
Classifying agents from characteristics point of view
Nature of agency. Agents can collaborate or negotiate.
Intelligence. Agents have varying degrees of reasoning and learning.
Mobility. Agents have degrees of mobility across platforms.
Classifying agents by intelligence level and power
Level 0 agents. These are agents that follow orders to retrieve documents such as Netscape
Navigator.
Level 1 agents. These are agents that are initiated by users to get relevant Internet pages such as
Yahoo, Alta Vista, Inforseek.
Level 2 agents. These are agents that monitor information and notify of relevant information
such as WebWatcher.
Level 3 agents. These are agents that learn and examine user profiles.
Classifying agents by application area

There are various types of agents under this category including:
Workflow and administrative agents that assist in organizing work schedules or tasks.
Collaborative agents that work with other agents or people.
Desktop application support agents that help users of the desktop PCs.
Electronic commerce support agents that help users engage in electronic business.
54
Information support and management agents that help users retrieve, store, search or interpret
information.
Mail message support agents that help users dealing with electronic mails.
Network control and management agents that help users in controlling and managing the
network.
User interface creation agents are the agents that help users create interfaces or present them with
useable interfaces.
Operating system agents are those agents that help users of various operating systems get
services.
INTERNET BASED SOFTWARE AGENTS

There are various agents associated with the internet that are discussed below.
E- mail agents are those agents that help e-mail users.
Web browsing agents are those that guide the user browsing the Web.
Frequently Asked Questions Agents are those agents that answer routing similar questions.
Intelligent search agents are those agents that help users traverse that net to get detailed results
such as Inforseek, Web Crawler, Lycos, Excite.
Internet Softbot is an agent associated with finding out user needs.
Internet management and monitoring is a group of agents that monitor, diagnose problems,
conduct security or manage Internet resources such as WatchGuard.
ELECTRONIC COMMERCE AGENTS

These are agents that perform tasks related commerce and examples include: Bargain Finder;
Good Stuff Cheap(GSC). Some agents in this category help users locate what they are looking
for.
OTHER AGENTS
User interface agents- graphical; natural language parsing.
Operating system agents wizards; add user accounts; group management; file & folder
management; printer additions; add/remove progrmas; network-client administration such as
licensing and equipment installation.
55
Spreadsheet agents; Software development agents; Data-mining agents; Detect & alert agents;
Collaboration agents.
DISTRIBUTED ARTIFICIAL AGENTS (DAI)

These are agents that provide intelligent part of the distributed problem. In DAI the task is
subdivided and each subtask is solved or get handled by an agent.
MULTIPLE AGENT SYSTEMS

These are systems that provide environments in which there are several agents possibly with
different goals each of which probably negotiating their own ways to achieve their ends such as
in electronic commerce.
EXAMPLE OF COLLABORATIVE AGENT ORGANIZATION
USER 2 USER 3
USER 1
Goals and Results

Task specs
Interface agent 1 Interface agent 2 Interface agent 3
Task
Conflict resolution Task Proposed solution
Task agent 1
Information Task agent j
integration
Information
request reply
collaborative query processing

Information Information Information
agent 1 agent 2 agent 3
Query Answer
Information
Information Information Information Source 4
Source 1 Source 2 56 Source 3
Source: K. Sycara, Distributed Intelligent Agents, IEEE Expert 1996.
Simple Reflex agent architecture
These agents find rules that match the current conditions and take action.
environment
Sensors (what the world is

like now)
agent
Condition action rules
Effectors(action to take)
Algorithm
Function simple-agent(percept) returns action
Static: rules, a set of condition-action rules
State = interpret-input(percept)
Rule = match-rules(state, rules)
Action = determine-action(rule)
Return Ation
57
Reflex agent with internal state
environment Sensors
what the world is
like now
state
agent how world evolves
what actions to do
Condition action rules
action to take
Effectors
Algorithm
Function internal-agent(percept) returns action

Static: state, description of current world state
rules, a set of condition-action rules
State = update-state(state, percept)

Rule = match-rules(state, rules)
Action = determine-action(Rule)
State = update-state(state, action)
Return Ation
58
Goal agent
environment Sensors
what the world is
like now
state
agent how world evolves
action type
what actions to do
action to take
goals
Effectors
Algorithm is an exercise.
59
EXERCISES
1. What is an intelligent agent?

2. What is a distributed intelligent agent?
3. What is a multi-agent system?
4. Discuss why agents are necessary.
5. Discuss the components of an agent.
6. Discuss the classification of agents.
60
ROBOTICS
Definition
A robot is a programmable multifunction manipulator designed to move material parts, tools, or

specific devices through variable programmable motions for the performance of a variety of
tasks.
An alternative definition presents a robot as an active artificial agent whose environment is the
physical world.
Autonomous robots are those robots that make decisions on their own, guided by feedback from
physical sensors.
Nature of real world
Inaccessibility. Sensors can only perceive close stimuli.

Non-derterminism. Uncertainties exist.
Non episodic. Effects of actions vary continuously.
Continuous. All configurations and motions that are possible cannot be enumerated.
Suitable uses of robots
Manufacturing and handling materials. Robots can be used for repetitive manufacturing tasks.
Moving items. Mobile robots can be used to distribute mails; or be used as moving vehicles such
as underwater vehicles.
Hazardous environments. Robots can be used in dangerous situations such as nuclear disaster
zones, lunar exploration, nuclear plant maintenance, toxic waste clean up, deep sea exploration.
Tele-presence and virtual reality. Robots can be used to monitor things going on in distant places
or simulate reality through computer controls (virtual reality).
Augment human abilities. Robots can be used to duplicate lost limbs; they may also be used as
blind guides.
61
COMPONENTS OF ROBOTS
Robots consist of several parts. These parts are discussed below.
Effectors. These are tools for action usually by which the robot affects the environment under its
control.
Actuator. This is a part of an effector that converts software commands to physical motion such
as motors, hydraulic or pneumatic cylinders. They will determine the degree of freedom.
Locomotion. Locomotion involves the change of position of a robot within its environment using
effectors.
Manipulation. Manipulation involves moving other objects in the environment using effectors.
LOCOMOTION IN ROBOTS
Legs may be used in motion but it is not easy to construct legs to be used by robots.
There are different types of motion that are possible that include:-
Statically stable walker. This involves walking where the robot can pause and not lose balance.
Dynamically stable hopper. This is where the robot can walk and stay stable as long as it
hopping, if it suddenly stops then it loses its balance.
Non holonomic robots. These are robots that cannot exhaust total degrees of freedom of
movement available. For example: the car can only move in two dimensions while there are three
possible dimensions.
Holonomic robots. These are robots the exhaust all possible degrees of freedom in movement.
For example the plane can move in all the three dimensions.
MANIPULATION BY ROBOTS
Manipulators are effectors that move objects in the environment.

Manipulators can allow rotary motion where rotation is around a hub.
Manipulators can allow prismatic motion in which the motion is only linear such as piston
movements.
62
SENSORS IN ROBOTS
Sensors are tools for perception. There are different types of sensing that are discussed below.
Propriperception. This involves telling where the joints are.
Force sensing. This invoves sensing and controlling position of joints.
Tactile sensing. This involves touch sensing where elastic materials may be used to measure
distortion.
Sonar sensing. This involves using sound for navigation and range finding usually necessary in
detecting objects.
Camera data. Camera data is mainly used for vision.
Structured light sensors are those sensors that project light.
Laser range finders are depth finders.
Cross-beam sensor and parallel beam sensor determine object identity.
EXERCISES
1. What is a robot?
2. Why are robots important?
3. Describe the nature of the real world.
4. Discuss the uses of robots.
5. Describe the components of a robot.
6. Discuss locomotion, manipulation, sensing, and vision in robots.
63
MACHINE LEARNING SYSTEMS
In this section we will examine machine learning and its related terms. Unlike other AI systems,
machine learning had limited successes but useful demonstrations. Much of the work is still
under research studies. Learning will be considered with agents in mind.
Terminology in machine learning
Learning
Learning is the process by which an agent uses percepts to improve its ability to act in future. As
a process it takes place as the agent interacts with the world, and when the agent assesses its own
decision-making processes.
Learning element
Learning element is the part of the agent that is responsible for making improvements.
Performance element
Performance element is the part of an agent that selects external actions. Knowledge about
learning element and some feedback on how the agent is doing are used to determine how the
performance element should be modified to do better in future.
Critic
This is the part of the learning agent that tells the element how well the agent is doing. A fixed
standard of performance may be used. This measure should possibly be conceptually outside the
agent.
Problem generator
This is the part of the agent that suggests actions that may lead to new informative experiences.
Exploratory actions are suggested.
Model of a learning agent

Environment
Sensors
CRITIC
PERFORMANCE
Feedback ELEMENT
changes
LEARNING
ELEMENT knowledge future adjustments
PROBLEM
Learning goals
GENERATOR
Effectors
Adapted from Russel & Novig P.526

64
Supervised learning
Supervised learning is the learning situation in which both the inputs and outputs can be
perceived. Sometimes a friendly teacher can supply the outputs.
Reinforcement learning
Reinforcement learning is a type of learning situation in which the agent does not know the
outcomes but is given some form of feedback on evaluating its action. It is however not told the
correctness of its action.
Unsupervised learning
Unsupervised learning is a type of learning in which the no hint is given at all about the correct
input.
Example
Example is the pair (x, f(x)) where x is the input and f(x) is the output of the function applied to
x.
Hypothesis
Suppose (x, f(x)) is an example, then an hypothesis, h, is an approximation of the function f.
APPLICATIONS OF MACHINE LEARNING

The main aim of machine learning is to make computer systems that can learn. If machines learn
then their ability to solve problems will be enhanced considerably. In research learning has found
applications that are related to knowledge acquisition; planning and problem solving. There
some areas, that are side effects of research in Machine learning, that have seen intensive
research in recent times that include data mining. Specifically some of these applications include:
Where there are very many examples and we have no function to generate the outputs,
machine learning techniques can be used to allow the system to search for suitable functions(
hypotheses).
Where we have massive amount of data and hidden relationships, we can use machine
learning techniques to discover the relationships (data mining).
Sometimes machines cannot be built to do what is required due to some limitations, if
machines can learn then they can improve their performance.
Where too much knowledge is available such that it is impossible for man to cope with it,
then machines can be used to learn as much as possible.
Environments change over time, so machines can adapt instead of re-design new ones.
New knowledge is being discovered by humans, new vocabulary arise, new world events stream
in and therefore new AI systems should be re-designed. Instead of doing this, learning systems
may be built.
65
(These reasons come from: Nils, J. Nilsson(1996). Introduction to Machine Learning. Internet)
TECHNIQUES USED IN MACHINE LEARNING
Machine learning depends on several methods that include induction, examples, observations,
and neural networks.
Induction
Pure inductive inference problem seeks to find a hypothesis, h, that approximates the function, f,
given the example (x, f(x)). Consider a plot of points. The possible curves that can be joined
suggest various functions (hypotheses, h) that can approximate the original function. Where
there is preference to hypothesis to a given example beyond consistency, we say there is a bias.
Consider an agent that has a reflex learning element that updates global variable, examples, and
that it holds a list of pairs of (percept, action). When it is confronted with a percept and it is
looking for an action it first checks the list. If the percept is there then it applies the action,
otherwise it must formulate a hypothesis, h, that is used for selecting the action. If the agent
instead of applying a new hypothesis adjusts the old hypothesis, then we say incremental
learning occurs. The skeleton algorithms for a reflex learning agent are given below.
Global examples {}
Function reflex-performance-element(percept) returns an action
If (percept, a) in examples then return a
Else
H induce(examples) i.e find a hypotheis based on examples
Return H(percept)
Procedure reflext-learning-element (percept, action)

Inputs: percept, feedback percept
Action, feedback action
Examples Examples {(percept, action)}
We consider two inductive learning methods namely decision trees and version spaces.
66
Decision trees
In decision tree, the inputs are objects or situations described by a set of properties while outputs
are either yes or no decisions. Each node consists of a test to the value of one of the properties
and the branches from the nodes are labeled with possible values of test result. Each leaf
specifies the Boolean value if that leaf is reached. An example is given below:
Patrons?
None some full
WaitEstimate
No Yes
<60 30-40 10-30 0-10

No Alternate ? Hungry? Yes
no yes no yes
Reservation? Fri/Sat? Yes Alternate?
no yes no yes no yes
Yes No Yes
Bar?
Yes Raining?
No Yes
No
Yes No Yes
Yes
No
A decision tree for deciding on where to wait for a table
Attributes for the above decision tree are:

Alternate: availability of a suitable alternative;
Bar: presence of a comfortable bar area to wait in;
Fri/Sat: true on Fridays and Saturday;
Hungry: whether one is hungry;
Patrons: number of people present;
Price: price range-not important;
Raining: if it is raining outside;
Reservation: whether reservation has been made;
Type: nature of restaurant- not important;
WaitEstimate: the estimated time to wait.
The logical version is given by:
X Patrons(X,Full) WaitEstimate(X, 0-10) Hungry(X, No) WillWait(X).
[Russel & Novig, pp. 531-535]

67
Inducing decision trees from examples
Decision trees may also be obtained from examples. In this case, a table is used and the goal
conditions for yes are collected together. Trivially, just take the attributes and enumerate its
possible options.
The table is processed attribute by attribute and selecting the attribute that minimizes noise or
maximizes information. A typical example here is ID3 algorithm.
Applicant Annual Assets Age Dependants Decision

income
Okello 50,000 100,000 30 3 Yes
Kamoro 70,000 None 35 1 Yes
Mulei 40,000 None 33 2 No
Wanjiru 30,000 250,000 42 0 Yes
Turban &Aronson, p.507
Assets
available ?
Yes No
Annual Income
Yes >40,000
No
Yes
No
Yes
Logically: A has_assets(A) annual_income(A, >40,000) Approve_loan_for(A).
A decision tree learning algorithm (Russel & Novig, 537)
Function Decision-tree-learning (examples, attributes, default) returns a decision tree

Inputs: examples, set of examples
Attributes, set of attributes
Default, default value for the goal predicate
If examples is empty then return default

Else if all examples have the same classification then return the classification
Else if attributes is empty then return majority-value(examples)
Else
Best choose-attribute(attributes, examples)
Tree a new decision tree with root Best
For each value vi of Best do
Examplesi {elements of examples with best = vi}
Subtree decision-tree-learning(examplesi, attributes Best,
majority-value(examples))
68
add a branch to tree with label vi and subtree subtree.

End
Return Tree.
Two success reports of decision tree learning

BP deployed expert system GASOIL in 1986, for gas-oil separation for offshore platforms that
had about 2500 rules. The attributes included relative proportions of gas, oil, and water and the
flow rate, pressure, density, viscosity, temperature and susceptibility to waxing. The decision
tree learning methods were applied to a database of existing designs and the system was
developed in less time with the performance better than human experts, saving BP millions of
dollars (Russel and Novig, P539).
A program was written to fly the flight simulator, by observing real flights about 30 times. The
embedded flight simulator could now do better than human beings in that it made fewer
mistakes.
Versioning
Versioning is another inductive technique that we will outline. This technique depends on
Hypotheses which are candidate functions that may be used to estimate the actual functions. For
instance the example above where a decision tree was used for the determining whether a patron
will wait may have the following hypotheses:
P willwait(P) patrons(P,Some H1
Patrons(P,Full) Hungry(P) H2
WaitEstimate(P,0-10) H3
Hungry(P) Alternative(P) H4
:
Hn
Consider the hypothesis space { H1, H2, .. Hn}. The learning algorithm considers that one of the
hypothesis is correct, especially the disjunction of the hypotheses: H1, H2, .. Hn
Each of the hypothesis predicts a set of examples and this is called the extension of the
predicate.
False negative examples. These are examples that according to the hypothesis should be
negative but they are actually positive.
False positive examples. These are examples that according to the hypothesis should be positive
but they are actually negative.
69
The idea is to readjust the hypotheses so that the classifications are correct without false
placements. There are two approaches that are used to maintain logical consistency of
hypotheses.
Current-best hypothesis search

A single hypothesis is maintained and is adjusted as new examples are encountered. Where a
hypothesis has been working well and a false negative occurs then it must be extended to include
the example. This is called generalization. However, when the hypothesis has been working and
a false positive occurs, then it must be minimized or cut down to exclude the example. This is
called specialization. An algorithm is given below that describes the process:
Function current-best-learning(examples) returns hypothesis

H any hypothesis consistent with the first examples
For each remaining example in examples do
If e is false positive for H then
H choose a specialization of H consistent with examples
Else if e is false negative for H then
H choose a generalization of H consistent with examples
If no consistent specialization/generalization can be found then fail
End
Return H.
Least-commitment search
Another technique of finding a consistent hypothesis is to start with original disjunction of all
hypotheses: H1, H2, .. Hn It is original set that is reduced as some hypotheses that are not
consistent are dropped. If this method is applied then the final set that remains is called a version
space. Version space learning algorithm is given below:
Function version-space-learning (examples) returns a version space

Local variables: V, the version space- the set of all possible hypotheses
V the set of all hypotheses
For each example e in examples do
If V is not empty then V Version-space-update(V,e)
End
Return V
Function version-space-update(V,e) returns an updated version space

V {h V: h is consistent with e}
70
OTHER TECHNIQUES USED IN MACHINE LEARNING
Neural networks
Neural networks as was seen earlier model the human neuron. The neural network consists of a
number of nodes that are connected using links. Each link has a numeric weight that is associated
with it. The learning occurs by adjusting the weights so that the inputs correspond to the outputs.
A neural network unit consists of a linear input function that computes the sum of weighted
inputs. It also has a non linear component called activation function that transforms the final
input values into a final activation value.
Neural networks have been used in pronunciation in which text streams are mapped to phonemes
(basic sound elements; handwritten text recognition; driving (AlVIN(1993)-learnt how to steer a
vehicle by observing the human driver.).
Belief Networks
Probability trees are constructed and are used decision making. Learning involves adjusting the
conditional probabilities as examples are examined.
Reinforcement learning
This is a learning method in which the learner tries to find out the utility function and uses it to
select states that maximize the expected utility of outcomes. Q-learning occurs where the agent
learns the action-value function that gives expected utility of taking a given action.
EXERCISES
1. What is learning?
2. What is machine learning?
3. Define the terms performance element, critic, problem solver, supervised learning,
reinforcement learning, unsupervised learning, example, hypothesis.
4. Describe a model of a learning agent.
5. Discuss applications of Machine Learning.
6. Describe the techniques used in inductive learning.
7. Show how decision trees are used in learning.
8. Describe learning by versioning.
9. Investigate other areas of machine learning.
71
OVERVIEW OF NATURAL LANGUAGE PROCESSING

(Based on Russel & Novig Chapters 22 & 23)
When considering natural language processing we focus on agents that can communicate.
Communication. Communication can be taken to mean intentional exchange of information

through producing and perceiving signs drawn for a shared system of conventional signs.
Language. A language is a complex, structured system of signs used by human beings to

communicate most of what they know about the world.
Speech act. Speech act is producing language and may involve all forms of using language
including typing or sign language.
Purposes of speech act include:

Informing each other about the world;
Querying others about various things;
Answering questions;
Requesting or commanding others to do things;
Promising to do things;
Acknowledging requests or offers;
Sharing feelings or experiences.
Understanding speech act

Speech act can be understood when a set of inputs are associated with the part of the world that
would be responsible for creating them. Understanding there requires the knowledge of syntax
and semantics of a language. It may involve plan recognition which involves forming a model of
agents beliefs. It also involves logical reasoning which enables forming complex phrases by
combining simple ones.
Formal language. A formal language is a set of strings, each consisting of a sequence of symbols
taken from a definite set, usually called terminal symbols.
Phrase. A phrase is any sub-string of a given category, such as the noun phrase (NP) or the verb
phrase (VP).
Sentence (S). A sentence is a string consisting of sub-strings from different categories that are
allowed for example, S NP VP
Non terminal symbols. These are symbols such as S, NP and VP.
72
Components of communication
The situation is such that there is a speaker, say S, who wants to convey some message, say
proposition P to the hearer H, using the words W. Seven processes are identified that may take
place:
Speakers role:
Intention: S wants H to believe P (S is considered to believe P);

(S must reason about beliefs of H and decide that H should know about P)
Generation: S selects the words W (according to S, W express the meaning of P);

(Knowledge about language is used to decide what to utter)
Synthesis: S utters the words W (usually addressing them to H);

(In AI, phonetic alphabet may be used for string sounds)
Hearers role:
Perception: H perceives W (required W = W, ideal case);

(perceptions is speech recognition in case of speed inputs or optical
character recognition in case of character inputs)
Analysis: H infers that W has possible meaning P1, P2, ,,, Pn (since words and
phrases have several meanings).
(Two activities are involved: syntactic interpretation- or parsing and
semantic interpretation-extracting the meaning. Parsing involves grouping
the words into phrases that have categories. Pragmatic interpretation is a
type of semantic interpretation where the situation is taken into account.
A parse tree may be used in parsing. A parse tree has nodes representing
phrases, links representing applications of grammar rules, and leaf nodes
representing the words)
Disambiguation: H infers that S intended to convey Pi, Pi = P (may be incorrect);

(Is necessary because the utterances have several possible interpretations)
Incorporation: H decides to believe Pi (or reject it if it is inconsistent with current beliefs)
73
Example using the statement The cat is good.
Speaker:
Intention: know(H, good(cat))
Generation: The cat is good
Synthesis: dikatiskud
Hearer:
Perception: The cat is good
Analysis: S
NP VP
Article Noun Verb Adjective
The cat is good
Semantic interpretation: good(cat), well(cat)

Pragmatic interpretation: good(cat), caught(rat, cat)
Disambiguation: good(cat)
Incorporation: tell(KB, good(cat))
Two Agents that communicate with language

language
Percepts language
KB
Actions Percepts Actions
KB
REASONING REASONING
Agent A Agent B
74
Algorithm
Function simple-communicating-agent (percept) returns action

Static: KB, a knowledge base;
T, a counter, initially 0, indicating time
TELL(KB, MAKE-PERCEPT-SENTENCE(KB,t))
words SPEECH-PART(percepts)
semantics DISAMBIGUATION(SEMANTICS(PARSE(words)))
If TYPE[sentence] = command then
action CONTENTS[semantics]
Else if TYPE[semantics] = statement then
TELL(KB,CONTENTS[semantics])
action ASK(KB, MAKE-ACTION-QUERY(percept,t))
Else if TYPE[semantics] = none then
action ASK(KB, MAKE-ACTION-QUERY(percept,t))
description GENERATE-DESCRIPTION (percept)
Return COMPOUND-ACTION(SAY(description), DO(action))
OVERVIEW OF A FORMAL GRAMAMAR FOR A SUBSET OF ENGLISH
Language processing is only possible if some set of symbols are used and some form of grammar
constructed on them. This will need a few considerations that are indicated below.
Lexicon. Lexicon is a list of allowable vocabulary words. The words are usually grouped into a
set of categories or parts of speech such as nouns (names to denote things), pronouns, verbs (for
events), adjectives(modify nouns) and adverbs(modify verbs). Other categories include articles
such as the, prepositions such as in, and conjuctions such as and.
Grammar gives rules for forming sentences. The sentence (S), noun phrase(NP), verb phrase
(VP), preposition phrase (PP), and relative clause (RelClause) may be used.
Example of a lexicon
Noun breeze | glitter | nothing | cat | pit | rat | meat | ..
Verb is | see | feel | chase | hear | catch | hide | kill | eat | smell | go |.
Adjective right | left | back | noisy | good |

75
Adverb here | there | nearby | ahead | right | left | back | .
Pronoun me | you | I | it .
Name John, |Mary | Ochieng | Kimeo | Mutiso | Kamau | .
Article the | a | an | .
Preposition to | in | on | near | .
Conjunction and | or | but |
Digit 0 | 1 | 2 | 3 | 4 | 5 | .| 9
Example of grammar with above lexicon

Sample phrase
S NP VP I feel a breeze
| S conjunction S I feel a breeze and I smell a rat.
NP Pronoun I
| Noun pit
| Article Noun the rat
| Digit Digit 56
| NP PP the rat + to the east
| NP RelClause the rat + that is good
VP Verb Smells
| VP NP feel + a breeze
| VP Adjective is + good
| VP PP chase + to the east
| VP Adverb go + ahead
PP Preposition NP to + the east
RelClause that VP that + is good
76
A parsing algorithm (Russel & Novig- p666)
Function BOTTOM-UP-PARSE(words, grammar) return parse tree

forest words
loop do
If LENGTH(forest) = 1 and CATEGORY (forest[1]) = START(grammar) then
return (forest)
else
i choose from {1.. LENGTH(forest)}
rule choose from RULES(grammar)
n LENGTH(RULE-RHS(rule))
subsequence SUBSEQUENCE(forest, i, i+n-1)
if MATCH (subsequence, RULE-RHS(rule)) then
forest[i..i+n-1] [MAKE-NODE(RULE-LHS(rule),
subsequence)]
else fail
end
NB forest is a list of various allowed phrases.
PRACTICAL APPLICATIONS OF NATURAL LANGUAGE PROCESSING
Machine translation- translate from one natural language to another. This has been a subject of
research since early 1960.
Database access use of natural language to access information from databases. This has given
rise to text interpretation as an additional area of research.
Information retrieval retrieval of text documents, usually containing key words in the query.
Text categorization placing texts in various categories for example placing various news
articles into relevant categories.
Extracting data from a text converting data in a text into a database format in which key values
of parameters are obtained.
77
EXERCISES
1. Define the terms communication and language.
2. Describe the importance of speech acts.
3. Discuss the meaning of the terms: intention, generation, synthesis, perception, analysis,
disambiguation, incorporation.
4. Describe the component steps of communication.
5. Describe a model for two agents communicating in a language.
6. Discuss the important items to have in natural language processing.
7. What is a lexicon?
8. What is the role of grammar in natural language processing?
9. How can a statement in natural language be parsed?
10. Discuss practical applications of natural language processing.
11. Discuss aspects of machine vision.
BIBLIOGRAPHY
[1] Stuart J. Russell, Peter Norvig(1995). Artificial Intelligence: A modern
approach.
[2] Ivan Bratko (1990). PROLOG: Programming for artificial intelligence. Addison-Wesley.
[3] Wolfgang Kreutzer, Bruce McKenzie (1991). Programming for Artificial Intelligence.
Addison Wesley.
[4] The INTERNET.
[5] Ephraim Turban, Jay E. Aronson(1998).Decision Support Systems and Intelligent Systems.
Prentice-Hall, 5th Edition.
[6] Nils J. Nillson (1996). Introduction to Machine Learning.
78

Knowledge Based Systems

Hochgeladen von

Dokumentinformationen

Copyright

Verfügbare Formate

Dieses Dokument teilen

Dokument teilen oder einbetten

Freigabeoptionen

Stufen Sie dieses Dokument als nützlich ein?

Sind diese Inhalte unangemessen?

Copyright:

Verfügbare Formate

Knowledge Based Systems

Hochgeladen von

Copyright:

Verfügbare Formate

BCT 2403 Knowledge Based Systems

Andy Carvin(Traditional view)

General intelligence(Source web-site: http://www.bsu.edu/classes/huffman2/edpsy350/chapter8/)

Alfred Binet (General intelligence)

Multiple Intelligence Theory(- several types of intelligence-Howard Gardner)

Possess a clear developmental history

Gardner has identified eight intelligences as : Linguistic-Verbal; Logical-Mathematical; Musical

Currently highly valued by society

Sternberg's Theory of Triarchic Intelligence(Professor Robert Sternberg of Yale

II. Scope - stylistic variables:

Emotional intelligence(Daniel Goleman)

Summary on the definition of intelligence

DEFINING ARTIFICIAL INTELLIGENCE

TURING TEST FOR INTELLIGENCE (ACT HUMANLY)

The human interrogator thinks he/she is communicating with a human.

To pass Turing Test the computer must:

Total Turing test included vision & robotics.

Cognitive Sciences view of artificial intelligence()

Some related terms

FOUNDATIONS OF ARTIFICIAL INTELLIGENCE

Philosophy (428 BC to present)

Mathematics (800 - now)

Psychology (1879 - now)

Computer Engineering (1940 - now)

Lingusitics (1957 - present)

HISTORY OF ARTIFICIAL INTELLIGENCE

Gestation period (1943 - 1956)

Claud Shannon (1950), Alan Turing (1953) wrote chess programs.

Enthusiasm and great expectation (1952- 1969)

Arthur Samuel (1952-) wrote many programs for checkers.

John McCArthy (1958) defined LISP which is a dominant AI language.

Challenges- a dose of reality (1966- 1974)

Knowledge based systems- key to power? (1969- 1979)

In summary, the importance of knowledge was discovered.

AI becomes an industry (1980-1988)

By 1988, DECs AI group deployed over 40 expert systems.

Return to Neural Networks (1986 - now)

Rumelheart & McClelland (1986) distributed a collection of works in parallel distributed

Recent events (1987 - now)

State of the art

5. Discuss the events in the development of AI.

APPLICATION DOMAINS OF ARTIFICIAL INTELLIGENCE

SOME APPLICATIONS OF ARTIFICIAL INTELLIGENCE

In 1980s there were interests in semi-automated operation weapons, development of smart

Natural language. Work has been reported on automatic programming, composition of

PROBLEM SOLVING TECHNIQUES IN ARTIFICIAL INTELLIGENCE

Human neuron nucleus

A general genetic algorithm

SEARCHING AS A PROBLEM SOLVING TECHNIQUE

Search conditions include:

Representing search problems

Defining a search problem

Example of a search case study [Bratko, p.259]

Goal state (final state)

The state space has 13 elements or nodes.

General search algorithm

Function Gsearch (Problem, QueuingFn): Solution|Failure;

EXHAUSTIVE SEARCH STRATEGIES

The branch (child) nodes are visited first.

The arrows show the depth first search progression.

The arrows show the breadth first search progression

LIMITATIONS OF EXHAUSTIVE SEARCH METHODS

Sometimes the state space can be inexhaustible, in that it is difficult to enumerate.

BEST-FIRST SERARCH HEURISTIC