Education, training and knowledge base design

different modes of use applicable to knowledge domains suitable for the 15-18 age range. This approach was adopted to ensure that critical aspects of continued interactions were not masked by the complexity of the knowledge domain being represented. A major consequence of this approach has been that we have worked with a far wider range of ability than is normal with other Intelligent Tutoring Systems (ITS) studies. If knowledge based systems are to be of use in education and training then they must be able to cope with a broad spectrum of ability and, in some sense, typical knowledge domains. In the long term, if a coherent approach to knowledge base structure is to be developed, a single system may be able to adopt various strategies for different learning goals. This offers not just the opportunity to accommodate different learning styles but Abstract: KnowIed<?e based systenrs ofer considerable potentialfor the c,reation o new also the potential to elucidate the differences bef leurning environments. In education such environments can he used with the mediation of tween education and training, bridging the divide u teucher and limits to the interpretation oj the knowledge do not nrressurily present that arises between the two approaches.

problems. Intelligent tutorinR systems cmnot rely on human mediation and all interpretation of learner understanding is restricted to semantics of the knowledge iqn.esenrution. Thus the choice of representuIion is jiindarnental to the design of an intelli<genttutoring system. Experience fi-om an engineering context suggests that the success ($a few intelligent tutorin<? systems depends on the formal nature of the topics considered. Tvpicul topics f o r education and training require the integration ofa formal understundin~g with an injbr-ma1 interpretation of consideruble complexity. A new approach to the use o knowledxe based systems, firmly centred on experience of f applications uctive1.y used in eduution andlor training, will be required to solve this problem

2. Educational background: a comparison of Logo and knowledge based systems

C.P. Tompsett
School of Information Systems, Kingston Polytechnic, En,glund

In education, a knowledge based system provides a new technology for the manipulation of knowledge. The immediate benefits are similar to those noted for Logo, interactivity and extensibility as noted by Harvey [ I ] and meaningful names and interconnection of knowledge [2]. The value of Logo as a computer language for education (or 1. Introduction Legoas a toy for young children) is marked by the Knowledge based systems offer the potential to facility with which complex objects or activities create new learning environments for both edu- can be created out of simpler ones. The philosophy cation and training. The learner has the opportunity inherent in this approach identifies two key feato interact with a computer system that handles tures of the environment created. The first is that knowledge in the same (or intended) way as the the process followed by the student provides a conuser. Such systems may offer total freedom to the crete experience of the relationships and interuser to explore the knowledge and manipulate it as actions between the components. The second is they will. A more common strategy is for the sys- that the system provides rapid and evident feedtem to behave on occasions in a pro-active role, back to the student when errors are made, allowing guiding the learner towards an identified goal of in- the student to learn from their mistakes and avoidcreased knowledge. ing the implied criticism normally conveyed when The general aim of such systems is to separate the mistakes are identified by teachers or tutors and so knowledge represented within the system from the on as put forward by Papert 131. manner in which it is used or presented. This imSome advocates of Logo have promoted its use as plies that the knowledge is inspectable, that a user the basis of a radical educational development alcan seethe knowledge that has been recorded and most claiming that what students wish to do with observe the way in which different pieces of Logo should de facto be what they should do in knowledge are combined to reach various conclu- schools. Amore conventional approach has been to sions. suggest that Logo can provide an exploratory enSuch knowledge based systems have been under vironment in which a problem domain can be indevelopment at the Information Technology De- vestigated by the student. This model presupposes velopment Unit at Kingston College of Further that the student understands some problem definiEducation. Our concern has been to investigate tion and can determine whether a solution is correct

~~

214

Expert Systems, November 1988, Vnl. 5 ,N o . 4.

I
The student must be able to interpret a Real Problem into the model, understand the solution of the problem in the model and finally interpret the Model Solution in the real world. Too often, too much emphasis is placed on one aspect only, the move from Model Problem to Model Solution. Typical results include the inability of many children to apply more than a small part of their mathematical knowledge in the real world, a breakdown in one of the two interpretation steps. The radical view of Logo appears to propose that the only important real world problems are those that can be represented in Logo, a position rightly questioned by Sleeman in his defence of the ITS position [5] who suggests that the range of educational experiences for which Logo is applicable requires serious investigation. The microworlds approach requires that the first two steps (up to Model Solution) are relevant, but that the overall learning that takes place is not controllable by the designer, i.e. that the student alone can determine whether they re-organise their knowledge to assimilate this new experience. The approach researched at Edinburgh [4] required that specific learning did take place, i.e. that the third step had to be completed.

ence required by a student to create appropriate models themselves is in excess of the relative importance of the knowledge to be learnt. The use of a Logo environment in this way in education embodies two assumptions about the learning process. Firstly, the process is directed towards the transfer of specific knowledge - the environment is designed to provide a model of the knowledge that is to be learnt by the student. Such a system requires that the student already understands the current set of basic procedures and their relevance to the prob, lem being considered, the mapping between the

4. The use of knowledge bases in education

The use of a knowledge based system for learning closely parallels that of Logo or Lego. We seek to separate the knowledge from the general methods that are used to manipulate the knowledge. Problems are to be solved by the application of the general methods to the building blocks of our knowledge. In the same way that Logo shows its working to students in standardised ways a knowledge based system also provides appropriate methods for viewing the system during the solution of a problem. At the simplest level students are asked to act as knowledge engineers, encapsulating what they know already within the system and then exercising this knowledge to produce results that they also know as recommended by Briggs [ 6 ] . Success is measured by the ability of students to structure their own knowledge in such a way that the system can produce the correct answers. This assumes, on the model above, that the two interpretation processes are clearly understood, that the student is sble to identify correct solutions to the real world problem already, but is unsure of the method of 3olution. As a generic approach to education this view can

3. The use of models in learning

The concept of a model as part of the education process has been around for a long time. For a model to be useful there are three processes that must work to achieve new knowledge. Real Solution <=====> Model Solution .. .. ..

.. ..

.. ... *

Real Problem <=====> Model Problem

Expert Systems, November 1988, Lbl. 5 , No. 4.

275

be questioned in the same way as the adoption of the Logo as a philosophy of education. However, as with effective use of Logo, the underlying technology can still be effective in many circumstances. If the technology provides a concrete or visible model of activities that are otherwise abstract, then the potential value to education is considerable. The most significant contribution is that the concrete model is external andcan be described and discussed amongst students and between students and teachers. Within a teacher supported environment the failings of a system to model a problem to sufficient accuracy can provide the focus for further learning without the use of a computer. The technology for knowledge based systems has now developed to a reasonable maturity and it is time to make clear the claims that are made for it. As with Logo we must be careful to identify the particular approach we are using when we apply the technology. If we do not then we cannot evaluate the claims that are made.

claim has overtones of the Logic as a Computer Language for Children project at imperial College for which Kowalski states: the project aims both to apply logic to other subjects taught in school, as well as to teach logic as a subject in its own right. [7] The significant claim here is that a computer model of knowledge assists the student to organise their own knowledge coherently, that representing knowledge in logic or some other representation system elucidates the real structure of the subject being studied. This emphasises the initial translation process from Real Problem to Model Problem, reinforced by solving problems using the model. At the top level we might demand that the representation system used is psychologically valid that the model of knowledge reflects the cognitive model of knowledge in this domain that the student wilVshould possess after they acquire their new knowledge. This top level makes strong demands on the model of knowledge to be used, not least because it assumes that such a model exists. The potential advantages in such a representation are considerable, particularly if we extend consideration to include a full intelligent tutoring system with management of dialogue acts and intentions. However, for knowledge representation related to education and training there are grounds to doubt how far this approach can be pursued. An individuals psychological model of knowledge exists in the same way as a private language does, or does not, see Carnap [8] and Wittgenstein [S]. Private view of knowledge only has genuine meaning when it can be applied in public. We can question the concept of private knowledge within the context of education and training precisely because we are concerned to transfer a public quantity with correctness determined by public performance. The systems considered above have value in their contribution to the process of education and exploring the logical relatedness of knowledge in small domains. However, where such systems fit into the normal education or training process they are frequently seen as repositories of knowledge that have to be transferred, on some measurable scale to the learner. In this situation we are concerned that the student acquires a body of knowledge and is able to apply it within the same range of contexts as is defined by the knowledge base itself. This provides a context for knowledge based systems that parallels that of the Edinburgh workers using Logo. What is of importance is the building

5. The goals of knowledge representation

Educationally three different models of knowledge base use can be identified, each of which represents a distinct claim for the status of the form of knowledge representation adopted. Each of these has potential as an educational technology but also faces limitations. The lowest level of representation claims that we provide the student with a knowledge representation that is sufficient to model the problem (or problem domain) under discussion. The translation from students mind to computer representation must be a relatively straightforward process, in order for the students attention to be directed to manipulation of the knowledge rather than a translation process. A minimum requirement for this is that the representation is correct and that the syntax allows straightforward representation of what the student considers to be knowledge. Above this we can envisage a representation/ elicitation system that assists a student to represent their knowledge and clarifies their understanding of the key factors in the problem while they do so. Such a system must have a clear semantic structure, so that the student can readily identify the consequences of any step he/she takes in the representation process. With the optimum system, the process of knowledge representation should not only highlight areas of poorly understood knowledge but also offer a knowledge structure appropriate to clarification of the misunderstanding. This

276

Expert Systems, November 1988, Vol.5 ,No. 4.

and application of new concepts.

6. Formal and informal aspects of knowledge

The view of an expert system as a model of the world has been discussed elsewhere [ 101, [ 111. A more theoretical view has been proposed by Popselov [ 121, who describes an expert system in terms of a theory, developed in the usual way with a formal language, axioms, methods of deduction and theorems. In a knowledge based system the formal language is the language of the representation scheme, the known facts and rules of a domain are the axioms (with additional knowledge contributed by a user as additional axioms), the inference mechanism as applied to the representation system contributes the methods of deduction and the theorems are those results that can be derived by the system. Such a comparison is clear when considering a knowledge based system by any computerbased system for manipulating knowledge can be described equivalently. If the three-step process required to use the model is considered, it is important to distinguish between those aspects of knowledge that can be contained in the knowledge base, referred to here as formal knowledge, and the resulting knowledge that we wish to transfer, termed informal knowledge. Formal knowledge is knowledge that can be derived from the formal symbolic structure of the knowledge based system. The informal knowledge represents the literal knowledge that we are trying to convey. Within education and training there is a major cognitive gap between these two types of knowledge. There are five major distinctions that can be identified that interfere with the transfer of knowledge and it is an open question whether we can find solutions to any or all of them.

pendent and independent knowledge. Though the ability of the system to reach such conclusions is the essence of their design it promotes a view of knowledge that does not match the commonly held view. In common usage knowledge is primarily that which is true, or can be known. Unless challenged to justify statements the ordinary person or student will treat knowledge of a subject as the set of theorems of the system, the collection of true statements, and not the theory, that is without attaching significance to which statements are provable and which are axioms (ignoring for the present the known differences between these two sets of statements for sufficiently powerful systems mentioned by Godel [ 141). As an example, students who show difficulty in learning fractions do so because the way they know that:
I

/2

+ 14 = 3/4

is utterly different from the way in which they are asked to know that:
2 1 / 3 + /4= /12

6.1 Orderedn ess

Formal knowledge distinguishes between core knowledge such as facts and rules as axioms,and the set of all results that are shown to be true with the inference mechanism, the theorems. Although we would not normally consider: INFECTION- 1 is PRIMARY-BACTEREMIA <Item I > E. Coli [ORGANISM-I] <Item 2> Pseudomonas-Aeruginosa [ORGANISM-11 as a theorem [13], the system has still been applying the same methods as if it were solving a problem in geometry; it is a distinction between de-

In the first instance it is likely that the statement has a quasi definitional status for the fraction 3/4. 3/4 is the result of joining half of an object to a quarter of it. The second statement requires most of us to be able to add fractions generally; you do not even need to understand what a fraction is to have the concept of 1 14. Similarly, experience from an engineer training project [ 151, involving considerable discussion with tutors in the domain revealed that they drew few distinctions themselves between the statements that represented dependent and independent knowledge. In establishing the knowledge base it was requested that one statement be reiterated even though the statement was already known to be derivable from previous knowledge. This is perhaps a reflection of the consequence of representing the knowledge as a theory. There are many theorems that can be deduced from a theory. It may be more efficient just to give the statements that are valid and useful than to find some technique to eliminate true but useless theorems. 5.2 Reference Within formal systems, results are true in virtue of the symbols that are used. Thus we can use meaningless symbols without losing the power to jerive the same set of results. However this presJmes that the knowledge is applicable in an equally symbolic fashion, which is clearly not the case. It is

Expert Systems, November 1988, Vol. 5 , No. 4

277

a significant assumption, or task to be achieved within a knowledge based system, that the reference of each of the terms is understood. Early applications of expert systems to training, such as Guidon, acting as a case method tutor, have assumed that the user is Familiar with the terminology concerned and only aims to transfer the empirical knowledge of the expert [ 161. However, systems that have to introduce novel concepts face major problems. The work by Goldstein on the mathematical game Wumpus [ 171, is able to introduce a more general concept, such as danger, to aid tutoring about specific dangers - squeaks, smells or draughts. But the success of this activity depends very much on our ordinary understanding of a danger. We can identify squeaks, smells and draughts as examples of dangers already. If we could only understand the general concept of a danger as exemplified by one or two of the examples it is less likely that generalisation would be of any benefit (try understanding squeaks and dimghts are both squodges). A system to do this must have some method of linking in a more general and unstructured understanding of the domain to provide a focus for further tutoring. Introduction of new concepts can be emulated as a teaching activity by the use of canned sequences, as with conventional CBT, but this faces the same limitations. What is required is a better understanding of the development of knowledge and the methods by which new learning can be linked to previous knowledge. The KBET system used knowledge base techniques to describe both the domain and a videodisc of related material. Two approaches were possible to attack the problem of developing new concepts before treating the formal meaning as represented in the knowledge base. For certain students the initial student model was able to assume that parts of the knowledge base were understood. It was therefore possible to develop new knowledge by extension of current understanding already represented as a knowledge base. Alternatively the use of video allowed the demonstration of ideas and concepts by example without the need to use a taught vocabulary - i.e. teaching the concept first. Such video sequences contain a large quantity of information to associate the new knowledge with previous knowledge, relating even to the look (but not the feel) of the environment. The system has not yet been completed to a stage where such ideas can be tested, but it is known that abstract concepts will still present significant difficulties. An example from this project would be the concept o f machineability - describing the potential difficulty of machining a particular material. This con-

ept is employed in the design of the production irocess to identify aspects of the design that might ause problems if not attended to correctly. The Gdeo can demonstrate the effect of the concept but lever the concept itself. i.3 Structuring The basis on which people use and manipulate their nformal knowledge takes advantage of a variety of ,tructuring mechanisms most, if not all of which ire consistent; that is, they do not lead to incorrect mowledge. Formal knowledge depends on the fornal knowledge representation that is used. The :oded knowledge is a representation structure with he semantics (inherent meaning) limited by the nteraction of the inference mechanism and the dructure. Efficient manipulation of the knowledge lomain is enhanced by the extent to which the repaesentation system is or is not structured. Thus jtructuring provides a restriction on the way in which the knowledge will be manipulated. This joes not necessarily reduce the efficacy of the system for teaching but it may imply that once the knowledge is understood that multiple restricted views of the knowledge must also be developed. Examples of these multiple views arose on the KBET project during analysis of multiple choice test materials prepared for a small portion of the domain. These responses illustrated a wide variety of modes, ranging from errors in the application of knowledge (as anticipated) to errors generated through word association (implying lack of reasoned support for the answer) but also including questions requiring meta-level understanding to discriminate between expressions such as must/ may and must have all of/must have one of. More significantly, other issues arose in the construction of the knowledge base using a structure based on KLONE [18]. (The same problems will arise with any project using a conceptual framework.) This system makes extensive use of conceptual hierarchies but in practice, for the purpose of teaching, these hierarchies reflected both intrinsic and extrinsic definitions of concepts. Thus some super-concepts were merely collections of sub-concepts and thus had properties defined by the set of members (extrinsic definition); as an example the concept of an end-mill was best defined by considering it to be defined by the set of tools that are called end-mills by tooling manufacturers. Others had a clear definition to which the sub-concepts had to conform (intrinsic definition). Examples of this latter case would be the concept of a measure (with sub-concepts angle, length, area, etc.) or cutting tool (with sub-concepts mill-

278

Expert Systems, November 1988, Vol.5 ,N o . 4

ing-tool, reamer, etc.). Within the representation system this distinction was of course lost.

6.4 Meta-level intepretation

Formal knowledge representations are typically flat structures, representing knowledge at a single object level. Some knowledge based systems used in training offer some meta-level shucturing (or model based structure) for standard problem domains, such as fault-finding or diagnosis [ 191. However normal human problem solving behaviour is able to treat knowledge on a variety of levels and exploit this for different purposes. Informal knowledge systems provide a large amount of information to focus in on knowledge that is relevant to solving a problem and that which is not. Similar to the distinction between extrinsic and intrinsic concepts above, some concepts represented meta-level constructs that could only be included as organisational, rather than structuring concepts. Thus certain geometric features of a tool were critical angles relating to cutting performance. These were described by the experts as cutting angles, and were clearly a subset of the concept angle (see above). However there is no object level statement that distinguishes between cutting angles and other angles. The distinguishing feature is a metalevel construct that these angles affect the cutting performance to a significant degree and this is determined by inspecting the knowledge relating to such decisions.

knowledge: however, if there is no generalisation, the underlying knowledge based system will be able to make little use of the knowledge. Similarly French [20] requires that each rule in his rule-based system is accompanied by a textual justification of that rule. In all these systems there is a separation between those steps of reasoning that are justified by the formal semantics and those that are justified by the formal semantics and those that are only supported as obviously true or justified. With informal knowledge there is no equivalent boundary to the knowledge that is used. In fact, if we were to consider other domains such as law, it is the ability to identify the weakness in a valid chain of reasoning that separates the top professionals from the newly qualified.

7 Conclusions .
The differences outlined above seem to suggest that there are significant problems in the development of an intelligent tutoring system that can teach new concepts effectively. These differences are focused on the distinction between formal and informal knowledge. It is perhaps a reflection of the emphasis on formal domains (programming, geometry, arithmetic) that such issues have not been raised earlier. Although this appears to suggest that formal domains are still a suitable area for research, I am concerned that, from the educational

6.5 Boundedness
A corollary to the formal nature of the inference

system used and the lack of reference within the formal representation is the restricted nature of the domain itself. Thus the justification of any result is dependent on the correctness of the inference mechanism and the accuracy of the rules/knowledge in the representation structures used. Such issues have been raised by Clancey [ 191 with respect to the justification of various compiled rules; he uses the example of the tetracycline rule as a therapy for young children. His approach is to seek a model for all contra-indication rules and to exclude the particular knowledge required to support the intermediate steps in: tetracycline ==> teeth discolouration ==> undesirable body change An alternative view would be to exclude inference steps that will not allow generalisation of

point of view, the difficulties in teaching arithmetic arise from the lack of reinforcement rather than lack of skill. Students are unable to relate what they have learnt to the real world and therefore have no needto obtain correct answers [21]. Even outside the realm of Intelligent Tutoring Systems, knowledge based systems have been discussed but, till recently the tools for work in schools and colleges or universities have not been widely enough available. The time has now come for the claims for such systems to be written down and tested. This requires sufficient input from those who will use them in the end, the learners and the teachers, and not just those who develop them.

8. References
[ 11 B. Harvey, Why Logo, in M. Yazdani (Ed),

New Horizons in Educational Technology, Ellis Horwood, 1984. [2] B. Lawler, Designing Computer Based Micro-worlds, in M. Yazdani (Ed), New Horizons in Educational Technology, Ellis Honvood, 1984.

Expert Systems, November 1988, Vol. 5 ,No. 4.

279

Ex-pert Systems, p. 177, Addison Wesley, S. Papert, Mindstorms -Children, Compu1984. ters and Powerful Ideas, Harvester, 1980. J. Howe, P. Ross, K. Johnson and R. Inglis, [ 141 K. Godel, Uber formale unentscheidbare S2tze der Principia Mathematica und verModel Building, Mathematics and Logo, in wanter Systeme I, Monatschufte fur MatheM. Yazdani (Ed), New Horizons in EducamatikundPhysik38,1931, pp. 173-198. tional Technology, Ellis Honvood, 1984. D. Sleeman, A1 and Education: Two Ideo- [ 151 Knowledge-Based Engineering Training, ALVEY Project, IKBS 094. Project partners logical Positions, AISB Quarterly, No 55. were Logica (Cambridge) Ltd, Imperial ColJ. Briggs, Expert Systems in the Classroom, lege London, Exeter University, Kingston Prolog in Educution Group Confetmce ReCollege of Further Education, and the Enginport, Exeter 1986. eering Industry Training Board. R. Kowalski, Logic as Computer Language for Children, in M. Yazdani (Ed), New Hori- [ 161 W.J. Clancey, Use of Mycin Rules for Tutoring, in Buchanan and Shortliffe (Eds) Rule zons in Educational Technology, Ellis HorBased Expert Systems, Addison Wesley, wood, 1984. 1984. R. Carnap, Foundations of the The Unity of I.P. Goldstein, The Genetic Graph, in D. Science: towards an international encycloSleeman and J.S. Brown (Eds), Intelligent Tupedia of unified science, University of Chitoring, Academic Press, 1982. cago Press (published in sixteen parts 1938R. Brachman,On the Epistemological Status 1968), p.79. of Semantic Networks in Associative NetL. Wittgenstein, Philosophical Investigaworks, in N. Findler (Ed), Associative Nettions, Blackwell, 1972, p.258. ~~orks-repr.esentution and use of knowledgc [ 101 W.J. Clancey, The Knowledge Engineer as by computers, Academic Press, 1979. Student - Metacognitive Bases for asking Good Questions, Report N o . KSL-87-12, [ 191 W.J. Clancey, The Epistemology of Rulebased Expert Systems - A Framework for Stanford University, 1987. Explanation, Art$cial Intelligence 20, [ 1 I j W.J. Clancey, Viewing Knowledge Bases as pp.2 15-25 1. Qualitative Models, Report N o . KSL-86-27, Stanford University, 1986. 1201 Private communication concerning design of ATR Trainer knowledge based training D. Popselov, Semiotic Models in Psyshell. Paul French is the Director of Advanced chology and A1 Systems, Human und ArtfiTraining Research, Kingston Polytechnic. ciul Intelligenc~e,in F. Klix (Ed), North Holland, 1979. 1211 W.H. Cockroft et al, Mathematics Counts, para.45 1, HMSO, 1982. B. Buchman and E. Shortliffe, Rule Based

About the Author

Chris Tompsett
Chris Tompsett is a Senior Lecturer in the School of Information Systems at Kingston Polytechnic. His current post involves the establishment of industrial links supporting companies in the application of A1 techniques through provision of training courses and collaborative research and development. This paper is based on his wide experience as Manager and/or Director of intelligent knowledge based applications in both education and training. This included work as Technical Director for the application side of the ALVEY KBET project described above. His specific research interests in AI are in the linked fields of Intelligent Tutoring Systems and Knowledge Elicitation. He is currently the Manager for a project funded by the Training Commission evaluating the effect of student modelling on trainee learning times.

Expert Systems, November 1988, Vol. 5. No. 3 .