Can we assess user education in the library; and if so, how?

Clive Wilson, Thames Valley University, UK (1997)

Line (1983) said that user education was 'one of the less sensible terms invented by librarians' because it makes absolutely no sense out of context. Unless one is a librarian it is not at all immediately obvious what is being referred to. Furthermore, Line (1983) suggests that the term is 'meaningless, inaccurate, pretentious and patronising' and that if only librarians would spend the time and effort to ensure that their libraries are more user friendly then they wouldn't have to spend so much time doing user education. This is at least partly unfair. Students have to use indexing and abstracting sources and computer catalogues, and Line (1983) himself said that if these are mystifying to the student it may well be the fault of the designer or publisher and not of the librarian. Combine this with the many different ways that indexes have been designed and the situation is reached where a student cannot proceed without a good deal of perseverance and practice. And so we have a need for user education, and most librarians and teachers would say that it is a good and necessary thing (Rathore, 1992). Put simply, and despite Maurice Line's above remarks, user education means educating the patron, whether student, staff, or member of the public, in how to use the library and its services. It is generally agreed that there are three levels of user education which have been summarised by Rathore (1992): 1. The library orientation is to be given at the beginning of every academic year or semester ... it should be applicable to all those who are using the library for the first time. 2. The second stage, i.e. subject oriented instruction for undergraduates at a stage when they are admitted to a special branch or subject of their choice or at the time of project work. 3. Literature search training ... should be provided at the beginning of their research work.' The literature search is specified as being for post-graduate students but with students developing as self-guided independent learners there is no reason why this should be the case. Depending on the student, the course, and the assignment there may be some overlap between the three stages. Knowing which level of user education is required by a particular group of students enables the librarian to determine the aims and objectives of the session. The three main aims of user education regardless of level are:

1. to train the user to exploit the library resources effectively. 2. to provide the user with the skills for independent information seeking. 3. to encourage the user to seek the assistance of library professionals. From these can be drawn the objectives of each session. The literature on teaching would have us state these explicitly as, ' At the end of this session the student will be able to ...' (Gibbs and Habeshaw, 1989). These objectives are seldom, if ever, expressed in such a manner in libraries and user education programmes. We do have objectives though. There would be little point otherwise. But on academic courses, what is the prime reason for having objectives? So that the teacher and student both know what is expected of them by the end of the class/course. And how does the teacher and student know that the objectives have been reached? By assessment. Assessment. Now there's a word not often used by librarians in the context of user education and when they do, it is usually being used to mean evaluation. Both Stevenson (1977) and Reid (1983) have highlighted how often librarians change their user education programme to try and find a better way of doing things. Here at Thames Valley University (TVU) we have tried three different approaches to orientation in the last three years. Librarians change their methods because of a perception that the current way is not successful. These perceptions are often reached through an evaluation of the user education programme. This may well be a valid way of approaching the problem but surely a better way to judge the success or failure of a programme is on how well the students perform. How do we measure how well students perform? We assess them. In Reid's (1983) case above, one of the methods used was to divide the user education session into two lectures. This is more pedagogically sound in theory but it proved to be less effective because of the poor levels of attendance at the second lecture. Why should this happen? The answer is related to the question 'Why do students do anything?' and as Race (1995) said, it is because they are assessed. Far more often than not, user education is not assessed. Because students are not assessed, because attendance at a user education class does not directly carry a mark which will be added to their degree, students do not always turn up. Or if they do, they may not pay full attention and forget very quickly what they have been shown. The question becomes, should user education be assessed and if so, how? It is certainly possible that a traditional exam could be the means of assessment and this has been done (Adams, 1983), but in these instances the user education has to be a required discipline within a course, or an extra credit course perhaps. In either case the student is motivated to do the exam because the assessment counts. In most cases in the UK therefore, this method will not work because user education is not required by many courses.

At TVU several of the Academic Liaison Librarians use worksheets for their user education sessions. These are sometimes used tutorial style, i.e. hand it in at your next class, but they are often used within the session to be completed and handed in by the end of the session. Usually practical based, the assessment exercise will require a number of tasks to be performed before the end of the session. This practical exercise means there is less stress on the memory of the student as they will 'discover' the answer, and it will test their retrieval skills which is the main objective here. The lecturer's co-operation and backing is essential, however, to ensure that students take the exercise seriously. The variety of tasks allows students to perform them in any order and ensures that resourcing is not a problem because it requires them to rotate around the resources. Answers are usually expected to be short because the emphasis is on the retrieval of the information. Depending on the formality of the session and the size of the group, the exercise can be done individually or in groups. Groups have the main disadvantage that the answer may be coming almost entirely from one or two of the brightest students in the group. If the group is well-balanced, however, and this is likely to be accidental rather than design, the students will help each other facilitating more effective learning. Feedback on these assignments is very important as the skills used will be beneficial throughout a student's course, if not beyond. It is possible for the same question to receive different answers depending on the source used and the students should be made aware of this. Where the assignment is formally assessed, the feedback is probably the most important result of the exercise. Ideas of where to find the right, or at least suggested, answer should be given, which enable the student to use the exercise as a referral tool during research for future assignments. This may be fine where the student experience happens over time. Marks and feedback can be given the following week, and the feedback can be implemented in the students personal learning strategy. We have a number of very short courses where this is not possible but we still want to assure ourselves that some library research skills are being developed. In some of these, the students have a morning or an afternoon specifically devoted to the Learning Resource Centre (LRC). When this happens we are able to require the students to do presentations. A short introduction to the LRC will be given, databases that may be useful will have been booked in advance, the students will be put into groups and told what their task is, e.g. researching a particular company. The groups then have to research for an hour and come back prepared to do a presentation. Because we are not interested in the company per se but in the process of information retrieval, the presentation should only focus half on what company information has been found. The other half should explain how the students went about their research. This enables marks to be given for division of labour, use of particular sources for particular information, asking at the enquiry desk, etc. Feedback on their information gathering in this context is again the most important element to give them more confidence in their local or workplace library. Presentations are considered traumatic (Race, 1996) and this is compounded by the fact that students have such a short time in which to prepare. However, many of the students on these short courses are expected to do

presentations in their workplace and so will either be practised at them, or it will be good practice for them. These are some of the ways in which we assess students and the rationale behind the assessment method. Some of the methods of assessment traditionally available are not going to work in the current environment at TVU. Exams, already mentioned, will not work unless the assessment counts, an issue that this paper stresses several times. Vivas, reports, and essays are simply not practical in the LRC due to the nature of our contact with students. Even if they were assessed, they are too time consuming for both the student and librarian given the relative importance of the assessment to the course. Forms of practical work are used as discussed and these are very useful but I am of the opinion that the best method of assessment for user education would be the portfolio, or workbook. For this to work, it would have to count as part of a course, or the extra credit scenario, because a workbook would take time to complete and the student would need an incentive to do it properly (Brevik, 1982). The workbook allows students to work at their own pace rather than trying to cram the development of skills which is a fault that our current methods suffer from. A wide variety of skills can be tested and the student can always call for assistance at the library enquiry desk (all librarians being fully briefed on the workbook so that they know to direct students rather than spoonfeed). Once completed and marked, the workbook also forms a useful tool for reference for the student and is much more complete than the one hour practical assessment. They may not use a particular database for a term or semester but the workbook may be sufficient to refresh their minds when they return to it, rather than relying on librarian assistance. The main drawback of this form of assessment is that it is very time intensive for the marker especially when assessment is not strictly part of the current role of many Academic Liaison Librarians. It is unfortunate but true, that at TVU we use the assessment of user education primarily as an evaluative exercise for the user education itself. E.g., which questions were consistently not answered or answered incorrectly? Was that the fault of the question or of the style of teaching. This is of course extremely useful in helping us develop our user education programmes to meet the needs of the students. We cannot however guarantee a true and fair view of students' performance because the work does not count towards some final mark. Studies have shown (Hardesty, 1979), by using two groups of students and pre-testing them to determine their initial level of knowledge, that user education does improve library performance. What isn't measured is whether students' coursework benefits as a result, which is the commonly held belief, or if students fall back on old methods of working (Dolphin, 1990). For user education to be its most effective, it should be assessed and it should be formally assessed. For this to happen user education should be integrated into the curriculum whether it is as part of current courses or as an extra credit course.

Carson and Miller (1984) give a working definition of what constitutes integrated user education. It has three essential elements. First, the user education should take place as part of subject specific classes. This would include an assignment involving use of information sources in the library. Secondly, because it is part of subject classes, librarians and teaching staff must work very closely together to ensure the quality and focus of the class. Full cooperation by teaching staff is essential. Thirdly, the main dose of instruction should be given in lectures although this does not preclude the student from seeking further help in the library. By making user education part of existing courses it could be assessed by including an element in a standard piece of coursework which meant that evidence of their research gained them marks, e.g. producing a full bibliography of sources searched during the assignment, whether they proved useful or not, as separate from the bibliography for the assignment itself. This would be marked by librarians and would lead to a piece of work that the students could see in the context of a real assignment rather than as an abstract piece of work. This would lead to deeper learning of the research skills (Morris, 1990). It combines the element of practical assessment, i.e. the students actually have to do the research, within the context of the essay. As the student has to do the research anyway in order to do the essay it should not take a significantly greater amount of work to produce an extended bibliography. This may be an advantage to this form of assessment but I would still choose the workbook as a preference because of its reference facility. Assessing user education, although it happens in isolated cases in the UK, is still very much the US model where it is integrated into the curriculum. If integration is such a beneficial concept why is it not more widespread in the UK? There are a few reasons. The first is due to attitudes. It has been suggested that librarians do user education in order to gain recognised academic status (Lester, 1979). This again is the American model where librarians in academic libraries generally are considered to be members of faculty. Secondly, a lot of time is devoted to preparing teaching materials for each lecture and there is a considerable amount of time to be spent marking. It may be argued that each subject area would only need significant preparation in the first year, creating the course, followed by minor updates in subsequent years, but few libraries have the resources to be able to spend the development time on those courses. Finally, although teaching staff are generally complimentary about the benefits of user education, would they still be as receptive if they had to make room on their course for it, possibly at the expense of one of their own topics? I, personally, am fortunate at TVU in that one of my subject responsibilities includes the BA Accounting module Personal Skills Development. This includes a one hour lecture to each group on facilities within the LRC and also has a tutorial exercise on use of the LRC. It is not formally assessed although it does encourage use of the LRC at an early stage in the course. This is, however, an exception rather than the norm here at TVU and I think we are still a long way away from the benefits that formal assessment of user education would bring.

Bibliography Adams, M (1983). Data gathering instruments. In: Evaluating bibliographic instruction: a handbook. Chicago, Association of College and Research Libraries, 1983. Brevik, P (1982). Planning the library instruction program. Chicago, ALA, 1982. Carlson, D and Miller, R (1984). Librarians and teaching faculty: partners in bibliographic instruction. College and research libraries 45(6), 483-491. Dolphin, P (1990). Evaluation of user education programmes. In: User education in academic libraries. Ed. Fleming, H. London, LA, 1990. Gibbs, G and Habeshaw, T (1989). Preparing to teach: an introduction to effective teaching in higher education. Melksham, Cromwell press, 1989. Hardesty, L (1979). Evaluating library use instruction. College and research libraries 40, 309-317 Lester, R (1979). Why educate the library user? Aslib proceedings 31(8), 366-386. Line, M (1983). Thoughts of a non-user, non-educator. In: Third International Conference on Library User Education. University of Edinburgh 19-22 July 1983. Eds. Peter Fox, Ian Malley. Infuse, 1983. Morris, B (1983). User education: time for a rethink. In: Third International Conference on Library User Education. University of Edinburgh 19-22 July 1983. Eds. Peter Fox, Ian Malley. Infuse, 1983. Morris, B (1990). Integrated user education. In: User education in academic libraries. Ed. Fleming, H. London, LA, 1990. Nash, V (1983). Workbooks in library skills - palliative, panacea or placebo? In: Third International Conference on Library User Education. University of Edinburgh 19-22 July 1983. Eds. Peter Fox, Ian Malley. Infuse, 1983. Race, P (1995). The art of assessing 1. New academic. Autumn 1995, 3-6. Race, P (1996). The art of assessing. New academic. Spring 1996, 3-6. Rathore, J (1992). User education programmes in academic libraries. Lucknow librarian, 24(3) 1992, 104-107 Reid, B (1983). Playing a game to introduce new undergraduates to the library. In: Third International Conference on Library User Education. University of Edinburgh 19-22 July 1983. Eds. Peter Fox, Ian Malley. Infuse, 1983.

Stevenson, M (1977). Education of users of libraries and information services. Journal of documentation 33(1), 1977, 53-78.