Peer mentoring in undergraduate clinical education of orthoptic students

Jean Pollock and Zoran Georgievski School of Orthoptics, Faculty of Health Sciences La Trobe University
Third year (final year) undergraduate students were invited to participate as mentors for first year undergraduate students in a clinical instrumentation subject taught within the School of Orthoptics, Faculty of Health Sciences, La Trobe University. 18 of a possible 22 students (82%) chose to participate as mentors. Mentors participated in the delivery of hands on practical demonstrations and supervision of mentees in their clinical, instrumentation class. First year students received their first semester of clinical instrumentation classes without mentors in order to compare the outcomes of the second semester mentored classes. Formative analyses indicated that mentees found the best aspects of studying this subject were the presence of the mentors (9/39: 23%) and the practical classes (20/39: 51%). 36 of 39 mentees (92%) would like peer mentoring to be structured into other subjects within the course. Mentors reported that the highlight of their involvement was that mentees learnt something from them (11/18: 61%) and that their own learning was reinforced (4/18: 22%). Further analyses of mentees responses indicated that mentors provided a worthwhile addition to their conventional learning environments despite that fact that statistical analysis indicated that there was no significant difference (t=0.98, df = 86, p=0.32), between the results of students final practical assessment in this subject in the mentored year compared with the previous non mentored year.

During 1998 a commitment to trial a peer mentored learning environment was made within the School of Orthoptics for undergraduate first year students. The impetus for this project resulted initially from a perceived need to seek ways to improve transition for first year students in order to improve retention rates. According to Jacobi (1991) this is a common use of mentoring to improve undergraduate academic success. Additional advantages of a more flexible learning environment were expected for both mentees, mentors and the subject tutor. Quality Assurance of Subjects (QAS) questionnaire responses over the previous 2 years (96, 97) had clearly indicated that first year students valued the clinical instrumentation classes in OR 180 Orthoptic Clinical Practice 1, as worthwhile. They had also expressed overwhelmingly that more practical time and/or more individual time with the tutor would be desirable. A mentored program was expected to enhance the ratio between the first year students and an available tutor (mentor), favouring more effective learning. It was also expected that first year students would feel less pressured by receiving instruction from a more experienced student rather than an academic staff member who often was perceived as 'the expert', and therefore a little unapproachable (Yates et al 1997). It was expected that mentees would be exposed to differences in competent clinical practice by their mentors, reflecting a range of possible clinical experience and that socialization to a professional role would influence retention rates. The benefits to mentors were expected to include enhancement of communication skills by gaining experience from leading small groups through a series of learning tasks and extension of their knowledge and understanding in a particular area. This would occur through their active review of the practical skills they would be demonstrating. Their

HERDSA Annual International Conference, Melbourne, 12-15 July 1999

Pollock and Georgievski

involvement in the development and implementation of the program in cooperation with academic staff was also expected to enhance their own desire to use staff actively as mentors. The benefits to the tutor were expected to be an opportunity to spend more time with individual students and the potential to develop less superficial relationships with all students. The preceding paragraphs highlight the wide range of potential benefits a peer mentored program was expected to deliver to all involved. Notwithstanding these expected benefits, attempts to operationally define peer mentoring have varied within different academic disciplines and programs (Jacobi 1991, Yates et al 1997). For the purposes of this study the peer mentoring program (PMP) was defined as: the development of a cooperative learning environment where more experienced undergraduate students provided academic and professional role models for less experienced students in demonstration and acquisition of specific clinical instrumentation skills. Method OR 180 Orthoptic Clinical Practice 1 was a first year core professional subject for all students. It ran all year, over two thirteen week semesters. The two components of the subject were i) a weekly one hour theory introduction to a specific piece of instrumentation used in the diagnosis and management of orthoptic patients, followed by a one hour practical class. ii) Attendance at two clinical placements during semester 2. The forty-six students enrolled in the subject were divided into two groups of twentythree each for the practical classes. The PMP began during semester 2. During semester 1 students received the theoretical class (1 hour) and the practical class (1 hour) from the tutor only. This enabled a comparison to be made within the same group of students of their perception of the benefits or otherwise of a PMP compared with a non PMP. Potential mentors (22 students) were invited to participate via an individual letter sent to their home addresses at the beginning of semester 1. Peer mentoring was described to them as:
... the two-way interaction between students of different levels of experience in order to promote and facilitate the acquisition and consolidation of skills and knowledge in a particular area.

The letter invited students to attend an introductory meeting to discuss the program. Students were also informed that participation in the program would provide them with an assessment bonus brought about by the restructure of an assessable component of their clinical subject. Their involvement earned them 10% toward their clinical 3rd year subject. Those choosing not to become involved understood that their 10% was earned during their clinical practice examination.

HERDSA Annual International Conference, Melbourne, 12-15 July 1999

Peer mentoring in undergraduate clinical education of orthoptic students

The introductory meeting was held at the School of Orthoptics during an evening outside normal academic hours. The aims of the meeting were to discuss the following three issues: 1. The impetus for development of the program: Difficulties with transition of first year students were cited as the major impetus. 2. An introduction and development of ideas about PM: Descriptions of examples of peer mentoring and its potential benefits were sought from students. 3. The consideration of implications for staff and students. Eighteen students (82%) agreed to become involved in the PMP. Descriptions were suggested for the timing (semester 2) of the program and the timetabling of students involvement. Each student was allocated to two separate OR 180 classes. This required their involvement for six class hours (theoretical class = 1 hour, practical class = 1 hour x 2 groups = 2 hours x 2 separate classes = 6 hours). Additionally, the students spent 2 hours at the introductory, recruitment meeting and a further 2 hours in a de briefing and final evaluation session at the end of semester 2. Their total time commitment was therefore 10 hours. Training As mentors were assigned to two specific classes, they knew which instruments and techniques they would be demonstrating prior to the class. All mentors had previously been enrolled in this same subject during their own first year of the course and would have used the techniques clinically with patients in their external clinical placements throughout second and third year. The only training requirement then was that they attend the one hour theoretical class presented by the tutor prior to the practical classes. Program Throughout semester 2, three mentors (on 6 occasions) and four mentors (on 6 occasions) joined the tutor to assist with demonstrations of clinical instrumentation techniques in the practical classes They acted as a resource with the tutor, in order to answer questions and assess the first year students techniques. Results Tools used QAS questionnaires Retention Rates t-Test

HERDSA Annual International Conference, Melbourne, 12-15 July 1999

Pollock and Georgievski

Semester 1 responses from mentees prior to PMP Tool used: standard Faculty approved Quality Assurance of Subjects (QAS) questionnaire (Appendix 1). In all, 28 of an enrolled 46 students (61%) responded to the questionnaire. Responses to all 13 questions were highly, positively rated. A mean of 82.54% of all responses were recorded from A True all of the time and B True most of the time. Semester 2 responses from mentees post PMP Tool used: standard Faculty approved QAS questionnaire with additional questions relating specifically to the PMP, (Appendix 2). In all, 39 of the enrolled 46 students (85%) responded to the questionnaire. Responses to the same 13 questions asked in semester 1 were rated at a mean of 68.43% of all responses from A True all of the time and B True most of the time. Responses to the five questions specifically related to the PMP were highly positively rated when combining the two most favourable responses: 71% considered the mentors had assisted their understanding of the material, 79% considered they had received considerable assistance with learning and acquiring practical skills, 82% felt that the presence of mentors enabled them to gain assistance more easily and more often, 67% said they would rather seek assistance from the mentor than the tutor, 67% thought that the PMP would still be worthwhile even if more staff tutors were available and finally, 79% felt the mentors provided good role models for what would be expected of them in two years time. 36 students (92%) would like PMP to be structured into other subjects in the course. Preceding non mentored year responses (all year) Tool used: standard Faculty approved QAS questionnaire (Appendix 1). In all, 34 out of an enrolled 42 students (81%) responded to the questionnaire. Responses to all 13 questions were highly, positively rated. A mean of 81.65% of all responses were recorded from A True all of the time and B True most of the time. Semester 2 written responses from mentors Tool used: evaluation form designed specifically to elicit responses from mentors (Appendix 3). All 18 mentors (100%) responded to the questionnaire. When asked to define their "role" as a peer mentor and explain what was involved on their part, they interpreted the question as either requiring an explanation of the particular piece of equipment they were working with or of the wider interactive experience with the mentees. For the role definition 11 students (61%) described the equipment while 6 students (33%) described a wider experience. Within these divisions, 7 students (39%) described transfer of knowledge as their role, 9 students (50%) described equipment demonstration and 1 student (6%) made no report. For definition of their involvement, 13 students (72%) described an explanation of the particular piece of equipment adding that this was an
HERDSA Annual International Conference, Melbourne, 12-15 July 1999 4

Peer mentoring in undergraduate clinical education of orthoptic students

exercise in knowledge transfer to the mentees. 1 student (6%), described their involvement as being one of general supervision, another student described reinforcement of previous knowledge and 3 students (17%) made no report. When asked to comment on what mentors had gained from the program, 13 (72%) said they had increased their understanding (of the use of the equipment), 3 (17%) had received a boost to their confidence, 3 (17%) had felt benefit from being relied upon in a position of responsibility, 3 (17%) had gained from getting to know the mentees, 2 (11%) had demonstrated how their own understanding had increased and 1 student (6%) reported an (unspecified) increase in their communication skills. Highlights of their involvement for 10 students (56%) were that the mentees had learnt something from them, for 3 students (17%) getting to know the students, for 2 (11%) increasing their own understanding, for a further 2 students (11%) boosting their confidence, for 1 student (6%) awareness of previously misunderstood material and for a further 1 student (6%) of being able to sell second hand resources to a mentee. When asked what mentors thought mentees gained from the program, 10 students (56%) considered that the increased ratio between mentees and the mentors was the most significant gain. 9 students (50%) thought mentees would be more comfortable asking another student for help and clarification than the tutor. 4 students (22%) described access to a variety of teaching styles, 4 students (22%) described increased understanding, 3 students (17%) described the information mentees could elicit regarding the course in general, employment and future opportunities. 1 student ( 6%) reported the benefit of having a familiar face to acknowledge outside of class. When asked what they thought the tutor gained from the PMP, 9 students (50%) thought that the increased ratio of "tutors" to mentees was the most significant gain. Five (28%) thought the tutor was less bombarded by questions from the whole cohort. Three students (17%), thought the tutor gained time for other administrative duties related to the subject. Three students (17%), felt the tutor could concentrate on students who needed him most. One student (6%) each made the following comments; knowledge of increased understanding and enjoyment of learning, better examination results, closer relationship and rapport with mentors, feedback on individual mentees progress and offering a students perspective on the level of explanation necessary to clarify a mentees confusion. When asked about perceived drawbacks of the program, mentors responses included the following: 11 students (61%) thought there were none, 2 students (11%) had missed a "very good" external clinical placement, 2 students (11%) were not able to develop strategies for dealing with mentees who were not interested to participate, 1 student (6%) would have liked to be involved for longer than a two week period, so that the mentees getting to know the mentors could be balanced against getting down to work and 1 student (6%) did not respond. When asked how the PMP could be improved 5 students (28%) suggested expanding the program to other subjects within the course. 4 students (22%) thought more mentors per class would be beneficial, 3 students (17%) thought mentors should be assigned to specific groups of students, 2 students (11%) could not think of any way of improving the program, while a further 2 students (11%) would like the program to extend throughout the whole year. 1 student (6%) each made the following comments; that
HERDSA Annual International Conference, Melbourne, 12-15 July 1999 5

Pollock and Georgievski

mentors choose the topic they were to teach, facilitate mentored relationships outside of class, no need for improvement of the program and no response was made from one student. Semester 2: transcribed verbal responses from mentors at the de briefing session All 18 mentors attended the final de briefing session at the end of the final week of semester. Although the PMP had been designed as an academic exercise, the mentors clearly saw a role for assisting the socialisation of mentees. They felt that an opportunity to consider expanding the program to increase social contacts could be worthwhile and suggested that a social event to introduce mentors and mentees would be desirable. It would seem that provision of a formalised structure enabled the interaction between mentees and mentors more successfully than just expecting it happen. This was highlighted by the mentors when they referred to a subject they were jointly enrolled in with 2nd year Orthoptic students. No mentoring programs had been included within this subject and little interaction occurred between students. They felt that their level of training was adequate, but that it would be more advantageous to them if they were able to individually choose the content area they would be mentoring in. They commented that they were initially lured by the marks that would accrue towards their own 3rd year clinical subject, but in retrospect would be happy to be involved in similar programs in the future without this incentive. They all felt that the 1st year students would have gained from the program and would view the interaction between the 3rd years and the tutor as an indication of a developing professional relationship. Two additional unplanned outcomes of the program were noted and perceived positively by mentors. These were occasions when 1st year students sought their mentor to clarify some issues outside of class time and one 3rd year student took one of their mentees to an external clinical placement. Retention rates From 1990 until 1999 the rates of students maintaining their enrolment in subsequent years of the course have ranged between 75.9% to 94.4% (mean = 84.2%). The retention rate for the group under study was 80.4%. Students who did not re enrol fell into three major categories: 1. deferred (one student), 2. failed substantial components of the whole course (4 students), and 3. passed the whole course, but transferred to other tertiary courses (4 students). The retention rate for the 1997 cohort was 92.8%. Students who did not re enrol fell into three major categories: 1. advised to defer for one year to improve English skills, chose to withdraw (1 student), 2. failed substantial components of the whole course (1 student), 3. illness prevented continuation (1 student).
HERDSA Annual International Conference, Melbourne, 12-15 July 1999 6

Peer mentoring in undergraduate clinical education of orthoptic students

Table 1: Comparison of assessment results between an unmentored (97) and a mentored (98) year
DF 86 Group: 97 98 Count: 42 46 Unpaired t Value: .982 Mean: 73.024% 70.089% Prob. (2-tail): .3291 Std. Dev.: Dev.: 11.775 15.683 Std. Error: 1.817 2.338

Discussion Despite receiving positive responses from mentors and mentees regarding their perceptions of the PMP, two of the primary seven goals of the project were not met. Retention rates dropped from 92.8% (97) to 80.4% (98). Possibly the authors had been excessively optimistic to believe that a PMP program in only one of the twelve subjects students studied, could make enough difference to ensure their continued enrolment. Further evidence of perhaps unrealistic expectations must be confronted when considering that one student had enrolled in this course specifically in order to transfer to another course within the faculty. The second major aim of the project was to enable more effective learning which was expected to occur by improving the ratio between mentees and their mentors. Ten mentors (56%) considered that the increased ratio was the greatest gain the PMP would have provided for mentees. However, the measurement of more effective learning by comparison of assessment results of the two cohorts did not show an improvement. Though no statistically significant difference was found between these; see Table 1 (t=0.98, df = 86, p=0.32.), the mean score for the 97 group was slightly better than for the 98 group. Yates et al (1997) made a similar finding when comparing a mentored with a non mentored group of nursing students. No discussion of their finding was made. When comparing the 97 with the 98 cohort the authors made some very broad assumptions and herein may lie some of the reasons for our findings. We did not ascertain whether in fact these cohorts were homogenous. It may have been possible that while the TER scores were similar between the cohorts, other notable differences could have existed. There may have been differences in the number of students who had English as a second language, the number of students who had previously begun or completed tertiary study elsewhere and the numbers of students who were the first within their family and cultural groups to attend a tertiary institution. Perhaps a more illuminating comparison could have been made between mean assessment scores for all subjects studied between the two cohorts. This may have exposed subtle improvements to results amongst the 98 group in this subject compared with other subjects studied. Student satisfaction as measured by the QAS questionnaires showed that more positive mean responses were made by the mentees during the first (non mentored) semester (82.54%), compared with the second (mentored) semester (68.43%), for similar
HERDSA Annual International Conference, Melbourne, 12-15 July 1999 7

Pollock and Georgievski

questions. However, despite this apparent higher level of satisfaction during the non mentored semester, analysis of the responses to the specifically designed questions regarding mentoring were higher (73.15%). Answers to the following questions would need to be clarified in order to interpret these findings: Was the content of the material studied in each semester of similar difficulty? Was the effect of the two external clinical placements (semester 2 only), perceived negatively or positively? Were there different demands in each semester with regard to assessments in other subjects? The remaining five goals were met to varying degrees. Certainly the prediction that mentors would be seen as more approachable than the tutor was supported (50%). Enhancement of communication skills was also reported (6%). An expectation to enhance the mentors desire to use staff as mentors was not measured directly. This particular cohort did however visit with and spend more time with the tutor than had been observed to occur in previous years (Pollock J, direct observation). Another factor that could have influenced this behaviour was the simultaneous development of a relatively unsupervised external clinical placement for 3rd year students. An opportunity for the tutor to spend more time with individual students was thought to have occurred (17%) as was the potential for the tutor to develop less superficial relationships with students (6%). Many additional benefits of a non academic nature were reported. These could be categorized as social by-products (Topping 1988, Yates et al 1977). These were reported from the mentors written and verbal responses at the de briefing session and included; the highlight of the program was getting to know the mentees (17%), gaining from getting to know the mentees (17%), information regarding the course in general, employment and future opportunities (17%), tutor able to develop a closer relationship with mentors (6%) and being able to sell second hand resources (6%). One of the limitations of this study was the choice not to use a control group that did not receive the PMP. However, Yates et al (1997) found that no significant difference existed between the nursing student groups receiving mentoring versus those not receiving mentoring, when analysed to compare a number of clinical competencies. A second limitation that may have affected the assessment results outcome was that no evaluation was undertaken of the mentors tutoring skills. It may also be important to consider a more structured training program for mentors (Goodlad & Hirst 1989), in order to attempt some recognized, shared goals. The need for this action was highlighted by the students who felt that they had no strategies for dealing with mentees who were not interested to participate (11%). The inclusion of the program into an existing timetabled subject enabled all to participate without undue complaint or drop-out. This is recommended by Goodlad & Hirst (1989) and Yates et al (1997). Conclusions Formative analyses using the QAS questionnaire responses provided detailed information about the students perceived gains and drawbacks of the PMP. Future PMP programs will therefore include these qualitative measurement tools. Comparison of retention rates as stand alone figures do not enable consideration of subtle issues that could not perhaps be influenced by any intervention programs (student illness and determined choices to enrol in other courses). In future therefore, numbers of students who fall into these types of categories will be selected out of the final analyses and presented separately for discussion. The use of a parametric statistical tool (t-test) is only valid if we assume a
HERDSA Annual International Conference, Melbourne, 12-15 July 1999 8

Peer mentoring in undergraduate clinical education of orthoptic students

normal population distribution. As mentioned previously, many assumptions were made about the homogeneity of the two cohorts and further measures should be used in future studies to validate these. Implications for the results for teaching and learning within the School of Orthoptics are manifold. The perceptions of students involved in the program, whether mentors or mentees were above average, were positive. For the future (1999), a further program is being designed for a subject that will see both first and second year students enrolled together. The subject involves teaching and learning of theoretical and practical components to both cohorts simultaneously. The subject Coordinator (Georgievski) is currently planning how mentoring will enable benefits to accrue between two groups who are separated by one year of experience within the course, but who will be learning together for this particular subject. References Goodlad S, & Hirst B, 1989 Peer tutoring: a guide to learning by teaching. Nichol Publishing, New York USA. Jacobi M, 1991 Mentoring and undergraduate academic success: a literature review. Review of Educational Research 61(4):505-532. Topping K, 1988 The peer tutoring handbook: promoting co-operative learning. Croom Helm, Kent UK. Yates P, Cunningham J, Moyle W & Wollin J, 1997 Peer mentorship in clinical education: outcomes of a pilot programme for first year students. Nurse Education Today 17:508-514.

HERDSA Annual International Conference, Melbourne, 12-15 July 1999

Appendix 1: SCHOOL OF ORTHOPTICS - Quality Assurance of Subjects Subject: OR180 Orthoptic Clinical Practice 1

Please circle the response which best describes your opinion of this subject. SDLP = Self directed learning packages
True all True most True half True little True none of the time of the time of the time of the time of the time

1. 2. 3. 4. 5. 6. 7.

The aims of this subject are clear. The subject is interesting. The subject is well organized. The workload is reasonable. I am able to understand the subject material. The student manual is useful. The lectures are clearly presented. The practical classes are clearly presented. The SDLP in the student manual are clearly presented.

A A A A A A A A A

B B B B B B B B B B B B B

C C C C C C C C C C C C C

D D D D D D D D D D D D D

E E E E E E E E E E E E E

8.

The lectures are worthwhile. A The practical classes are worthwhile. A The SDLP in the student manual are worthwhile.A Overall I am satisfied with this subject. What were the best things about this subject? A

9. 10.

........................................................... ........................................................... ........................................................... 11. How could this subject be improved? ........................................................... ........................................................... ........................................................... Thank you for your participation.

HERDSA Annual International Conference, Melbourne, 12-15 July 1999

10

Peer mentoring in undergraduate clinical education of orthoptic students

Appendix 2: SCHOOL OF ORTHOPTICS - Quality Assurance of Subjects Subject: OR180 Orthoptic Clinical Practice 1 Please circle the response which best describes your opinion of this subject. SDLP = Self directed learning packages
True all True most True half True little True none of the time of the time of the time of the time of the time

1. 2. 3. 4. 5. 6. 7.

Overall I am satisfied with this subject. The aims of this subject are clear. The subject is interesting. The subject is well organized. The workload is reasonable. I am able to understand the subject material.

A A A A A A

B B B B B B B B B B B B

C C C C C C C C C C C C

D D D D D D D D D D D D

E E E E E E E E E E E E

The lectures are clearly presented. A The practical classes are clearly presented. A The student manual SDLP are clearly presented.A The lectures are worthwhile. The practical classes are worthwhile. The student manual SDLP are worthwhile. On peer mentoring in the practical classes... The third years significantly assisted my understanding of the subject material. The third years significantly assisted my learning & acquisition of practical skills. The presence of the third years during the practical classes meant that I was able to gain assistance more readily & more often than would otherwise have been possible. I would prefer to seek assistance from the lecturer rather than a third year student. Peer mentoring would not be worthwhile if additional lecturing staff were available. A A A

8.

9.

A A

B B

C C

D D

E E

A A A

B B B B

C C C C

D D D D

E E E E

The third years provided good role models of what is expected of me in two years' time. A 10.

Would you like peer mentoring to be structured into other subjects in the course? YES NO (please circle one)
11

HERDSA Annual International Conference, Melbourne, 12-15 July 1999

Pollock and Georgievski

Appendix 3: SCHOOL OF ORTHOPTICS - Mentors Evaluation of the PMP 1998 1. How did you see your role as a peer mentor in the OR180 classes? What was involved on your part? Please explain as if to one of your classmates who elected not to participate in PMP. What, if anything, do you feel that you gain from being involved in the PMP? What was the highlight of your involvement in the PMP? What, if anything, do you feel that the 1st year students gained from having mentors in the practical class? What, if anything, do you feel that the tutor gained from having mentors in the classes? Do you feel that there were any drawbacks of the PMP? How can the PMP be improved and/or expanded?

2. 3. 4.

5. 6. 7.

HERDSA Annual International Conference, Melbourne, 12-15 July 1999

12