Evaluation of General Nursing and Maternal-Child Health Nursing Student Competencies in Mozambique

May 2013

OBJECTIVES
The primary objective is to determine whether the new curriculum for general nurses (GNs) and for maternal-child health (MCH) nurses resulted in an increase in their respective competencies among students at the time of graduation, as compared to the old curriculum. The secondary objective is to identify specific areas of poor performance for students graduating from programs using the new curriculum, in order to inform any necessary revisions or enhancements to this new program.

RESEARCH QUESTION
The purpose of this evaluation is to determine whether there are differences in the competencies of general nursing and maternal-child health nursing students trained under the old curriculum compared to those trained under the revised training curriculum at the time of graduation.

METHODS
This evaluation will use a cross-sectional approach to evaluate and compare knowledge and skills of two groups of students graduating from different training curricula. Differences in mean scores on the competency exam will be assessed as well as differences in the overall pass rate for each combined group.

EXPECTED DIFFERENCE
Measure Average score % of students who pass Estimated for Comparison Group 45% 40% Estimated for Intervention Group 67.5% 60%

PARTICIPANTS
The participants include: General nursing students attending programs where the old curriculum is used and graduating in December 2012. MCH nursing students attending programs where the old curriculum is used and graduating in 2012. General nursing students attending programs where the revised competency-based curriculum is used and graduating in July 2013. MCH nursing students attending programs where the revised competency-based curriculum is used and graduating in July 2013.
6

STUDY PROCEDURES
The Objective Structured Clinical Exam (OSCE) consists of stations the participants will go through. The stations relating to knowledge will be grouped so that participants can respond to written questions, test type in others stations they will demonstrate skills in a simulated environment with anatomic models. Dana: I dont understand text I made red There will be a set of OSCE stations for each group (general nursing students and MCH nursing students) and participants will rotate only through the stations aimed at their group.

ANALYSIS PLAN
Basic descriptive statistics of student OSCE results and pass rates will be generated for each group of students. Data on admission scores, academic performance and percentage of part-time and full-time teachers who received support to improve training methods will be collected. To evaluate the difference in student performance under the two curricula, the final exam scores within each cadre will be analyzed using a general linear model multilevel analysis with one factor (type of curriculum), nesting students within a school under a type of curriculum, and adjusting a schools performance by a surrogate measure of past performance.
10

ANALYSIS PLAN
A limitation to the analysis is that faculty and curriculum enhancements each may have contributed to benefits that cannot be differentiated in this study.

11

CONSENT PROCESS AND DOCUMENTATION

The study team consists of members with research experience and training in research ethics using the guidelines field of the Johns Hopkins Bloomberg School of Public Health (JHSPH), adapted for this study. Dana: Not sure what guidelines field is Before beginning, the data collectors will obtain informed consent, ensuring confidentiality to the participant. The data collectors will have a script that they use to approach each potential participant. The consent process will be conducted in Portuguese. The study team will work with the school to identify some private locations where they will follow the informed consent process.
12

FIRST PHASE EVALUATION OF CURRICULUM BASED IN CONTENT

13

 After the approval of the Research Protocol by the National Bioethics Committee last November, the assessment of the skills of students with the curriculum based on content (old curriculum) was carried out December 37, 2012, in previously selected training institutions (TIs) of Beira, Nampula, Pemba, Inhambane, Maputo and Chicumbane. We evaluated a total of 329 students (242 GN and 87 MCH nurses) using the OSCE methodology. For that purpose, 5 evaluation teams were constituted, 3 for General Nursing and 2 for Nursing of SMI, with 9 members of their respective technical areas. Define SMI?

14

 The teams were properly trained in the OSCE evaluation process in a specific training held at Avenida Hotel, Maputo City, from November 1216, in which each of the members became acquainted with their respective functions for the evaluation process; also, the validation of the assessment tools was simultaneously made.

15

ORGANIZATION OF THE STATIONS

Orientation of the station sequencing

Station name Material Patient name

Patient

Making a Simple and Dry Dressing Station

EVALUATION PROCESS
Evaluator

Student doing the technique

Supervisor

Nasogastric Intubation Station

EVALUATION PROCESS
Students doing the techniques of CPR, IV catheterization and IM injection

PRELIMINARY RESULTS
This section provides unweighted results from both GN and MCH nursing OSCE and written assessments, allowing for a coarse glimpse of student performance. Equal weight was attributed to all stations and to all assessment questions. Subsequently, calculations will place weights on these to produce additional results.

HEALTH TRAINING INSTITUTIONS OF MOH WITH OLD CURRICULA COURSES

HTC Tete

HTC Pemba

HSI Nampula

HSI Beira

HTC Inhambane

HTC Chicumbane HIS Maputo 20

HEALTH TRAINING INSTITUTIONS OF MOH WITH GENERAL NURSING OLD CURRICULA COURSES
HSI Nampula

HSI Beira

HTC Inhambane HTC Chicumbane HIS Maputo

21

Evaluation of Student Competencies 1st Phase General Nursing Station Results from 5 Training Institutions
100 90 80 70 60 50 40 30 20 10 0

1st Hygienic Hand Washing

2nd Measurement of Vital Signs

3rd Insertion of IV Catheter (IV)

4th 5th Intramuscular Vesical Injection (IM) Catheter in Women

6th Nasogastric Intubation

7th CPR

8th Making a Simple Dressing

9th PTIC

 The main difficulties are observed in performing Cardiopulmonary Resuscitation, Vesical Female Catheterization, Inserting an Intravenous Catheter and Making a Simple Dressing. The institutions located in the South (Maputo, Inhambane and Chicumbane) showed better performance in carrying out tasks at the stations, approaching the 80% mark. They also had a uniform but not satisfactory yield in the theoretical modules, being surpassed by TI Beira, which had the highest score. The average obtained in the stations was 70.5%; the average score for theoretical modules was 54.5%. The global average was 64.1% , with TI Nampula having the lowest performance, at 52.8%.
23

HEALTH TRAINING INSTITUTIONS OF MOH WITH MCH NURSING OLD CURRICULA COURSES
HTC Pemba

HSI Nampula

HTC Inhambane

24

Evaluation of Student Competencies 1st Phase MCH Nursing Station Results from 3 Training Institutions
100 90 80 70 60 50 40 30 20 10 0
1st Physical Exam Prenatal Control 2nd Care in Normal Birth 3rd Active Management 3rd Stage 4th Newborn Resuscitation 5th Manual Removal of Placenta 6th Management of Convulsions Eclampsia 7th 8th Management IMCI - Case of of Measles Hypovolemic Shock 9th Technique of BCG Vaccine

 The students graduating from health training centers had better performance in practice than the students from the Institutes: the fewest number of classes and students in the training centers as well as the existence of more internship sites for clinical practices could have allowed more learning opportunities in comparison with the situation for Nampula students. The teachers of the former classes are the same who were trained for the development of lessons for the competencybased curriculum; thus it is likely that they will use the same teaching materials for both competency- and content-based classes.

26

SECOND PHASE EVALUATION OF CURRICULUM BASED IN COMPETENCIES

27

 The assessment of the skills of students with the curricula based on competencies (new curriculum) will be carry on July 2013 in the TIs of Maputo, Beira, Inhambane, Chimoio e Tete for General Nursing student evaluation, and TIs of Beira, Chicumbane, Inhambane, Chimoio and Lichinga for MCH nursing students. The sample include 225 GN students and 201 MCH nursing students (total of 426 students).

28

HEALTH TRAINING INSTITUTIONS OF MOH WITH GENERAL NURSING CURRICULA BASED IN COMPETENCIES COURSES
HSI Nampula

Training Institutions HSI Maputo HSI Beira HSI Nampula HTC Inhambane HTC Chimoio HTC Tete Total

HTC Chimoio

HSI Beira

HTC Inhambane

Class and Number of Students Class 11 31 Students Class 2 26 Students Class 3 27 Students Class 4 34 Students Class 5 31 Students Class 6 27 Students Class 7 27 Students Class 8 22 Students 225 Students

HSI Maputo

29

HEALTH TRAINING INSTITUITIONS OF MOH WITH MCH NURSING CURRICULA BASED IN COMPETENCIES COURSES
HTC Lichinga

Training Institutions HSI Beira HTC Chicumbane HTC Inhambane HTC Chimoio HTC Lichinga Total

HTC Chimoio

HSI Beira

HTC Inhambane HTC Chicumbane

Class and Number of Students Class 1 29 Students Class 2 27 Students Class 3 27 Students Class 4 30 Students Class 5 28 Students Class 6 29 Students Class 7 31 Students 201 Students

30

HEALTH TRAINING INSTITUTIONS OF MOH WITH GENERAL AND MCH NURSING CURRICULA BASED IN COMPETENCIES COURSES
HTC Lichinga HTC Pemba

HTC Tete

HSI Nampula

HTC Chimoio HSI Beira

HTC Mocuba HSI Quelimane

HTC Nhamatanda HTC Massinga HTC Inhambane HTC Chicumbane HSI Maputo

Currently there are 33 MCH nurse cohorts (total: 1,061 students) and 44 General nurse cohorts (total: 1,341 students) running in 13 TIs with the support of Jhpiego.

31

THANK YOU FOR YOUR ATTENTION

32